Dual camera stream

Abstract

Image enhancement effects are provided on an apparatus including a display and a camera device. A first image frame stream captured by a camera device is received, and an augmented reality effect is applied to the first image frame stream to generate an augmented image frame stream. A stream of enhanced image frames is displayed on a display in real-time. A second image frame stream corresponding to the first image frame stream is concurrently saved to the initial video file. A second image frame stream may then be retrieved from the initial video file and an augmented reality effect applied to the second image frame stream independently of the first image frame stream.

Description

Background Art

[0001] Social networking and messaging applications provide a vehicle for sharing user content such as photos or videos. In some instances, photos or videos may be supplemented with augmented reality or other effects that are generated live on the camera feed and displayed on the mobile device's display for preview. The user may be able to select and manipulate the effects to be applied to the live camera feed, and when satisfied with the results, capture an image or record a video including the effects. The captured video or photo may then be shared on a social networking platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] In the accompanying drawings (which are not necessarily drawn to scale), similar reference numerals may describe similar components in different views. To easily identify the discussion of any particular element or action, the highest digit or digits in the reference numeral refer to the figure number in which the element is first introduced. Some non-limiting examples are shown in the accompanying drawings, in which:

[0003] Figure 1 is a diagrammatic representation of a networking environment in which the present disclosure may be deployed, according to some examples.

[0004] Figure 2 is a diagrammatic representation of a messaging system having both client-side and server-side functionality according to some examples.

[0005] Figure 3 Recording and display process flows and playback process flows for AR-augmented video in a single-camera stream implementation according to some examples are shown.

[0006] Figure 4 A recording and display process flow for AR-enhanced video in a dual camera stream implementation according to some examples is shown.

[0007] Figure 5 A playback / re-recording process flow for AR-augmented video in a dual-camera stream implementation is shown according to some examples.

[0008] Figure 6 A playback / re-recording process flow for AR-augmented video in a dual-camera stream implementation is shown according to some examples.

[0009] Figure 7 A playback / re-recording process flow for AR-augmented video in a dual-camera stream implementation is shown according to some examples.

[0010] Figure 8An architecture for AR-enhanced video in a dual camera stream implementation according to some examples is shown.

[0011] Fig. 9 is a block diagram illustrating a software architecture within which examples may be implemented.

[0012] Fig.10 is a diagrammatic representation of a machine in the form of a computer system according to some examples within which a set of instructions may be executed for causing the machine to perform any one or more of the methodologies discussed herein. DETAILED DESCRIPTION

[0013] Disclosed are systems and methods for providing improved video capture, display, or forwarding in augmented reality (AR) devices.

[0014] As referred to herein, the term "augmented reality experience" includes or refers to various image processing operations corresponding to image modifications, filters, media overlays, transformations, and the like. In some examples, these image processing operations provide an interactive experience of a real-world environment in which objects, surfaces, backgrounds, lighting, and the like in the real world are enhanced by computer-generated perceptual information. The augmented reality experience may also include associated audio, such as a soundtrack or sound effects. In this case, the "AR effect" includes a collection of data, parameters, and other assets required to apply the selected augmented reality experience to an image or video feed. In some examples, the augmented reality effects are provided by Snap Inc. under the registered trademark LENSES.

[0015] AR effects are applied to the video stream captured by the camera in the AR device in use to provide an enhanced user experience. However, the video stream can also be used for many different purposes, including object detection and tracking, AR device positioning and orientation detection using image processing techniques such as simultaneous positioning and tracking, and QR code detection. AR effects can be rendered onto the video stream for display to the user, for recording, and for forwarding to other users.

[0016] The demands placed on AR devices and the video processing pipeline in the AR devices may cause the video stream to freeze, which provides an undesirable user experience. This may negatively impact both the local rendering of the AR-enhanced video stream to the display of the AR device and the rendering of AR-enhanced video recorded from the video stream for later viewing or forwarding to other users. In particular, the camera stream may freeze due to the requirements of applying AR effects to the camera stream, rendering the AR-enhanced stream to the display (or "viewfinder") of the AR device for real-time viewing by the user, and rendering the AR-enhanced stream for recording.

[0017] To address this issue, two camera streams are provided. The first stream is provided to the device display (or "viewfinder") for real-time viewing by the user, while the second stream is provided directly to the video encoder for recording. By directly recording the second stream instead of also presenting it for display and without applying AR effects, the second stream is less likely to include any jams. Therefore, any jams that may occur in the independent first stream will not be reflected in the recorded video file. AR effects can then be applied to the recorded stream later when the unenhanced video file is accessed, such as when playing it back for later viewing by the user, or when forwarding or streaming it to other users.

[0018] In some examples, a method performed by one or more processors is provided for providing an image enhancement effect on a device including a display and at least one camera, the method comprising: receiving a first image stream captured by the at least one camera; applying an augmented reality effect to the first image stream captured by the at least one camera to generate an enhanced image stream; displaying the enhanced image stream on the display; receiving a second image stream corresponding to the first image stream captured by the at least one camera; and saving the second image stream to an initial video file.

[0019] The second image stream may have a higher resolution than the first image stream. The method may further include: saving identification information of the augmented reality effect together with the initial video file.

[0020] In some examples, the method may also include: retrieving a second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate an additional enhanced image stream; and displaying the additional enhanced image stream on the display. The additional advanced image stream may then be saved as an enhanced video file.

[0021] The method may also include: after completing displaying the additional enhanced image stream on the display: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further enhanced image stream; and saving the further enhanced image stream as an enhanced video file.

[0022] The method may also include: retrieving a second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate an additional augmented image stream; and saving the additional augmented image stream to the augmented video file. Retrieving the second image stream from the initial file may be initiated in response to a playback user input, the method also includes: retrieving the additional augmented image stream from the augmented video file; and displaying the additional augmented image stream on the display.

[0023] In some examples, a non-transitory computer-readable storage medium is provided, which includes instructions that, when executed by a computer, cause the computer to perform operations for providing image enhancement effects on a device including a display and at least one camera, the operations including any of the above methods and limitations, including but not limited to: receiving a first image stream captured by at least one camera; applying an augmented reality effect to the first image stream captured by at least one camera to generate an enhanced image stream; displaying the enhanced image stream on the display; receiving a second image stream corresponding to the first image stream captured by at least one camera; and saving the second image stream to an initial video file.

[0024] In some examples, a computing device is provided that includes at least one camera, a display, one or more processors, and a memory that stores instructions that, when executed by the one or more processors, configure the device to perform operations for providing image enhancement effects, the operations including any of the above methods and limitations, including but not limited to: receiving a first image stream captured by at least one camera; applying an augmented reality effect to the first image stream captured by at least one camera to generate an enhanced image stream; displaying the enhanced image stream on the display; receiving a second image stream corresponding to the first image stream captured by at least one camera; and saving the second image stream to an initial video file.

[0025] Figure 1 1 is a block diagram illustrating an example messaging system 100 for exchanging data (e.g., messages, media, and associated content) over a network. The messaging system 100 includes multiple instances of a user device 102, each of the multiple instances hosting multiple applications including a messaging client 104 and other applications 106. Each messaging client 104 is communicatively coupled to other instances of the messaging client 104 (e.g., hosted on respective other client devices 102), a messaging server system 108, and a third-party server 110 via a network 112 (e.g., the Internet). The messaging client 104 may also communicate with the locally hosted application 106 using an application program interface (API).

[0026] The messaging clients 104 are able to communicate and exchange data with other messaging clients 104 and messaging server systems 108 via the network 112. The data exchanged between the messaging clients 104 and between the messaging clients 104 and messaging server systems 108 include functions (e.g., commands for invoking functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0027] The messaging server system 108 provides server-side functionality to specific messaging clients 104 via the network 112. Although certain functions of the messaging system 100 are described herein as being performed by the messaging client 104 or by the messaging server system 108, the location of certain functions within the messaging client 104 or within the messaging server system 108 may be a design choice. For example, it may be technically preferred to initially deploy certain technologies and functions within the messaging server system 108, but later migrate the technologies and functions to the messaging client 104 where the user device 102 has sufficient processing power.

[0028] The messaging server system 108 supports various services and operations provided to the messaging clients 104. Such operations include sending data to the messaging clients 104, receiving data from the messaging clients 104, and processing data generated by the messaging clients 104. As examples, the data may include message content, client device information, geographic location information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. The data exchange within the messaging system 100 is activated and controlled by functions available via the user interface (UI) of the messaging client 104.

[0029] Turning now specifically to the messaging server system 108, an application program interface (API) server 116 is coupled to the application server 114 and provides a programming interface to the application server 114. The application server 114 is communicatively coupled to a database server 120, which facilitates access to a database 126 that stores data associated with messages processed by the application server 114. Similarly, a web server 128 is coupled to the application server 114 and provides a web-based interface to the application server 114. To this end, the web server 128 processes incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0030] The application program interface (API) server 116 receives and sends message data (e.g., commands and message payloads) between the user device 102 and the application server 114. Specifically, the application program interface (API) server 116 provides a set of interfaces (e.g., routines and protocols) that can be called or queried by the messaging client 104 to activate the functions of the application server 114. The application program interface (API) server 116 exposes various functions supported by the application server 114, including: account registration; login functionality; sending messages from a particular messaging client 104 to another messaging client 104 via the application server 114; sending media files (e.g., images or videos) from the messaging client 104 to the messaging server 118 and for possible access by another messaging client 104; setting up a collection of media data (e.g., a story); retrieving a friend list of a user of the user device 102; retrieving such a collection; retrieving messages and content; adding and removing entities (e.g., friends) from an entity graph (e.g., a social graph); locating friends in a social graph; and opening application events (e.g., related to the messaging client 104).

[0031] The application server 114 hosts multiple server applications and subsystems, including, for example, a messaging server 118, an image processing server 122, and a social network server 124. The messaging server 118 implements multiple message processing technologies and functions, particularly those related to aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of the messaging client 104. Text and media content from multiple sources can be aggregated into collections of content (e.g., called stories or galleries). These collections are then made available to the messaging client 104. Given the hardware requirements for other processor and memory intensive processing of data, such processing can also be performed on the server side by the messaging server 118.

[0032] The application server 114 also includes an image processing server 122 that is dedicated to performing various image processing operations, typically with respect to images or videos within the payload of messages sent from the messaging server 114 or received at the messaging server 118 .

[0033] The social network server 124 supports various social networking functions and services and makes them available to the messaging server 118. To this end, the social network server 124 maintains and accesses an entity graph within the database 126. Examples of functions and services supported by the social network server 124 include identifying other users with whom a particular user of the messaging system 100 has relationships or that the particular user is "following," and also includes identifying interests and other entities of a particular user.

[0034] Figure 2 1 is a block diagram showing additional details about the messaging system 100 according to some examples. Specifically, the messaging system 100 is shown to include a messaging client 104 and an application server 114. The messaging system 100 contains several subsystems that are supported on the client side by the messaging client 104 and on the server side by the application server 114. These subsystems include, for example, a user interface 202, a collection management system 204, an enhancement system 208, a mapping system 210, and a gaming system 212.

[0035] The user interface 202 is responsible for providing output to and receiving input from a user of the messaging client 104 on the user device 102. As is known in the art, the user interface provides a user-manipulatable display output on the display of the user device 102 (see further Fig.10 In one example, the user interface includes a chat interface whereby a user can send and receive messages and associated content to and from one or more remote users. The user interface 202 also allows the user to manipulate live or captured media, such as by providing augmented reality effects on captured photos or videos or on a live video feed from a camera of the user device.

[0036] The collection management system 204 is responsible for managing collections or collections of media (e.g., collections of text, images, video, and audio data). Collections of content (e.g., messages, including images, videos, text, and audio) can be organized into "event libraries" or "event stories". Such collections can be made available for a specified time period (e.g., the duration of the event to which the content relates). For example, content related to a concert can be made available as a "story" for the duration of the concert. The collection management system 204 can also be responsible for publishing an icon to the user interface of the messaging client 104 that provides notification of the existence of a particular collection.

[0037] In addition, the collection management system 204 also includes a curation interface 206, which allows collection managers to manage and curate specific content collections. For example, the curation interface 206 enables an event organizer to curate a collection of content related to a specific event (e.g., to delete inappropriate content or redundant messages). In addition, the collection management system 204 uses machine vision (or image recognition technology) and content rules to automatically curate content collections. In some examples, compensation can be paid to users for including user-generated content in a collection. In such a case, the collection management system 204 operates to automatically pay such users for using their content.

[0038] The enhancement system 208 provides various functions that enable users to enhance (e.g., annotate or otherwise modify or edit) media content associated with a message. For example, the enhancement system 208 provides functions related to generating and publishing media overlays (media overlays) for messages processed by the messaging system 100. The enhancement system 208 is operable to provide media overlays or enhancements (e.g., image filters) to the messaging client 104 based on the geographic location of the user device 102. In another example, the enhancement system 208 is operable to provide media overlays to the messaging client 104 based on other information such as social network information of the user of the user device 102. Media overlays can include audio and visual content and visual effects. Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Examples of visual effects include color overlays. Audio and visual content or visual effects can be applied to media content items (e.g., photos) at the user device 102. For example, media overlays can include text or images that can be superimposed on top of photos taken by the user device 102. In another example, the media overlay includes a location identifier (e.g., Venice Beach) overlay, the name of a live event, or a business name (e.g., Beach Cafe) overlay. In another example, the augmentation system 208 uses the geographic location of the user device 102 to identify a media overlay that includes a business name at the geographic location of the user device 102. The media overlay may include other logos associated with the business. The media overlay may be stored in the database 120 and accessed by the database server 120.

[0039] The mapping system 210 provides various geo-location functions and supports presentation of map-based media content and messages by the messaging client 104. For example, the mapping system 210 enables display of user icons or avatars (e.g., stored in the profile data 316) on a map to indicate the current or past locations of the user's "friends" and media content (e.g., a collection of messages including photos and videos) generated by such friends in the context of the map. For example, on the map interface of the messaging client 104, messages posted by the user to the messaging system 100 from a particular geographic location can be displayed to the "friends" of a particular user within the context of that particular location on the map. The user can also share his or her location and status information with other users of the messaging system 100 (e.g., using appropriate status avatars) via the messaging client 104, where the location and status information is similarly displayed to the selected user within the context of the map interface of the messaging client 104.

[0040] The gaming system 212 provides various gaming functions within the context of the messaging client 104. The messaging client 104 provides a gaming interface that provides a list of available games that can be launched by a user within the context of the messaging client 104 and played with other users of the messaging system 100. The messaging system 100 also enables a particular user to invite other users to participate in playing a particular game by sending an invitation to such other users from the messaging client 104. The messaging client 104 also supports both voice messaging and text messaging within the context of game play (e.g., chatting), provides leaderboards for games, and also supports providing in-game rewards (e.g., game coins and items).

[0041] Figure 3 A recording and display process flow 302 and a playback process flow 304 for AR-augmented video in a single-camera flow implementation are shown according to some examples.

[0042] In the recording and display process flow 302, a camera server 308 receives a video stream of a user 306 generated by a camera on a user device. The camera server 308 passes the video stream to a camera frame scheduler 310, which provides the video frames to the augmentation system 208, which in turn applies augmented reality effects to the video frames and, therefore, to the video stream.

[0043] The augmented video stream is then rendered in a rendering operation 314 for display on a viewfinder or display 316. The augmented video stream is then passed to a rendering operation 318 which records the video stream to an AR augmented video file 320.

[0044] For convenience, in the figures, the AR-augmented user 312 depicted in the augmented video stream or in the AR-augmented video file 320 is shown to include a dog's tongue, ears, and nose to distinguish from the un-augmented video of the user 306 .

[0045] In playback process flow 304 , the AR augmented video file 320 is retrieved and decoded by a video decoder 322 , and played back by a video player 324 , which renders the augmented video stream in a rendering operation 326 for display on the display 316 as before.

[0046] Figure 4 A recording and display process flow 402 for AR-enhanced video in a dual camera stream implementation is shown according to some examples.

[0047] In the recording and display process flow 302, the camera server 308 receives a video stream of the user 306 captured by the camera on the user device 102. The camera server 308 passes the first video stream 404 to the camera frame scheduler 310, which provides the video frames to the augmentation system 208. The augmentation system 208 applies augmented reality effects to the video frames and thus to the video stream (e.g., Figure 3 Then, in a rendering operation 410, an augmented video stream including a depiction of the AR-augmented user 312 is rendered for display on a viewfinder or display 316.

[0048] However, with Figure 3 Unlike the display processing flow 302 shown, in the recording and display processing flow 402, the camera server 308 also provides a second video stream 406 of the user 306. The second video stream 406 does not have any AR effects applied to it and is provided to the video codec 408, which encodes the initial or unenhanced video file 412 and then saves it to local or remote storage. Associated with the unenhanced video file 412 are descriptors that identify the AR effects and any associated parameters applied by the augmentation system 208 before displaying the AR-enhanced user 312 in the display 316. These descriptors are associated with the unenhanced video file 412 for later use, for example, by saving the descriptors and any associated parameters with the unenhanced video file 412 as metadata, or in a separate file that has a link or identifier to the unenhanced video file 412.

[0049] Figure 5A playback / re-recording process flow 502 for AR-enhanced video in a dual-camera stream implementation according to some examples is shown. The process flow is initiated in response to receiving a user input requesting playback of an enhanced version of the non-enhanced video file 412 (because the user believes the file has been saved).

[0050] In the playback / re-recording process flow 502, the unenhanced video file 412 is retrieved and decoded by the video decoder 322 and played back by the video player 324 to generate an unenhanced video stream. The unenhanced video stream is provided to the augmentation system 208, which retrieves and applies the AR effects and any associated parameters previously used by the augmentation system 208 to provide an AR-enhanced display 316 (e.g., Figure 4 The resulting AR-enhanced video stream of the user 312 is then provided to a rendering operation 504 and displayed by the display 316.

[0051] In addition to or as an alternative to displaying the AR-enhanced video stream on the display 316 , the AR-enhanced video stream may also be saved to a file 506 .

[0052] As another alternative to display 316, the augmented video stream may be streamed to a remote location (e.g., a user device 102 of a remote user), or may be sent to an activity feed on a social networking website or social networking application, or sent as a message to a remote user by a messaging client 104. Such streaming or sending may occur in real time directly after the augmented reality effect is applied by augmentation system 208, or a saved file 506 may be sent.

[0053] By providing separate recording and display processing streams 402 and playback / re-recording processing stream 502 in parallel processing streams rather than in one serial processing stream, the frame rate and amount of stuttering of both the live display of the enhanced video stream and any subsequent display of the recording of the enhanced video stream can be improved.

[0054] The first video stream 404 and the second video stream 406 may be identical. However, in some examples, the first video stream 404 may be a lower resolution computer vision grade video stream suitable for performing AR-related analysis such as object detection, tracking, QR code recognition, etc., while the second video stream 406 is a full resolution video stream more suitable for messaging and other social media applications. Capturing the first video stream 404 at a lower resolution may reduce the overall computing resource usage associated with both capturing and processing the first video stream 404.

[0055] However, the first video stream and the second video stream are consistent in that they reflect the same video subject and are captured by the same camera at the same time in most cases (but not all cases).

[0056] Figure 6 A playback and forwarding / saving processing flow 602 for AR-enhanced video in a dual camera stream implementation according to some examples is shown. The processing flow is initiated in response to receiving a user input requesting playback of an enhanced version of the unenhanced video file 412 (because the user believes the file has been saved). In some cases, the user will play back the AR-enhanced video immediately after the recording is completed to preview or review the completed video for user approval. For example, the user may have just completed a recording and is presented with the options of "Play", "Save", "Discard", and "Forward".

[0057] In the playback and forwarding / saving process flow 602, in response to receiving a user input of a "play" option, the video decoder 322 retrieves and decodes the unenhanced video file 412 in operation 604, and the video player 324 plays back the unenhanced video file 412 in operation 606 to generate an unenhanced video stream. The unenhanced video stream is provided to the augmentation system 208, which in operation 608 performs an AR augmentation based on the live display 316 (e.g., the augmentation system 208 previously used to provide the AR augmentation user 312) . Figure 4 The AR effect identifier and any associated parameters of the AR effect identifier are retrieved and applied to the corresponding AR effect. Then, in the presentation operation 610, the resulting video stream of the AR enhanced user 312 is presented to the display 316 for the user to watch.

[0058] However, with Figure 5 Differently, the enhanced video stream received from the augmentation system 208 and presented to the display in the rendering operation 610 is not directly saved to a file. Due to the real-time requirements of applying the AR effects and instantly presenting the enhanced video stream, there may be pauses in the enhanced video stream when presenting the enhanced video stream for display to the user. Therefore, the displayed version of the enhanced video stream will not be as Figure 5 Although the initial stuttering in the enhanced video stream displayed to the user is less than ideal, Figure 6 The advantage of the method is that the user can watch / preview the enhanced video stream immediately when the recording is completed.

[0059] When the save or forward user input is received in operation 612, the unenhanced video file is decoded in operation 614, the unenhanced video file is played back by the video player in operation 616, the AR effect is applied in operation 618, and the resulting enhanced video stream is saved to a file in operation 620. The user input for saving or forwarding the enhanced video stream may be received after the display of the video in the rendering operation 610 is paused or completed. Alternatively, the user input for saving or forwarding the enhanced video stream may be received in advance without playing back the enhanced video picture as described in the playback operations 604 to 610. In this case, the method starts from operation 612.

[0060] Since the enhanced video stream will not be displayed to the user, operations 614 to 618 need not be performed in real time at a rate that allows simultaneous video playback, but can be performed strictly to ensure that no hitches occur in the enhanced video stream saved to the file in operation 620. The benefit of this is that any future viewing of the enhanced video stream by the user or recipient will not experience any hitches because it is being played back from an otherwise standard video file that already includes the enhanced video stream.

[0061] If a forward input is received in operation 612, the user device 102 sends the file containing the enhanced video stream to a remote location (e.g., the user device 102 of the remote user), to an activity feed on a social networking site or social networking application, or to the remote user as part of a message by the messaging client 104 in operation 622 as specified in the forward user input. If a save input is received in operation 612, the method ends at operation 620 with the enhanced video file being saved. Any further playback of the enhanced video will then be based on playback of the file, rather than on operations 604 to 612, thereby avoiding potential stuttering issues that may be associated with these operations.

[0062] Figure 7 A playback and forwarding / saving process flow 702 for AR-enhanced video in a dual-camera stream implementation according to some examples is shown. The process flow is initiated in response to receiving a user input requesting playback of an enhanced version of the un-enhanced video file 412 (because the user believes that the video file has been saved, in some cases for immediate preview or review after recording is complete for user approval).

[0063] In the playback and forwarding / saving process flow 702, the unenhanced video file 412 is retrieved and decoded by the video decoder 322 and played back by the video player 324 to generate an unenhanced video stream. In operation 708, the unenhanced video stream is provided to the augmentation system 208, which retrieves and applies the AR effect and any associated parameters that were previously used by the augmentation system 208 to provide an AR augmented user 312 live display (e.g., Figure 4 described above).

[0064] However, with Figure 6 Differently, in operation 710, the enhanced video stream received from the enhanced system in operation is not presented for display, but is saved to a file. Since the enhanced video stream is not displayed to the user, operations 704 to 706 do not need to be performed at a rate that allows simultaneous video playback, but can be strictly performed to ensure that there is no lag in the enhanced video stream saved in operation 710. This has the advantage that no lag will occur when viewing / previewing the enhanced video stream, but has the disadvantage that there will be a delay between the receipt of the user input for viewing / previewing the enhanced video stream and the display of the enhanced video stream when playback is first requested.

[0065] After being saved in operation 710, the enhanced video file is then decoded in operation 712, played back by the video player in operation 714, and the resulting enhanced video stream is presented to the display 316 for viewing by the user in operation 716. Thus, the user has an accurate preview of what any recipient of the enhanced video file will see.

[0066] If forwarding user input is received in operation 718, then in operation 720, the file containing the enhanced video stream is sent to a remote location (e.g., a user device 102 of the remote user), an activity feed on a social networking site or social networking application, or sent as a message to the remote user by the messaging client 104, as specified in the forwarding user input.

[0067] In some examples, operations 704 to 710 may be started automatically in the background as soon as the recording of the video is complete, provided there are sufficient available resources (e.g., processing power, battery life, etc.). In this case, operation 704 may be run as a lower priority task so as not to impact other functions of the user device 102. Starting these operations immediately has the advantage of reducing or possibly eliminating any delay associated with rendering to the display in operation 716, but has the disadvantage of not being necessary if user input to discard the file is received without viewing / previewing the file.

[0068] Figure 8An architecture 800 for AR-enhanced video in a dual camera stream implementation according to some examples is shown. The architecture 800 is embodied in a user device 102, a camera server 802 is a service provided by an operating system, and an application 804 runs on the device.

[0069] It can be seen that the camera server 802 provides a frame 806 in the first video stream 404 of camera frames to the application 804. The application 804 generates a texture 808 from the frame 806, for example, by applying an augmented reality effect to the frame 806. The application 804 then renders the texture 808 to a frame 812, and the frame 812 is displayed to the user on the display 810.

[0070] The camera server 802 also provides a frame 816 to the application 804 in a second video stream 406 of camera frames. The frame 816 may be the same as the frame 806, but may also be different, such as having a higher resolution. The frame 816 is received by a recorder 814, which includes an encoder 818, a multiplexer 820, and a file system 822. The encoder 818 encodes the stream of frames 816 into an appropriate video format and passes it to the multiplexer 820, where the stream of frames 816 is combined with associated audio (e.g., audio received from a microphone associated with the user device 102). The resulting encoded video stream is then passed to the file system 822, where it is saved as a file for future viewing, enhancement, forwarding, etc.

[0071] Fig. 9 900 is a block diagram illustrating a software architecture 904 that can be installed on any one or more of the devices described herein. The software architecture 904 is supported by hardware such as a machine 902, which includes a processor 920, a memory 926, and an I / O component 938. In this example, the software architecture 904 can be conceptualized as a stack of layers, where each layer provides specific functionality. The software architecture 904 includes the following layers, such as an operating system 912, a library 910, a framework 908, and an application 906. In operation, the application 906 invokes an API call 950 through the software stack and receives a message 952 in response to the API call 950.

[0072] The operating system 912 manages hardware resources and provides public services. The operating system 912 includes, for example, a core 914, a service 916, and a driver 922. The core 914 is used as an abstraction layer between the hardware and other software layers. For example, the core 914 provides functions such as memory management, processor management (e.g., scheduling), component management, networking, and security settings. The service 916 can provide other public services for other software layers. The driver 922 is responsible for controlling the underlying hardware or interfacing with the underlying hardware. For example, the driver 922 may include a display driver, a camera driver, a Bluetooth or Bluetooth low energy driver, a flash drive, a serial communication driver (e.g., a USB driver), a WI-FI driver, an audio driver, a power management driver, etc.

[0073] The library 910 provides a common low-level infrastructure used by the application 906. The library 910 may include a system library 918 (e.g., a C standard library) that provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the library 910 may include an API library 924, such as a media library (e.g., a library for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework for presenting in two dimensions (2D) and three dimensions (3D) in graphics content on a display), a database library (e.g., SQLite providing various relational database functions), a web library (e.g., WebKit providing web browsing functions), etc. The library 910 may also include various other libraries 928 to provide many other APIs to the application 906.

[0074] The framework 908 provides a common high-level infrastructure used by the applications 906. For example, the framework 908 provides various graphical user interface (GUI) functions, high-level resource management, and high-level positioning services. The framework 908 can provide a wide range of other APIs that can be used by the applications 906, some of which can be specific to a specific operating system or platform.

[0075] In the example, the application 906 may include a home application 936, a contact application 930, a browser application 932, a book reader application 934, a location application 942, a media application 944, a messaging application 946 (e.g., a messaging client 104), a game application 948, and a variety of other applications such as a third-party application 940. The application 906 is a program that executes the functions defined in the program. Various programming languages ​​can be used to create one or more of the applications 906 constructed in various ways, such as object-oriented programming languages ​​(e.g., Objective-C, Java, or C++) or procedural programming languages ​​(e.g., C language or assembly language). In a specific example, a third-party application 940 (e.g., an application developed using an ANDROID or IOS software development kit (SDK) by an entity other than a supplier of a specific platform) can be a mobile software running on a mobile operating system such as IOS, ANDROID, WINDOWS Phone, or another mobile operating system. In this example, the third-party application 940 can activate an API call 950 provided by the operating system 912 to facilitate the functions described herein.

[0076] Fig.10It is a diagrammatic representation of a machine 1000 (e.g., user device 102) within which instructions 1010 (e.g., software, programs, applications, applet, app, or other executable code) for causing the machine 1000 to perform any one or more of the methods discussed herein may be executed. For example, the instructions 1010 may cause the machine 1000 to perform any one or more of the methods described herein. The instructions 1010 transform a general-purpose, unprogrammed machine 1000 into a specific machine 1000 that is programmed to perform the functions described and illustrated in the manner described. The machine 1000 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1000 may operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing instructions 1010 specifying actions to be taken by the machine 1000. In addition, although only a single machine 1000 is shown, the term "machine" should also be considered to include a collection of machines that individually or jointly execute instructions 1010 to perform any one or more of the methods discussed herein. For example, the machine 1000 may include a user device 102 or any of a number of server devices that form part of a messaging server system 108. In some examples, the machine 1000 may also include both a client system and a server system, wherein certain operations of a particular method or algorithm are performed on the server side and certain operations of the particular method or algorithm are performed on the client side.

[0077] The machine 1000 may include a processor 1004, a memory 1006, and an input / output I / O component 1002 that may be configured to communicate with each other via a bus 1040. In an example, the processor 1004 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1008 that executes instructions 1010 and a processor 1012. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions simultaneously. Although Fig.10 Multiple processors 1004 are shown, but the machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0078] The memory 1006 includes a main memory 1014, a static memory 1016, and a storage unit 1018, which are all accessible by the processor 1004 via the bus 1040. The main memory 1006, the static memory 1016, and the storage unit 1018 store instructions 1010 that embody any one or more of the methodologies or functions described herein. The instructions 1010 may also reside, completely or partially, within the main memory 1014, within the static memory 1016, within the machine-readable medium 1020 within the storage unit 1018, within at least one of the processors 1004 (e.g., within a cache memory of a processor), or within any suitable combination thereof during execution thereof by the machine 1000.

[0079] I / O components 1002 may include various components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O components 1002 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine will most likely not include such a touch input device. It should be understood that I / O components 1002 may include Fig.101020 may include a plurality of other components not shown in the drawings. In various examples, the I / O components 1002 may include a user output component 1026 and a user input component 1028. The user output component 1026 may include a visual component (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an acoustic component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The user input component 1028 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input component), a point-based input component (e.g., a mouse, a touch pad, a trackball, a joystick, a motion sensor, or other pointing instrument), a tactile input component (e.g., a physical button, a touch screen or other tactile input component that provides the location and force of a touch or a touch gesture), an audio input component (e.g., a microphone), etc.

[0080] In other examples, the I / O component 1002 may include a biometric component 1030, a motion component 1032, an environment component 1034, or a positioning component 1036, as well as various other components. For example, the biometric component 1030 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 1032 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, and a rotation sensor component (e.g., a gyroscope).

[0081] Environmental components 1034 include, for example, one or more cameras (with still image / photo and video capabilities), lighting sensor components (e.g., a photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., a barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., an infrared sensor that detects nearby objects), gas sensors (e.g., a gas detection sensor that detects concentrations of hazardous gases for safety or measures pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment.

[0082] With respect to cameras, user device 102 may have a camera system including, for example, a front-facing camera on a front surface of user device 102 and a rear-facing camera on a rear surface of user device 102. The front-facing camera may, for example, be used to capture still images and videos (e.g., “selfies”) of a user of user device 102, which may then be enhanced with the enhancement data (e.g., filters) described above. For example, the rear-facing camera may be used to capture still images and videos in a more conventional camera mode, where the images are similarly enhanced with the enhancement data. In addition to the front-facing camera and the rear-facing camera, user device 102 may also include a 360° camera for capturing 360° photos and videos.

[0083] Additionally, the camera system of user device 102 may include dual rear cameras (e.g., a main camera and a depth sensing camera), or even triple, quad, or quintuple rear camera configurations on the front and back sides of user device 102. For example, these multiple camera systems may include a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, a macro camera, and a depth sensor.

[0084] The positioning component 1036 includes a position sensor component (for example, a GPS receiver component), an altitude sensor component (for example, an altimeter or a barometer that detects air pressure, and the altitude can be obtained based on the air pressure), an orientation sensor component (for example, a magnetometer), etc.

[0085] A variety of technologies may be used to achieve communication. The I / O component 1002 also includes a communication component 1038 that is operable to couple the machine 1000 to the network 1022 or the device 1024 via a corresponding coupling or connection. For example, the communication component 1038 may include a network interface component or other suitable device that interfaces with the network 1022. In other examples, the communication component 1038 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, a Bluetooth component (e.g., Bluetooth low energy), a Wi-Fi component, and other communication components that provide communication via other modalities. The device 1024 may be any peripheral device of another machine or a variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0086] In addition, the communication component 1038 can detect the identifier or include a component that can be operated to detect the identifier. For example, the communication component 1038 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting a one-dimensional barcode such as a universal product code (UPC) barcode, a multi-dimensional barcode such as a Quick Response (QR) code, an Aztec code, a data matrix, a data symbol (Dataglyph), a maximum code (MaxiCode), PDF417, Ultra Code, UCC RSS-2D barcode, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying a marked audio signal). In addition, various information can be derived via the communication component 1038, such as deriving a location via Internet Protocol (IP) geolocation, deriving a location via Wi-Fi signal triangulation, deriving a location via detecting an NFC beacon signal that can indicate a specific location, etc.

[0087] Various memories (e.g., main memory 1014, static memory 1016, and memory of processor 1004) and storage unit 1018 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. When executed by processor 1004, these instructions (e.g., instructions 1010) cause various operations to implement the disclosed examples.

[0088] The instructions 1010 may be sent or received over the network 1022 using a transmission medium via a network interface device (e.g., a network interface component included in the communication component 1038) and using any of a number of well-known transmission protocols (e.g., the Hypertext Transfer Protocol (HTTP)). Similarly, the instructions 1010 may be sent or received via a coupling (e.g., a peer-to-peer coupling) with the device 1024 using a transmission medium.

[0089] Glossary

[0090] "Carrier signal" refers to any intangible medium that can store, encode or carry instructions for execution by a machine and includes digital or analog communications signals or other intangible media that facilitates communication of such instructions. Instructions may be sent or received over a network using a transmission medium via a network interface device.

[0091] "Client Device" refers to any machine that interfaces with a communications network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a portable digital assistant (PDA), a smart phone, a tablet computer, an ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronics product, a game console, a set-top box, or any other communications device that a user may use to access a network.

[0092] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a part of the Internet, a part of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, A network, other type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate for GSM evolution (EDGE) technology, the third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other data transmission technologies defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.

[0093] "Component" refers to a logic, device or physical entity with boundaries, which is defined by function or subroutine calls, branch points, APIs or other technologies that provide partitioning or modularization for specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processing. Components can be packaged functional hardware units designed for use with other components and can be part of a program that generally performs specific functions in related functions. Components can constitute software components (e.g., codes implemented on machine-readable media) or hardware components. "Hardware components" are tangible units that can perform certain operations and can be configured or arranged in a certain physical manner. In various examples, one or more computer systems (e.g., independent computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., applications or application parts) to operate to perform certain operations described herein. Hardware components can also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component may include a dedicated circuit or logic that is permanently configured to perform certain operations. The hardware component may be a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware component may also include a programmable logic or circuit system that is temporarily configured to perform certain operations by software. For example, the hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) that is uniquely customized to perform the configured function, and is no longer a general-purpose processor. It should be understood that it can be decided whether to mechanically implement the hardware component in a dedicated and permanently configured circuit or in a circuit temporarily configured (for example, configured by software) for cost and time considerations. Therefore, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, that is, an entity that is physically constructed, permanently configured (for example, hardwired) or temporarily configured (for example, programmed) to operate in some way or perform certain operations described herein. Considering an example in which a hardware component is temporarily configured (for example, programmed), it is not necessary to configure or instantiate each of the hardware components at any time. For example, in the case where a hardware component includes a general-purpose processor that is configured by software to become a special-purpose processor, the general-purpose processor can be configured into different special-purpose processors (e.g., including different hardware components) at different times. Software accordingly configures one or more specific processors to, for example, constitute a specific hardware component at one time, and to constitute different hardware components at different times. Hardware components can provide information to other hardware components and receive information from other hardware components. Accordingly, the described hardware components can be considered to be communicatively coupled.In the case of multiple hardware components being present at the same time, communication can be achieved by signal transmission (e.g., by appropriate circuits and buses) between or among two or more hardware components. In examples where multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessible to multiple hardware components and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device coupled to it in communication. Then, another hardware component can access the memory device at a subsequent time to retrieve the stored output and process it. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (e.g., a collection of information). The various operations of the example methods described herein can be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform related operations. Whether it is temporarily configured or permanently configured, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented components" refer to hardware components implemented using one or more processors. Similarly, the method described in this article can be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some of the operation of the method can be performed by one or more processors 1004 or the parts implemented by the processor. In addition, one or more processors can also operate to support related operations in a "cloud computing" environment or as "software as a service" (SaaS) execution. For example, at least some of the operations in the operation can be performed by a group of computers (as an example of a machine including a processor), wherein these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., API). The execution of some operations in the operation can be distributed between processors, can not only reside in a single machine, but also can be deployed across several machines. In some examples, a processor or a part implemented by a processor can be located in a single geographical location (e.g., in a home environment, an office environment, or a server cluster). In other examples, a processor or a part implemented by a processor can be distributed across several geographical locations.

[0094] "Computer-readable storage media" refers to both machine storage media and transmission media. Therefore, these terms include storage devices / media and carrier / modulated data signals. The terms "machine-readable medium", "computer-readable medium" and "device-readable medium" mean the same thing and can be used interchangeably in this disclosure.

[0095] "Transient messages" are messages that are accessible for a limited duration. Transient messages can be text, images, videos, etc. The access time for transient messages can be set by the sender of the message. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is transient.

[0096] "Machine storage media" refers to a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Thus, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage media", "device storage media", and "computer storage media" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage media", "computer storage media", and "device storage media" expressly exclude carrier waves, modulated data signals, and other such media (at least some of which are encompassed by the term "signal media").

[0097] “Non-transitory computer-readable storage medium” refers to tangible media capable of storing, encoding, or carrying instructions to be executed by a machine.

[0098] "Signal medium" refers to any intangible medium that is capable of storing, encoding or carrying instructions executed by a machine and includes digital or analog communication signals or other intangible media that facilitate the communication of software or data. The term "signal medium" should be deemed to include any form of modulated data signals, carrier waves, etc. The term "modulated data signal" refers to a signal that has one or more of its various characteristics set or changed in such a manner as to encode information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

Claims

1. A method executed by one or more processors for providing an image enhancement effect on a device including a display and at least one camera, the method comprising: receiving a first image stream captured by the at least one camera; applying an augmented reality effect to a first image stream captured by the at least one camera device to generate an augmented image stream; displaying the enhanced image stream on the display; receiving a second image stream corresponding to the first image stream captured by the at least one camera; as well as The second image stream is saved to the initial video file.

2. The method according to claim 1, wherein: The second image stream has a higher resolution than the first image stream.

3. The method according to claim 1, further comprising: The identification information of the augmented reality effect is saved together with the initial video file.

4. The method according to claim 1, further comprising: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; as well as The additional enhanced image stream is displayed on the display.

5. The method according to claim 4, further comprising: The additional advanced image stream is saved as an enhanced video file.

6. The method according to claim 4, further comprising: After completing displaying the additional enhanced image stream on the display: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; as well as The further advanced image stream is saved as an enhanced video file.

7. The method according to claim 1, further comprising: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; as well as The additional enhanced image stream is saved to an enhanced video file.

8. The method according to claim 7, wherein: Retrieving the second image stream from the initial file is initiated in response to playback user input, the method further comprising: retrieving the additional enhancement image stream from the enhancement video file; and The additional enhanced image stream is displayed on the display.

9. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform operations for providing an image enhancement effect on a device comprising a display and at least one camera, the operations comprising: receiving a first image stream captured by the at least one camera; applying an augmented reality effect to a first image stream captured by the at least one camera device to generate an augmented image stream; displaying the enhanced image stream on the display; receiving a second image stream corresponding to the first image stream captured by the at least one camera; as well as The second image stream is saved to the initial video file.

10. The non-transitory computer-readable storage medium of claim 9, wherein: The operations also include: The identification information of the augmented reality effect is saved together with the initial video file.

11. The non-transitory computer-readable storage medium of claim 9, wherein: The operations also include: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; and The additional enhanced image stream is displayed on the display.

12. The non-transitory computer-readable storage medium of claim 11, wherein: The operations further include, after completing displaying the additional enhanced image stream on the display: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; and The further advanced image stream is saved as an enhanced video file.

13. The non-transitory computer-readable storage medium of claim 9, wherein: The operations also include: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; and The additional enhanced image stream is saved to an enhanced video file.

14. The non-transitory computer-readable storage medium of claim 13, wherein: The operations also include: retrieving the additional enhancement image stream from the enhancement video file; and The additional enhanced image stream is displayed on the display.

15. A computing device comprising: at least one camera device; monitor; one or more processors; as well as a memory storing instructions that, when executed by the one or more processors, configure the device to perform operations for providing an image enhancement effect, the operations comprising: receiving a first image stream captured by the at least one camera; applying an augmented reality effect to a first image stream captured by the at least one camera device to generate an augmented image stream; displaying the enhanced image stream on the display; receiving a second image stream corresponding to the first image stream captured by the at least one camera; and The second image stream is saved to the initial video file.

16. The computing device of claim 15, wherein: The second image stream has a higher resolution than the first image stream.

17. The computing device of claim 15, wherein: The operations also include: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; and The additional enhanced image stream is displayed on the display.

18. The computing device of claim 17, wherein: The operations further include, after completing displaying the additional enhanced image stream on the display: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; and The further advanced image stream is saved as an enhanced video file.

19. The computing device of claim 15, wherein: The operations also include: retrieving the second image stream from the initial video file; applying the augmented reality effect to the second image stream to generate a further augmented image stream; and The additional enhanced image stream is saved to an enhanced video file.

20. The computing device of claim 19, wherein: The operations also include: retrieving the additional enhancement image stream from the enhancement video file; and The additional enhanced image stream is displayed on the display.

Images