SYSTEM AND METHOD FOR PERFORMING A REWIND OPERATION WITH A MOBILE IMAGE CAPTURE DEVICE - Patent application

Abstract

To provide a method for performing a rewind operation using a mobile image capture device.SOLUTION: A method for performing a rewind operation includes: providing a live video stream showing at least a part of an image capture system's current field of view for display within a viewfinder portion of a user interface; storing in a temporary image buffer a video segment including a plurality of image frames captured by the image capture system from the live video stream; receiving user input directed to the viewfinder portion of the user interface requesting the rewind operation; and performing the rewind operation within the viewfinder portion of the user interface with respect to the video segment in response to such user input. In the rewind operation, at least two of the image frames of the video segment are provided for display within the user interface in reverse chronological order.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates generally to mobile image capture systems and methods, and more particularly, to systems and methods for performing rewind operations with a mobile image capture device. [Background technology]

[0002]

[0003] An increasing number of individuals are using computing devices to capture, store, share, and interact with visual content such as photographs and videos. In particular, for some individuals, handheld computing devices such as smartphones or tablets are the primary devices used to capture visual content such as photographs and videos.

[0003] However, when capturing photos or videos of dynamic objects via a handheld computing device, a user may often miss the moment they intended to capture. For example, a user may open a camera application, point the handheld computing device at the object they intend to capture, and then operate a shutter button (e.g., a virtual shutter button in a user interface) to instruct the handheld computing device to capture an image. However, due to the dynamic nature of the object, it is easy for the user to press the shutter button too late or too early. As a result, the captured photo or video may not include what the user intended to capture. This problem is particularly serious when attempting to capture images of a particular event and / or images showing a non-compliant object (e.g., a child or pet). As a result, even when a user has the camera application open and is pointing at the desired object, the user may fail to capture the photo at just the right time to capture the desired image. For example, a user may fail to capture an image at the exact moment when their young child turns both eyes toward the camera and smiles naturally.

[0004] One action commonly taken to address this problem in existing systems is for the user to simply capture a large number of images around the time the desired event is expected to occur. However, this attempted solution has several problems. First, even if the user captures a large number of images, there is still no guarantee that the user will actually capture the desired moment. Second, this attempted solution results in the storage of a large number of photographs on the user's device. This either results in insufficient allocation of memory resources (e.g., storage of blurry or unnecessary photographs) or requires the user to manually review their photo collection and delete images that are unnecessary or otherwise do not show the desired subject. Summary of the Invention [Means for solving the problem]

[0005] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the description that follows, or may be learned from the description, or may be learned through practice of the embodiments.

[0006] One exemplary aspect of the present disclosure is directed to a mobile image capture device. The mobile image capture device may include an image capture system operable to capture image frames. The mobile image capture device may include one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations may include providing a live video stream for display within a viewfinder portion of a user interface. The live video stream may represent at least a portion of a current field of view of the image capture system of the mobile image capture device. The operations may include storing a video segment from the live video stream in a temporary image buffer. The video segment may include multiple image frames captured by the image capture system. The operations may include receiving a user input directed to the viewfinder portion of the user interface requesting a rewind operation. The operations may include performing a rewind operation on the video segment within the viewfinder portion of the user interface in response to receiving the user input requesting the rewind operation. During the rewind operation, at least two of the image frames of the video segment may be provided for display within the user interface in reverse chronological order.

[0007] Another exemplary aspect of the present disclosure is directed to a mobile image capture device. The mobile image capture device may include an image capture system operable to capture image frames from multiple sources. Each source may have a respective field of view. The mobile image capture device may include one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations may include providing a live video stream for display within a viewfinder portion of a user interface. The live video stream may include a composite environmental stream generated from multiple sources. The operations may include storing a video segment from the live video stream in a temporary image buffer. The video segment may include multiple image frames captured by the image capture system. The operations may include receiving a user input directed to the viewfinder portion of the user interface requesting a rewind operation. The operations may include performing a rewind operation on the video segment within the viewfinder portion of the user interface in response to receiving the user input requesting the rewind operation. During the rewind operation, at least two of the image frames of the video segment may be provided for display in the user interface in reverse chronological order.

[0008] Another exemplary aspect of the present disclosure is directed to a method for performing a rewind operation. The method may include providing, by one or more computing devices, a live video stream for display within a viewfinder portion of a user interface. The live video stream may represent at least a portion of a current field of view of an image capture system of a mobile image capture device. The method may include storing, by the one or more computing devices, a video segment from the live video stream in a temporary image buffer. The video segment may include a plurality of image frames captured by the image capture system. The method may include receiving, by the one or more computing devices, a user input directed to the viewfinder portion of the user interface and requesting a rewind operation. The method may include performing, by the one or more computing devices, a rewind operation within the viewfinder portion of the user interface on the video segment in response to receiving the user input requesting the rewind operation. During the rewind operation, at least two of the image frames of the video segment may be provided for display within the user interface in reverse chronological order.

[0009] Other aspects of the present disclosure are directed to various systems, apparatus, non-transitory computer-readable media, user interfaces, and electronic devices.

[0010] These and other features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain associated principles.

[0011] Detailed descriptions of embodiments directed to persons skilled in the art are set forth herein, which refer to the accompanying drawings. [Brief explanation of the drawings]

[0012] [Figure 1A] FIG. 1 is a block diagram of an exemplary computing system according to an exemplary embodiment of the present disclosure. [Figure 1B] FIG. 1 is a block diagram of an exemplary computing system according to an exemplary embodiment of the present disclosure. [Figure 1C] FIG. 1 is a block diagram of an exemplary computing system according to an exemplary embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram of a machine learning image selection model according to an exemplary embodiment of the present disclosure. [Figure 3] FIG. 1 is a diagram of a machine learning image capture model according to an exemplary embodiment of the present disclosure. [Figure 4] FIG. 1 is a diagram of a machine learning source selection model according to an exemplary embodiment of the present disclosure. [Figure 5A] FIG. 10 is a diagram of an exemplary user interface prior to performance of a rewind operation, according to an exemplary embodiment of the present disclosure. [Figure 5B] 10A-10C are diagrams of exemplary user interfaces during a rewind operation, according to exemplary embodiments of the present disclosure. [Figure 5C] 10A-10C are diagrams of exemplary user interfaces at various stages during a rewind operation, according to an exemplary embodiment of the present disclosure. [Figure 5D] 10A-10C are diagrams of exemplary user interfaces during editing of image frames to be stored, according to exemplary embodiments of the present disclosure. [Figure 5E] 10A-10C are diagrams of exemplary user interfaces displaying a pop-up window suggesting image frames for storage while performing a rewind operation, according to exemplary embodiments of the present disclosure. [Figure 6] 10A-10C are diagrams of exemplary user interfaces at various stages during a switching operation associated with a primary live video feed and a secondary live video feed, according to an exemplary embodiment of the present disclosure. [Figure 7]FIG. 10 is a flowchart diagram of an exemplary method for performing a rewind operation, according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Reference numbers repeated across the figures are intended to identify like features in various implementations.

[0014] Overview Generally, the present disclosure is directed to systems and methods that provide an improved user experience for capturing images and / or videos with a handheld computing device, such as a smartphone or tablet. The systems and methods described herein can provide a user with the ability to "rewind" time and capture video or images from an earlier moment in time when the user failed to explicitly operate the device to capture it. This functionality can enable a user to capture the "perfect shot" even after that moment has passed.

[0015] Specifically, in one example, a user can point a camera of a handheld device at a subject of interest and view a live stream from the camera in the viewfinder portion of the device's display. The device can temporarily store video segments (e.g., a set of captured images) from the live video stream in a temporary image buffer. Rather than attempting to press a capture button at the exact moment the user wants to capture, the user can watch until the moment has passed. The user can then request a rewind operation by providing user input (e.g., a swipe or other motion or gesture). In response, the device can immediately (or quickly) provide the user with the ability to “rewind time” to review previous moments displayed on the viewfinder. The previous moments may be displayed in reverse chronological order, and / or the user can control playback by, for example, scrolling through the video segment in a scrubbing operation. The user can select one or more of the images as a photo or video, e.g., capturing a “perfect shot” that the user missed, and the selected images may be saved to the device's non-transitory memory. In that way, the proposed system allows users to capture the "perfect shot" even after the moment has passed, but without cluttering the user's camera roll with unnecessary images or requiring the user to manually go through and delete numerous unnecessary photos.

[0016] More specifically, according to one aspect of the present disclosure, a mobile image capture device (e.g., a smartphone or tablet) may include an image capture system operable to capture image frames. The mobile image capture device may be configured to perform an operation. The operation may include providing a live video stream for display within a viewfinder portion of a user interface. The live video stream may indicate at least a portion of a current field of view of the image capture system of the mobile image capture device. As an example, the mobile image capture device may include cameras (e.g., forward-facing and / or rearward-facing cameras). The mobile image capture device may be configured to provide a live stream of images captured by one or more of the cameras on the viewfinder portion of the user interface. The mobile image capture device may include a touch-sensitive display screen that displays the user interface.

[0017] The mobile image capture device may be configured to store a video segment from a live video stream in a temporary image buffer. The video segment may include multiple image frames captured by the image capture system. As an example, the video segment may include a moving time window of, for example, a previous time period before the current time. User input may be received directed to a viewfinder portion of the user interface and requesting a rewind operation. The user input may include a user touch action on a touch-sensitive display screen. As an example, a user may swipe, tap, or otherwise touch the touch-sensitive display screen. In response to receiving the user input, the mobile image capture device may perform a rewind operation on the video segment within the viewfinder portion of the user interface.

[0018] During a rewind operation, at least two of the image frames of the video segment may be provided for display in a user interface in reverse chronological order. As one example, at least a portion of the video segment may be played in reverse at a time rate corresponding to a normal forward playback speed. As another example, images of the video segment may be displayed in a manner responsive to continued user input (e.g., via a touchscreen), for example, in a scrubbing operation.

[0019] In some implementations, the mobile image capture device may be configured to receive a user selection of one of the image frames and store the selected image frame in a non-transitory memory location. In other words, the user can navigate through the image frames of a video segment and select which image frame to store for later viewing (e.g., as a photo or video). During a rewind operation, the user can selectively view the image frames of the video segment. When the user views a frame that they want to save as a photo, the user can provide user input requesting that the photo be stored in non-transitory memory. One example of such user input includes tapping a virtual capture button displayed within the user interface.

[0020] In some implementations, the mobile image capture device may allow a user to edit one or more image frames during a rewind operation (e.g., prior to storing them in a non-transitory memory location). The user may perform a user input that requests a cropping or zooming function on the currently displayed image frame. As an example, the user may perform a two-finger user input (e.g., a pinching motion) to control the cropping / zooming function.

[0021] As another example, a user may rotate a mobile image capture device from a first orientation (e.g., portrait orientation) associated with a first image aspect ratio (e.g., portrait aspect ratio) to a second orientation (e.g., landscape orientation) associated with a second image aspect ratio (e.g., landscape aspect ratio). In response, the mobile image capture device may change from displaying the currently displayed image frame in the first image aspect ratio to the second image aspect ratio. In some implementations, the mobile image capture device may capture an image frame having a wide field of view (e.g., using a wide-angle camera). The field of view of the image frame may be larger than the first image aspect ratio and / or the second image aspect ratio (e.g., wider than the landscape aspect ratio and / or taller than the portrait aspect ratio). When displaying an image frame in the portrait orientation, the mobile image capture device may display a portion of the image frame having the portrait aspect ratio (e.g., a portrait-cropped version). Similarly, when displaying an image frame in a landscape orientation, the mobile image capture device may display a portion of the image frame having a landscape aspect ratio (e.g., a landscape cropped version). When the user finishes editing the image frame, the image frame may be stored in a non-transitory memory location. Thus, the user may quickly crop the image frame before storing it for later viewing by simply rotating the mobile image capture device during a rewind operation.

[0022] As yet another example, a user may perform user input requesting adjustment of one or more characteristics of an image frame (e.g., brightness, contrast, color saturation, etc.) prior to storing the image frame in a non-transitory memory location. The mobile image capture device may be configured to allow a user to perform various suitable image or video editing functions on one or more image frames of a video segment prior to storing them in a non-transitory memory location.

[0023] In some implementations, a user can initiate a rewind operation, store one or more image frames, and terminate the rewind operation with a single gesture. As an example, a user can initiate a swipe gesture with their finger on the touchscreen to request a rewind operation. The user can maintain contact between their finger and the touchscreen (e.g., while performing a scrubbing operation) until an image frame they want to save as a photograph is displayed. The user can then remove their finger from the touchscreen to request that the currently displayed image frame be saved as a photograph in a non-transitory memory location. The rewind operation can also be terminated at will by removing their finger from the touchscreen. In response, the mobile image capture device can resume displaying the live video stream within the viewfinder portion of the user interface.

[0024] A user can similarly save portions of a video segment in non-transitory memory during a rewind operation for later viewing. For example, during a rewind operation, a user can mark the beginning and end of a portion that the user wants to save. The user can mark the beginning and end using various user input actions, including, for example, pressing a virtual marker button, momentarily lifting their finger from a touchscreen, or any other suitable gesture or input.

[0025] In some implementations, as described above, a mobile image capture device may be configured to perform a scrubbing operation in response to user input movement, for example, during a rewind operation. The scrubbing operation may be configured to allow a user to navigate among various image frames of a video segment (e.g., in newest and / or forward chronological order). As an example, the user input may have a velocity, and the user interface may transition between displays of different image frames (e.g., in oldest and / or newest) at a rate positively correlated to the rate of the user input. More specifically, the user interface may transition between a first image frame of a video segment to at least a second image frame of the video segment at a rate positively correlated to the rate of the user input. A user can control whether the image frames are displayed in oldest or newest order by controlling the direction of the user input. As an example, a user may swipe left or downward to display image frames in newest order. A user may swipe right or upward to display image frames in oldest order. Thus, during a rewind operation, a user can quickly and intuitively navigate through the image frames of a video segment, for example, to locate one or more of the frames that the user wishes to save.

[0026] In some implementations, a mobile image capture device may be configured to display a thumbnail strip during a rewind operation. The thumbnail strip may include multiple thumbnail images. The thumbnail images may be arranged in chronological order. A current image of the multiple images may be highlighted or marked. The thumbnail strip may include a subset of multiple images of a video segment, and the subset may indicate the relative location of occurrence of the current image within the video segment (e.g., with respect to temporally adjacent image frames). Thus, the thumbnail strip can provide a user with a contextual understanding of the relationship of the current image within the video segment.

[0027] In some implementations, video segments from a live video stream may be stored in a temporary image buffer in a first-in, first-out (FIFO) configuration associated with a moving time window. The moving time window may have an associated time period (e.g., 5 seconds) going back in time from the current moment. In other words, the moving time window may include 5 seconds prior to the current moment. A mobile image capture device may store images from the live video stream in the temporary image buffer. The mobile image capture device may discard (e.g., delete or overwrite) images from the temporary image buffer when the images are older than that time period (e.g., when the images "leave" the moving time window). In this regard, the storage may be represented as a first-in, first-out configuration. Thus, the temporary image buffer may provide temporary storage of video segments for the moving time window.

[0028] In some implementations, the mobile image capture device may be configured to automatically begin storing video segments without a user pressing a capture button. As an example, when a live video stream is provided for display in a viewfinder portion of the interface, video segments may be automatically stored from the live video stream into a temporary image buffer. A user may open a camera application, and in response to the camera application being opened, the mobile image capture device may automatically begin storing video segments (e.g., without further user input).

[0029] As another example, a camera application may be operable in various modes, and video segments may be automatically stored when the camera application is operated in at least two of the modes. When the camera application is first opened, video segments may be automatically stored regardless of whether the camera application is in image capture mode or video capture mode. Thus, a user may open the camera application and point a mobile image capture device at a subject. If the user then misses a moment they intended to capture (e.g., in a video or photo), the user may request a rewind operation to "go back" and capture the missed moment, even if the user did not intend to use the rewind operation when they first opened the camera application.

[0030] In some implementations, the mobile image capture device may be configured to provide the user with one or more suggestions for which image frames of the video segment to save. During a rewind operation, the mobile image capture device may highlight or otherwise draw the user's attention to one or more of the image frames. As an example, a pop-up window may appear to display a preview of one or more image frames suggested for storage. The pop-up window may provide the user with options to automatically store such image frames in the future, view the suggested image frames in a larger window (e.g., in the entire user interface), and / or discard the suggested image frames.

[0031] The mobile image capture device may be configured to select and recommend one or more image frames for storage based on, for example, various photographic characteristics. Examples of such characteristics may include composition, lighting, and context, such as temporal context, associated with the image frame relative to the video segment or adjacent portions of the video segment. If the user consents to the mobile device learning about the user's preferences for such characteristics, the mobile image capture device may select image frames to recommend for storage based on the information learned about the user's preferences for such photographic characteristics.

[0032] Importantly, users may be provided with controls that allow them to make choices both about whether and when the systems, programs, or features described herein may enable the collection of user information (e.g., preferences). Additionally, some data may be treated in one or more ways before being stored or used so that personally identifiable information is removed. For example, a user's identification information may be treated so that personally identifiable information cannot be determined about the user. Thus, users may have control over what information is collected about them, how that information is used, and what information is provided to them.

[0033] In some implementations, a mobile image capture device can utilize a machine learning image selection model to select image frames to suggest to a user for storage. The machine learning image selection model can be configured to receive a plurality of image frames of a video segment. In response to receiving the plurality of image frames, the machine learning image selection model can output a frame selection set representing a selection of at least one of the plurality of image frames. The mobile image capture device can be configured to input the plurality of image frames of the video segment to the machine learning image selection model. The frame selection set can be received as an output of the machine learning image selection model. The mobile image capture device can provide a selection suggestion for display within a user interface that includes at least one of the image frames represented by the frame selection set. Thus, the mobile image capture device can select images via the machine learning image selection model and suggest the selected images for storage by the user.

[0034] In some implementations, if a user consents, the mobile image capture device can automatically capture image frames from a live video stream without receiving any user input. As an example, the mobile image capture device can utilize a machine learning image capture model to select one or more image frames from the live video stream for storage (e.g., in non-transitory memory). The machine learning image capture model may be configured to receive image frames from the live video stream. In response to receiving the image frames, the machine learning capture model may output a capture decision indicating whether to store the image frames (e.g., in non-transitory memory). The mobile image capture device may be configured to sample the live video stream (e.g., at regular intervals) to obtain image frames that are input to the machine learning image capture model. The capture decision may be received as an output of the machine learning image capture model. The machine learning capture model may be configured to select images to capture based on various factors, including, for example, characteristics of the image frames and, if the user consents, user preferences.

[0035] The mobile image capture device can automatically capture images based on the capture decision. Thus, the mobile image capture device can leverage machine learning image capture models to automatically capture image frames from a live video stream without receiving any user input.

[0036] In some implementations, the temporary image buffer may be deleted or cleared at the end of the rewind operation or when a camera application associated with performing the rewind operation is closed. As an example, a user may request a rewind operation and select one or more image frames for storage in a non-temporary memory location. The rewind operation may automatically terminate upon storage of the image frames. Alternatively, the user may request that the rewind operation be terminated, for example, by pressing a virtual “end” button displayed in the user interface or another suitable user input. In response, the mobile image capture device may terminate the rewind operation and resume providing the live video stream for display. The mobile image capture device may clear or overwrite the temporary image buffer at the end of the rewind operation when the live video stream is provided for display. Alternatively, the mobile image capture device may retain the temporary image buffer from the rewind operation and begin storing video segments from the live video stream in a secondary temporary image buffer in anticipation of a second rewind operation. In such implementations, the mobile image capture device may retain temporary image buffers associated with one or more rewind operations until, for example, a camera application associated with the rewind operation is closed, when the camera application is closed, the temporary image buffers may be cleared to free up device resources for subsequent operations.

[0037] According to another aspect of the present disclosure, a mobile image capture device can provide a live video stream for display within a viewfinder portion of a user interface, and the live video stream may include a composite environmental stream generated from multiple sources. The multiple sources may correspond to multiple on-device cameras (e.g., forward-facing and rear-facing cameras). The mobile image capture device may be configured to generate the composite environmental stream by spatially arranging and / or joining (e.g., “stitching”) two or more video streams from the multiple sources. In one example, the composite environmental stream may include a narrow-angle image combined with a portion of a wide-angle image, where the portion of the wide-angle image is smaller than the entire wide-angle image and the portion of the wide-angle image surrounds at least a portion of the periphery of the narrow-angle image. This may enable the composite environmental stream to show a larger environment surrounding a scene shown by only the narrow-angle image. In some implementations, the composite environmental stream may be at least partially panoramic. Thus, in one example, the composite environment stream can show a narrow-angle image captured by a rear-facing camera and also a portion of a wide-angle image captured by a frontward-facing camera (or vice versa), thereby providing an image showing the environment both in front of and behind the mobile image capture device.

[0038] In some implementations, the user interface can present multiple live video streams within the same interface. For example, both a primary live video stream and a secondary live video stream may be presented within the same interface. Multiple video segments corresponding to the multiple live video streams may be stored in one or more temporary buffers. Each live video stream may correspond directly to a camera's field of view, and / or one of the live video streams may be derived from multiple cameras (or subcombinations thereof).

[0039] In some implementations, the primary live video stream may be displayed larger than the secondary live video stream. The primary live video stream may include images from a first source (e.g., a rear-facing camera) and may be displayed within a primary viewing portion of a viewfinder portion of the user interface. The secondary live video stream may include images from a second source (e.g., a forward-facing camera) and may be displayed within a secondary viewing portion of the viewfinder. The primary viewing portion may be larger than the secondary viewing portion. For example, the secondary viewing portion may be overlaid or superimposed on a portion of the primary viewing portion. For example, the secondary viewing portion may be presented within a bubble that is superimposed on a portion of the primary viewing portion.

[0040] The primary and / or secondary live video streams can provide the user with greater contextual information associated with the captured image frames. As one example, the secondary live video stream can provide the user with a view of their facial expression when looking at an object displayed in the primary live video stream. As another example, the secondary live video stream can act as a "rear-view mirror" by providing the user with visual information about their surrounding environment when viewing the primary live video stream.

[0041] The composite environmental stream may have a wide field of view that can provide the user with visual information about the user's environment. The field of view of the composite environmental stream may be larger than the field of view of each of the primary and secondary live streams individually. As an example, the primary and secondary live streams may have complementary or overlapping fields of view. As a result, in some implementations, the composite environmental stream may have a 360-degree field of view. In one example, the composite environmental stream may include a composite of some or all of the images captured by a narrow-angle camera and also some or all of the images captured by a wide-angle camera. For example, a portion of the image captured by the wide-angle camera may be added around the image captured by the narrow-angle camera. Thus, the composite environmental stream can provide the user with visual information about the surrounding environment in addition to the object the user intends to capture.

[0042] A rewind operation may be performed on one or both of the streams. More specifically, a video segment including the composite environmental stream may be stored in a temporary image buffer, and a rewind operation may be performed on the video segment. Alternatively, a primary video segment may be stored from the primary live video stream, and a secondary video segment may be stored from the secondary live video stream. In response to a user input requesting a rewind operation, the mobile image capture device may perform a rewind operation on one or both of the live video streams.

[0043] A user may be able to control which of multiple sources is displayed in the primary viewing portion of the viewfinder portion of the user interface. The user can use this control before and / or during a rewind operation. As an example, a user may perform a user input requesting a change of which source is displayed in the primary viewing portion of the viewfinder. The user may touch the secondary viewing portion of the viewfinder to request that the source displayed in the secondary viewing portion be displayed in the primary viewing portion of the viewfinder. The mobile image capture device may switch between the respective sources displayed in the primary and secondary viewing portions.

[0044] In some implementations, a user may be able to control (e.g., select) which source is displayed in the primary viewing portion during a rewind operation. The user may be able to store video of the composite ambient stream that reflects their selection. More specifically, the video may switch sources during playback at the same time and in the same manner as the user controlled during a rewind operation.

[0045] As an example, a user can point a rear-facing camera of a mobile image capture device at a subject. The forward-facing camera of the mobile image capture device can be pointed at the user. The user can request a rewind operation to view image frames from a stored video segment from a previous time interval (e.g., the previous 5 seconds). The user can scrub through the image frames (at least partially in reverse order) to locate and select a portion of the video segment they want to store as a video in non-transitory memory. The user can then preview the portion of the video segment they want to store, for example, by viewing the portion of the video segment chronologically at a normal time rate. During this preview, the user can select which live video stream (e.g., from the forward-facing camera or the rear-facing camera) to display in the primary viewing portion of the viewfinder. The user can then save the video, in which the video stream displayed in the primary viewing portion of the viewfinder is changed at the same time and in the same way as they controlled during the preview. Alternatively, the user can store a composite video in non-transitory memory that can be similarly controlled during playback at a later time. More specifically, during playback of such composite video, the user can select which source is displayed within the primary viewing portion of the viewfinder portion of the user interface.

[0046] In some implementations, the mobile image capture device may be configured to automatically control (e.g., change or switch) which source is displayed within the primary viewing portion of the viewfinder. For example, the mobile image capture device may be configured to switch between displaying a front-facing camera or a rear-facing camera within the primary viewing portion of the viewfinder. The mobile image capture device may be configured to perform this switching during live viewing (e.g., when providing a live video stream for display within the viewfinder portion of the user interface) and / or during a rewind operation. Additionally or alternatively, the mobile image capture device may be configured to perform this switching during playback of a composite video stored in non-transitory memory.

[0047] As an example, a mobile image capture device can utilize a machine learning source selection model to select which source is displayed within the primary viewing portion of a viewfinder. The machine learning source selection model can be configured to receive a first set of image frames from a first video source and a second set of image frames from a second video source. The first and second video sources may correspond to respective live video streams or stored videos of the respective live video streams. The first and second video sources may correspond to live video streams from a forward-facing camera and a rear-facing camera. In response to receiving the first and second video sources, the machine learning source selection model can output a source selection output representing a time-aligned instruction of which source to display within the primary viewing portion of the viewfinder (e.g., when to switch from displaying the video stream from the forward-facing camera to the video stream from the rear-facing camera). The mobile image capture device can be configured to input images from multiple sources (e.g., live video streams or stored videos) into the machine learning source selection model. The source selection output can be received as an output of the machine learning source selection model. The mobile image capture device may control the display of the source in the primary viewing portion and / or the secondary viewing portion based on the source selection output.

[0048] Alternatively, the mobile image capture device can provide suggestions to the user regarding which sources to display within the primary viewing portion at which times. Such suggestions can be associated with each image frame or each time within a video segment. Such suggestions can be provided during rewind operations and / or while watching a stored composite video.

[0049] The systems and methods of the present disclosure provide several technical effects and benefits. As one example, the systems and methods described herein can perform rewind operations using minimal computational resources, which can result in faster and more efficient execution compared to capturing video, storing it in non-transitory memory, and then reviewing the stored video for image frames and / or video segments to extract. For example, in some implementations, the systems and methods described herein can be quickly and efficiently executed on a user computing device, such as a smartphone, due to reduced computational demands. Thus, aspects of the present disclosure can improve the accessibility and effectiveness of video capture using such devices, for example, in scenarios where cloud computing is unavailable or otherwise undesirable (e.g., due to improved user privacy and / or reduced communication costs).

[0050] In this manner, the systems and methods described herein can provide more efficient operations for mobile image capture. Capturing and storing segments of video in temporary memory can improve the efficiency with which specific images can be extracted and stored in non-transitory memory. Specifically, the capture of short-lived and / or unpredictable events, such as capturing laughs or smiles, or capturing sporting or weather events, can be improved. Thus, the systems and methods described herein avoid image capture operations that are less efficient, such as burst photography, or that require additional equipment, such as external sound / motion triggers.

[0051] As one example, the systems and methods of the present disclosure may be included or otherwise employed within the context of an application, a browser plug-in, or other context. Thus, in some implementations, the models of the present disclosure may be included within or otherwise stored in and implemented by a user computing device, such as a laptop, tablet, or smartphone. As yet another example, the models may be included within or otherwise stored in and implemented by a server computing device that communicates with the user computing device according to a client-server relationship. For example, the models may be implemented by the server computing device as part of a web service (e.g., a web email service).

[0052] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the figures.

[0053] Exemplary Devices and Systems 1A illustrates a block diagram of an exemplary computing system 100 for performing a rewind operation, according to an exemplary embodiment of the present disclosure. System 100 includes a user computing device 102, a server computing system 130, and a training computing system 150, communicatively coupled over a network 180.

[0054] The user computing device 102 is typically a mobile image capture device such as a smartphone or tablet. In other implementations, the user computing device 102 may be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0055] The user computing device 102 includes one or more processors 112 and memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple operatively connected processors. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing device 102 to perform operations. The memory 114 may also include non-transitory memory locations 120 and a temporary image buffer 122. The temporary image buffer 122 may correspond to a non-transitory computer-readable storage medium suitable for temporary storage of information, such as, for example, RAM. The non-transitory memory locations 120 may correspond to non-transitory computer-readable storage media suitable for non-transitory storage of information, such as flash memory devices, magnetic disks, and the like.

[0056] The user computing device 102 may store or include one or more machine learning image selection models 123, machine learning image capture models 124, and / or machine learning source selection models 125. For example, the machine learning models 123, 124, 125 may be or otherwise include various machine learning models, such as neural networks (e.g., deep neural networks) or other multi-layer nonlinear models. The neural networks may include recurrent neural networks (e.g., long short-term memory recurrent neural networks), feedforward neural networks, or other forms of neural networks. Exemplary models 123, 124, 125 are described with reference to FIGS. 2-4.

[0057] In some implementations, one or more models 123, 124, 125 may be received from server computing system 130 over network 180, stored in user computing device memory 114, and used or otherwise implemented by one or more processors 112. In some implementations, user computing device 102 may implement multiple parallel instances of one or more of models 123, 124, 125 (e.g., to perform parallel operations across multiple instances of models 123, 124, 125).

[0058] Additionally or alternatively, one or more models 140, 142, 144 may be included within or otherwise stored in and implemented by a server computing system 130 that communicates with the user computing device 102 according to a client-server relationship. For example, one or more models 140, 142, 144 may be stored and implemented at the user computing device 102 and / or one or more models 140 may be stored and implemented at the server computing system 130.

[0059] The user computing device 102 may also include one or more user input components 126 that receive user input. For example, the user input component 126 may be a touch-sensitive component (e.g., a touch-sensitive display screen or a touchpad) that is sensitive to the touch of a user input object (e.g., a finger or stylus). The touch-sensitive component may serve to implement a virtual keyboard. Other exemplary user input components include a microphone, a traditional keyboard, or other means by which a user can enter communications.

[0060] The user computing device 102 may also include one or more cameras 128. For example, the user computing device 102 may include a forward-facing camera and / or a rear-facing camera. As an example, the user computing device 102 may correspond to a smartphone, and the rear-facing camera may be positioned adjacent to the smartphone's display such that when the user is holding the smartphone and looking at the display, the rear-facing camera faces the user behind. Similarly, the front-facing camera may be positioned such that when the user is holding the smartphone and looking at the display, the front-facing camera faces away from the user. The user computing device 102 may include any number of different types and arrangements of cameras 128 with a variety of different characteristics. In one example, the device 102 may have multiple forward-facing cameras and / or multiple rear-facing cameras. The cameras 128 may be narrow-angle cameras, wide-angle cameras, or a combination thereof. The cameras 128 may have different filters and / or be capable of accepting different wavelengths of light (e.g., one infrared camera and one visible light spectrum camera). In one example, device 102 may have a first rear-facing camera (e.g., with a wide-angle lens and / or an f / 1.8 aperture), a second rear-facing camera (e.g., with a telephoto lens and / or an f / 2.4 aperture), and a front-facing camera (e.g., with a wide-angle lens and / or an f / 2.2 aperture). In another specific example, device 102 may include the following cameras: a rear-facing camera (e.g., with 12.2 megapixels, laser autofocus, and / or dual pixel phase detection), a first front-facing camera (e.g., 8.1 megapixels and / or an f / 1.8 aperture), and a second front-facing camera (e.g., 8.1 megapixels, a wide-angle lens, and / or variable f / 1.8 and f / 2.2 apertures).

[0061] The server computing system 130 may include one or more processors 132 and memory 134. The one or more processors 132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple operatively connected processors. The memory 134 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 may store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0062] In some implementations, server computing system 130 includes or is otherwise implemented by one or more server computing devices. In instances in which server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a serial computing architecture, a parallel computing architecture, or some combination thereof.

[0063] As described above, the server computing system 130 may store or otherwise include one or more machine learning models 140, 142, 144. For example, the models 140, 142, 144 may be or otherwise include various machine learning models, such as neural networks (e.g., deep recurrent neural networks) or other multi-layer nonlinear models. Exemplary models 140, 142, 144 are described with reference to FIGS. 2-4.

[0064] The server computing system 130 can train the model 140 through interaction with a training computing system 150 that is communicatively coupled over a network 180. The training computing system 150 can be separate from the server computing system 130 or can be part of the server computing system 130.

[0065] The training computing system 150 includes one or more processors 152 and memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple operatively connected processors. The memory 154 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 may store data 156 and instructions 158 that are executed by the processor 152 to cause the training computing system 150 to perform operations. In some implementations, the training computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0066] The training computing system 150 may include a model trainer 160 that trains one or more of the machine learning models 140, 142, 144 stored at the server computing system 130 using various training or learning techniques, such as, for example, backward propagation of errors. In some implementations, performing backpropagation may include performing truncated backpropagation through time. The model trainer 160 may perform several generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization capability of the model being trained.

[0067] Specifically, model trainer 160 can train one or more of models 140, 142, 144 based on a set of training data 162. Training data 162 can include labeled or unlabeled sets of model inputs and / or outputs, for example, as described below with reference to FIGS.

[0068] In some implementations, if the user provides consent, training examples may be provided by the user computing device 102 (e.g., based on communications previously provided by the user of the user computing device 102). Thus, in such implementations, the models 140, 142, 144 provided to the user computing device 102 may be trained by the training computing system 150 on user-specific communication data received from the user computing device 102. In some instances, this process may be referred to as model individualization. For example, one or more of the models 140, 142, 144 may be trained based on user preferences (e.g., for photo characteristics).

[0069] Model trainer 160 includes computer logic utilized to provide desired functionality. Model trainer 160 may be implemented in hardware, firmware, and / or software controlling a general-purpose processor. For example, in some implementations, model trainer 160 includes program files stored on a storage device, loaded into memory, and executed by one or more processors. In other implementations, model trainer 160 includes one or more sets of computer-executable instructions stored in a tangible computer-readable storage medium, such as RAM, a hard disk, or optical or magnetic media.

[0070] Network 180 may be any type of communications network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. In general, communications on network 180 may be carried over any type of wired and / or wireless connection using a wide variety of communications protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0071] 1A illustrates one exemplary computing system that may be used to implement the present disclosure. Other computing systems may be used as well. For example, in some implementations, a user computing device 102 may include a model trainer 160 and a training dataset 162. In such implementations, both the training and use of the models 140, 142, 144 may occur locally at the user computing device 102. In some such implementations, the user computing device 102 may implement the model trainer 160 to individualize the models 140, 142, 144 based on user-specific data.

[0072] 1B illustrates a block diagram of an exemplary computing device 10 implemented in accordance with an exemplary embodiment of the present disclosure. Computing device 10 may be a user computing device or a server computing device.

[0073] Computing device 10 includes several applications (e.g., applications 1-N). Each application includes its own machine learning library and machine learning model. For example, each application may include a machine learning model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc.

[0074] 1B , each application may communicate with several other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application may communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0075] 1C illustrates a block diagram of an exemplary computing device 50, implemented in accordance with an exemplary embodiment of the present disclosure. Computing device 50 may be a user computing device or a server computing device.

[0076] Computing device 50 includes several applications (e.g., applications 1-N). Each application is in communication with a central intelligence layer. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and models stored therein) using an API (e.g., a common API across all applications).

[0077] The central intelligence layer includes several machine learning models. For example, as shown in FIG. 1C , each machine learning model (e.g., model) may be provided for each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine learning model. For example, in some implementations, the central intelligence layer may provide a single model (e.g., a single model) for all of the applications. In some implementations, the central intelligence layer is included within or otherwise implemented by the operating system of computing device 50.

[0078] The central intelligence layer can communicate with a central device data layer. The central device data layer can be a centralized repository of data for the computing device 50. As shown in FIG. 1C , the central device data layer can communicate with several other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0079] Example Model Configuration 2 shows a block diagram of an exemplary machine learning image selection model 200, according to an exemplary embodiment of the present disclosure. In some implementations, the machine learning image selection model 200 may be configured to receive a plurality of image frames 202 (e.g., of a video segment). In response to receiving the plurality of image frames 202, the machine learning image selection model 200 may output a frame selection set 204 representing a selection of at least one of the plurality of image frames 202.

[0080] 3 shows a block diagram of an example machine learning image capture model 300. The machine learning image capture model 300 may be configured to receive an image frame 302 (e.g., from a live video stream). In response to receiving the image frame 302, the machine learning capture model may output a capture decision 304 indicating whether to store the image frame (e.g., in the non-transient memory location 120 described above with reference to FIG. 1A).

[0081] 4 shows a block diagram of an example machine learning source selection model 400 for selecting which source is displayed within a primary viewing portion of a viewfinder portion of a user interface, e.g., as described below with reference to FIG. 6. The machine learning source selection model 400 may be configured to receive a first set of image frames from a first video source 402 and a second set of image frames from a second video source 404. The first video source 402 and the second video source 404 may correspond to respective live video streams or stored video of respective live video streams. As an example, the first video source 402 and the second video source 404 may correspond to live video streams from forward-facing and rear-facing cameras 128, respectively. In response to receiving the first video source 402 and the second video source 404, the machine learning source selection model 400 may output a source selection output 406 representing a time-aligned instruction of which source to display within the primary viewing portion of the viewfinder (e.g., when to switch from displaying the video stream from the front-facing camera to the video stream from the rear-facing camera), for example, as described below with reference to FIG. 6 .

[0082] Generally, the present disclosure is directed to systems and methods that provide an improved user experience for capturing images and / or videos with a handheld computing device, such as a smartphone or tablet. The systems and methods described herein can provide a user with the ability to "rewind" time and capture video or images from an earlier moment in time when the user failed to explicitly operate the device to capture it. This functionality can enable a user to capture the "perfect shot" even after that moment has passed.

[0083] 5A and 5B, in one example, a user can point the camera of a handheld device 502 at a subject of interest and view a live stream from the camera in a viewfinder portion 504 of a display 505 of the device 502. In this example, the subject of interest is a woman throwing a hat over a man's head. The device 502 can temporarily store video segments (e.g., a set of captured images) from the live video stream in a temporary image buffer 122 (FIG. 1A). Rather than attempting to press a capture button 508 (also known as a "shutter release button") at the exact moment the user wants to capture, the user can watch until the moment has passed. The user can then request a rewind operation by providing user input (e.g., a swipe or other motion or gesture, as indicated by the user's hand 506 contacting the display screen 505). In response, the device 502 can immediately (or quickly) provide the user with the ability to "rewind time" (as shown in FIG. 5B) to review a previous moment displayed on the viewfinder portion 504. Previous moments may be displayed in reverse chronological order, and / or the user may control playback, for example, by scrolling through the video segment in a scrubbing operation. The user may select one or more of the images, for example, as a photo or video that captures the "perfect shot" that the user missed, and the selected images may be saved in a non-transitory memory location 120 (FIG. 1A) of device 102. In that way, the proposed system allows the user to capture the "perfect shot" even after the moment has passed, but without cluttering the user's camera roll with unwanted images or requiring the user to manually go through and delete numerous unwanted photos.

[0084] More specifically, with reference to FIG. 5C , according to one aspect of the present disclosure, a mobile image capture device (e.g., a smartphone or tablet) may include an image capture system operable to capture image frames. In FIG. 5C , the user interface of the mobile image capture device is shown in four different chronological frames 552, 554, 558, and 562. The mobile image capture device may be configured to perform an operation. With reference to the first frame 552, the operation may include providing a live video stream for display within the viewfinder portion 504 of the user interface. The live video stream may show at least a portion of the current field of view of the image capture system of the mobile image capture device. As an example, the mobile image capture device may include cameras, such as forward-facing and / or rear-facing cameras (e.g., corresponding to camera 128 described above with reference to FIG. 1A ). The mobile image capture device may be configured to provide a live stream of images captured by one or more of the cameras on the viewfinder portion 504 of the user interface. The mobile image capture device may include a touch-sensitive display screen that displays the user interface.

[0085] The mobile image capture device may be configured to store video segments from a live video stream in temporary image buffer 122 (FIG. 1A). A video segment may include multiple image frames captured by the image capture system. As an example, a video segment may include a moving time window, e.g., of a previous time period before the current time.

[0086] Referring to frame 554 of FIG. 5C , user input may be received directed toward the viewfinder portion 504 of the user interface and requesting a rewind operation. The user input may include a user touch action on the touch-sensitive display screen. For example, the user input is illustrated in frame 554 of FIG. 5C by the dotted outline of a user's hand and accompanying arrow 556, which represents a swipe input on the touch-sensitive display screen. As another example, the user taps or otherwise touches the touch-sensitive display screen. In response to receiving the user input, the mobile image capture device may perform a rewind operation on the video segment within the viewfinder portion of the user interface.

[0087] 5C , during a rewind operation, at least two of the image frames of the video segment may be provided for display within a user interface in reverse chronological order. As one example, at least a portion of the video segment may be played in reverse at a time rate corresponding to a normal forward playback speed. As another example, images of the video segment may be displayed in a manner responsive to continued user input (e.g., via a touchscreen), for example, in a scrubbing operation (indicated by a dotted outline of a user's hand and an accompanying set of arrows pointing left and right 560).

[0088] A user can navigate among various image frames of a video segment (e.g., in reverse chronological order and / or reverse chronological order) via a scrubbing operation (represented by arrow 560). As an example, the user input can have a velocity, and the user interface can transition between displaying different image frames (e.g., in reverse chronological order and / or reverse chronological order) at a rate that is positively correlated to the rate of the user input. More specifically, the user interface can transition from a first image frame of a video segment to at least a second image frame of the video segment at a rate that is positively correlated to the rate of the user input. A user can control whether the image frames are displayed in reverse chronological order or reverse chronological order by controlling the direction of the user input. As an example, a user can swipe left or downward to display image frames in reverse chronological order. A user can swipe right or upward to display image frames in reverse chronological order. Thus, during a rewind operation, a user can quickly and intuitively navigate through the image frames of a video segment, for example, to locate one or more of the frames they wish to save.

[0089] In some implementations, the mobile image capture device may be configured to display a thumbnail strip 510 during a rewind operation. The thumbnail strip 510 may include multiple thumbnail images. The thumbnail images may be arranged in chronological order. A current image 518 of the multiple images may be highlighted or otherwise marked. The thumbnail strip 510 may include a subset of the multiple images of a video segment, and the subset may indicate the relative location of occurrence of the current image within the video segment (e.g., with respect to temporally adjacent image frames). Thus, the thumbnail strip 510 can provide a user with a contextual understanding of the relationship of the current image within the video segment.

[0090] Referring to frame 562 of FIG. 5C , in some implementations, the mobile image capture device may be configured to receive a user selection of one of the image frames and store the selected image frame in non-transitory memory location 120 ( FIG. 1A ). In other words, a user can navigate through the image frames of a video segment and select which image frames to store for later viewing (e.g., as a photo or video). During a rewind operation, a user can selectively view image frames of a video segment. When a user views a frame that they want to save as a photo, the user can provide user input requesting that the photo be stored in non-transitory memory (e.g., non-transitory memory location 120 described with reference to FIG. 1A ). One example of such user input includes tapping a virtual capture button 508 displayed in the user interface. Additionally, in some implementations, a user can save all temporarily stored images in non-transitory memory location 120 by, for example, tapping a “save all” button 568. The mobile image capture device may provide a notification 570 in the user interface indicating that the images have been saved.

[0091] Referring to FIG. 5D , in some implementations, a mobile image capture device may allow a user to edit one or more image frames during a rewind operation (e.g., prior to storing them in non-transitory memory). FIG. 5D includes multiple frames 580, 586, 588 of a mobile image capture device at various stages of editing an image of a man lifting a girl to “dunk” a basketball. Referring to frame 580 of FIG. 5D , a user may perform a user input requesting a cropping or zooming function on the currently displayed image frame. As an example, a user may perform a two-finger “pinching motion” user input (represented by a series of circles 582 in frame 580 of FIG. 5D ) to control the cropping / zooming function. More specifically, the mobile image capture device may save an image file corresponding to the portion of the image displayed within the user interface to non-transitory memory location 120. A user may request that the image capture device save the image by tapping capture button 508.

[0092] 5D illustrate another technique for editing one or more image frames during a rewind operation. A user may rotate a mobile image capture device from a first orientation, e.g., as shown in frame 586, to a second orientation, e.g., as shown in frame 588. The first orientation (e.g., portrait orientation) may be associated with a first image aspect ratio (e.g., portrait aspect ratio). The second orientation (e.g., landscape orientation) may be associated with a second image aspect ratio (e.g., landscape aspect ratio). In response, the mobile image capture device may change from displaying the image frame currently displayed in the first image aspect ratio to the second image aspect ratio. The user may then save the image as currently displayed in the viewfinder, e.g., by tapping capture button 508.

[0093] In some implementations, the mobile image capture device can capture image frames having a wide field of view (e.g., using a wide-angle camera). The field of view of the image frame can be larger than the first image aspect ratio and / or the second image aspect ratio (e.g., wider than a landscape aspect ratio and / or taller than a portrait aspect ratio). When displaying an image frame in a portrait orientation, the mobile image capture device can display a portion of the image frame having a portrait aspect ratio (e.g., a cropped portrait version). Similarly, when displaying an image frame in a landscape orientation, the mobile image capture device can display a portion of the image frame having a landscape aspect ratio (e.g., a cropped landscape version). When a user finishes editing an image frame, the image frame can be stored in non-transitory memory, for example, in response to the user tapping the capture button 508. Thus, a user can quickly crop an image frame before storing it for later viewing by simply rotating the mobile image capture device during a rewind operation. Thus, in some implementations, entire image frames may be stored in a temporary image buffer, allowing the user to review / modify and edit spatially within such image frames after the fact during a rewind operation.

[0094] As yet another example, a user may perform user input requesting adjustment of one or more characteristics of an image frame (e.g., brightness, contrast, color saturation, etc.) prior to storing the image frame in non-transitory memory. The mobile image capture device may be configured to allow a user to perform various suitable image or video editing functions on one or more image frames of a video segment prior to storing them in a non-transitory memory location.

[0095] In some implementations, a user can initiate a rewind operation, store one or more image frames, and terminate the rewind operation with a single gesture. As an example, a user can initiate a swipe gesture with their finger on the touchscreen to request a rewind operation. The user can maintain contact between their finger and the touchscreen (e.g., while performing a scrubbing operation) until an image frame they want to save as a photograph is displayed. The user can then remove their finger from the touchscreen to request that the currently displayed image frame be saved as a photograph in non-transitory memory. The rewind operation can also be terminated at will by removing their finger from the touchscreen. In response, the mobile image capture device can resume displaying the live video stream within the viewfinder portion of the user interface.

[0096] A user can similarly save portions of a video segment in non-transitory memory during a rewind operation for later viewing. For example, during a rewind operation, a user can mark the beginning and end of a portion that the user wants to save. The user can mark the beginning and end using various user input actions, including, for example, pressing a virtual marker button, momentarily lifting their finger from a touchscreen, or any other suitable gesture or input.

[0097] In some implementations, video segments from a live video stream may be stored in a temporary image buffer 122 (FIG. 1A) in a first-in, first-out (FIFO) configuration associated with a moving-time window. The moving-time window may have an associated time period (e.g., 5 seconds) going back in time from the current moment. In other words, the moving-time window may include the previous 5 seconds from the current moment. The mobile image capture device may store images from the live video stream in the temporary image buffer 122 (FIG. 1A). The mobile image capture device may discard (e.g., delete or overwrite) images from the temporary memory (e.g., the temporary image buffer 122 described above with reference to FIG. 1A) when such images become older than that time period (e.g., when the images "leave" the moving-time window). In this regard, the storage may be represented as a first-in, first-out configuration. Thus, the temporary image buffer 122 (FIG. 1A) may provide temporary storage of video segments of a moving-time window.

[0098] In some implementations, the mobile image capture device may be configured to automatically begin storing video segments without a user requesting such storage, for example, by pressing a capture button (e.g., capture button 508 described above with reference to FIGS. 5A-5D). As an example, when a live video stream is provided for display within a viewfinder portion of the interface, video segments may be automatically stored from the live video stream into temporary image buffer 122 (FIG. 1A). A user may open a camera application, and in response to the camera application being opened, the mobile image capture device may automatically begin storing video segments (e.g., without further user input).

[0099] As another example, a camera application may be operable in various modes, and video segments may be automatically stored when the camera application is operated in at least two of the modes. When the camera application is first opened, video segments may be automatically stored regardless of whether the camera application is in image capture mode or video capture mode. Thus, a user may open the camera application and point a mobile image capture device at a subject. If the user then misses a moment they intended to capture (e.g., in a video or photo), the user may request a rewind operation to "go back" and capture the missed moment, even if the user did not intend to use the rewind operation when they first opened the camera application.

[0100] Referring to FIG. 5E , in some implementations, a mobile image capture device may be configured to provide a user with one or more suggestions for which image frames of a video segment to save. Frame 590 of FIG. 5E shows a mobile image capture device during a rewind operation. As the user reviews images for potential saving, the mobile image capture device may highlight or otherwise draw the user's attention to one or more of the image frames. As an example, referring to frame 592 of FIG. 5E , a pop-up window 594 may appear to display a preview of one or more image frames 596 suggested for storage. The pop-up window 594 may provide the user with options to automatically store such image frames in the future, view the suggested image frames in a larger window (e.g., in the entire user interface), save the suggested image frames, and / or discard the suggested image frames.

[0101] The mobile image capture device may be configured to select and recommend one or more image frames for storage based on, for example, various photographic characteristics. Examples of such characteristics may include composition, lighting, and context, such as temporal context, associated with the image frame relative to the video segment or adjacent portions of the video segment. If the user consents to the mobile device learning about the user's preferences for such characteristics, the mobile image capture device may select image frames to recommend for storage based on the information learned about the user's preferences for such photographic characteristics.

[0102] Importantly, users may be provided with controls that allow them to make choices both about whether and when the systems, programs, or features described herein may enable the collection of user information (e.g., preferences). Additionally, some data may be treated in one or more ways before being stored or used so that personally identifiable information is removed. For example, a user's identification information may be treated so that personally identifiable information cannot be determined about the user. Thus, users may have control over what information is collected about them, how that information is used, and what information is provided to them.

[0103] In some implementations, a mobile image capture device can utilize a machine learning image selection model, for example, as described with reference to FIG. 2, to select image frames to suggest to a user for storage. Referring again to FIG. 2, the mobile image capture device can be configured to input a plurality of image frames 202 of a video segment into the machine learning image selection model 200. A frame selection set 204 can be received as an output of the machine learning image selection model 200. The mobile image capture device can provide a selection suggestion for display within a user interface (e.g., within the pop-up window 594 described above) that includes at least one of the image frames represented by the frame selection set 204. Thus, the mobile image capture device can select images via the machine learning image selection model 200 and suggest the selected images for storage by the user.

[0104] In some implementations, if a user consents, the mobile image capture device can automatically capture image frames from a live video stream without receiving any user input. The mobile image capture device can utilize a machine learning image capture model 300, e.g., as described with reference to FIG. 3, to select one or more image frames from the live video stream for storage (e.g., in non-transitory memory location 120). For example, referring again to FIG. 3, the machine learning image capture model can be configured to receive an image frame 302 from the live video stream. In response to receiving the image frame 302, the machine learning capture model 300 can output a capture decision 304 indicating whether to store the image frame (e.g., in non-transitory memory location 120). The mobile image capture device can be configured to sample the live video stream (e.g., at regular intervals) to obtain image frames 302 that are input to the machine learning image capture model 300. The capture decision can be received as an output of the machine learning image capture model 300. The machine learning capture model 300 may be configured to select images for capture based on various factors, including, for example, characteristics of the image frame and, if the user consents, user preferences.

[0105] The mobile image capture device can automatically capture images based on a capture decision. Thus, the mobile image capture device can leverage the machine learning image capture model 300 to automatically capture image frames from a live video stream without receiving any user input.

[0106] In some implementations, the temporary image buffer 122 may be deleted or cleared at the end of the rewind operation or when a camera application associated with performing the rewind operation is closed. As an example, a user may request a rewind operation and select one or more image frames for storage in the non-transitory memory location 120. The rewind operation may automatically terminate upon storage of the image frames. Alternatively, the user may request that the rewind operation be terminated by, for example, pressing a virtual “end” button that may be displayed within the user interface (e.g., represented by “X” 598 in FIG. 5E ). In response, the mobile image capture device may terminate the rewind operation and resume providing the live video stream for display. The mobile image capture device may clear or overwrite the temporary image buffer 122 at the end of the rewind operation when the live video stream is provided for display. Alternatively, the mobile image capture device may retain the temporary image buffer 122 from the rewind operation and begin storing video segments from the live video stream in a secondary temporary image buffer in anticipation of a second rewind operation. In such implementations, the mobile image capture device may retain temporary image buffers associated with one or more rewind operations until, for example, a camera application associated with the rewind operation is closed, when the camera application is closed, the temporary image buffers may be cleared to free up device resources for subsequent operations.

[0107] 6, according to another aspect of the present disclosure, a mobile image capture device can provide a live video stream for display within a viewfinder portion of a user interface, where the live video stream may include a composite environmental stream generated from multiple sources. The multiple sources may correspond to multiple on-device cameras (e.g., forward-facing and rear-facing cameras). The mobile image capture device may be configured to generate the composite environmental stream by spatially arranging and / or joining (e.g., "stitching") two or more video streams from the multiple sources.

[0108] In some implementations, the user interface can present multiple live video streams within the same interface. For example, both a primary live video stream and a secondary live video stream may be presented within the same interface. Multiple video segments corresponding to the multiple live video streams may be stored in one or more temporary buffers. Each live video stream may correspond directly to a camera's field of view, and / or one of the live video streams may be derived from multiple cameras (or subcombinations thereof).

[0109] In some implementations, the primary live video stream may be displayed larger than the secondary live video stream. The primary live video stream may include images from a first source (e.g., a rear-facing camera) and may be displayed within the primary viewing portion 604 of the viewfinder portion of the user interface. The secondary live video stream may include images from a second source (e.g., a forward-facing camera) and may be displayed within the secondary viewing portion 606 of the viewfinder. The primary viewing portion 604 may be larger than the secondary viewing portion 606. For example, the secondary viewing portion 606 may be overlaid or superimposed on a portion of the primary viewing portion 604. For example, the secondary viewing portion 606 may be presented within a bubble that is superimposed on a portion of the primary viewing portion 604.

[0110] The primary and / or secondary live video streams can provide the user with greater contextual information associated with the captured image frames. As one example, the secondary live video stream can provide the user with a view of their facial expression when looking at an object displayed in the primary live video stream, e.g., as shown in first frame 602 of Figure 6. As another example, the secondary live video stream (in secondary viewing portion 606) can act as a "rear-view mirror" by providing the user with visual information about their surrounding environment when viewing the primary live video stream, e.g., as shown in second frame 608 of Figure 6.

[0111] The composite environmental stream may have a wide field of view that can provide the user with visual information about the user's environment. The field of view of the composite environmental stream may be larger than the field of view of each of the primary and secondary live streams individually. As an example, the primary and secondary live streams may have complementary or overlapping fields of view. As a result, in some implementations, the composite environmental stream may have a 360-degree field of view. In one example, the composite environmental stream may include a composite of some or all of the images captured by a narrow-angle camera and also some or all of the images captured by a wide-angle camera. For example, a portion of the image captured by the wide-angle camera may be added around the image captured by the narrow-angle camera. Thus, the composite environmental stream can provide the user with visual information about the surrounding environment in addition to the object the user intends to capture.

[0112] A rewind operation may be performed on one or both of the streams. More specifically, a video segment including the composite environmental stream may be stored in a temporary image buffer, and a rewind operation may be performed on the video segment. Alternatively, a primary video segment may be stored from the primary live video stream, and a secondary video segment may be stored from the secondary live video stream. In response to a user input requesting a rewind operation, the mobile image capture device may perform a rewind operation on one or both of the live video streams.

[0113] Referring to frame 608 of FIG. 6 , a user may be able to control which of multiple sources is displayed in the primary viewing portion 604 of the viewfinder portion of the user interface. The user can use this control before and / or during a rewind operation. As an example, a user can perform a user input requesting a change in which source is displayed in the primary viewing portion of the viewfinder. The user can touch the secondary viewing portion 606 of the viewfinder to request that the source displayed in the secondary viewing portion be displayed in the primary viewing portion of the viewfinder, for example, as shown by the dotted outline 610 of the user's hand in frame 608 of FIG. 6 . The mobile image capture device can switch between the respective sources displayed in the primary viewing portion 604 and the secondary viewing portion 606, for example, as shown in frame 612 of FIG. 6 .

[0114] In some implementations, a user may be able to control (e.g., select) which source is displayed in the primary viewing portion during a rewind operation. The user may be able to store video of the composite ambient stream that reflects their selection. More specifically, the video may switch sources during playback at the same time and in the same manner as the user controlled during a rewind operation.

[0115] As an example, a user can point a rear-facing camera of a mobile image capture device at a subject. The forward-facing camera of the mobile image capture device can be pointed at the user. The user can request a rewind operation to view image frames from a stored video segment from a previous time interval (e.g., the previous 5 seconds). The user can scrub through the image frames (at least partially in reverse order), for example, as described above with reference to frame 558 of FIG. 5C , to locate and select a portion of the video segment they want to store as video in non-transitory memory. The user can then preview the portion of the video segment they want to store, for example, by viewing the portion of the video segment chronologically at a normal time rate. During this preview, the user can select which live video stream (e.g., from the forward-facing camera or the rear-facing camera) to display in the primary viewing portion 604 of the viewfinder. The user can then save the video, in which the video stream displayed in the primary viewing portion 604 of the viewfinder is changed at the same time and in the same manner as they controlled during the preview. Alternatively, the user can store a composite video in non-transitory memory that can be similarly controlled during playback at a later time. More specifically, during playback of such a composite video, the user can select which sources are displayed in the primary viewing portion 604 of the viewfinder portion of the user interface.

[0116] In some implementations, the mobile image capture device may be configured to automatically control (e.g., change or switch) which source is displayed in the viewfinder's primary viewing portion 604. For example, the mobile image capture device may be configured to switch between displaying a front-facing camera or a rear-facing camera in the viewfinder's primary viewing portion 604. The mobile image capture device may be configured to perform this switching during live viewing (e.g., when providing a live video stream for display in the viewfinder portion of the user interface) and / or during a rewind operation. Additionally or alternatively, the mobile image capture device may be configured to perform this switching during playback of a composite video stored in non-transitory memory.

[0117] As an example, a mobile image capture device may utilize a machine learning source selection model, such as that described with respect to FIG. 4, to select which source is displayed within the primary viewing portion 604 of the viewfinder. Referring again to FIG. 4, the machine learning source selection model 400 may be configured to receive a first set of image frames from a first video source 402 and a second set of image frames from a second video source 404. The first video source 402 and the second video source 404 may correspond to respective live video streams or stored video of respective live video streams. The first video source 402 and the second video source 404 may correspond to live video streams from a forward-facing camera and a rear-facing camera. In response to receiving the first video source 402 and the second video source 404, the machine learning source selection model 400 can output a source selection output 406 that represents a time-aligned instruction of which source to display in the primary viewing portion 604 of the viewfinder (e.g., when to switch between displaying the video stream from the front-facing camera and the video stream from the rear-facing camera). The mobile image capture device may be configured to input images from multiple sources (e.g., live video streams or stored videos) to the machine learning source selection model 400. The source selection output 406 may be received as an output of the machine learning source selection model 400. Referring again to FIG. 6 , the mobile image capture device can control the display of sources in the primary viewing portion 604 and / or the secondary viewing portion 606 based on the source selection output 406.

[0118] Alternatively, the mobile image capture device can provide suggestions to the user regarding which sources to display at which times in the primary viewing portion 604. Such suggestions can be associated with each image frame or each time within a video segment. Such suggestions can be provided during a rewind operation and / or while watching a stored composite video.

[0119] The systems and methods of the present disclosure provide several technical effects and benefits. As one example, the systems and methods described herein can perform rewind operations using minimal computational resources, which can result in faster and more efficient execution compared to capturing video, storing it in non-transitory memory, and then reviewing the stored video for image frames and / or video segments to extract. For example, in some implementations, the systems and methods described herein can be quickly and efficiently executed on a user computing device, such as a smartphone, due to reduced computational demands. Thus, aspects of the present disclosure can improve the accessibility and effectiveness of video capture using such devices, for example, in scenarios where cloud computing is unavailable or otherwise undesirable (e.g., due to improved user privacy and / or reduced communication costs).

[0120] In this manner, the systems and methods described herein can provide more efficient operations for mobile image capture. Capturing and storing segments of video in temporary memory can improve the efficiency with which specific images can be extracted and stored in non-transitory memory. Specifically, the capture of short-lived and / or unpredictable events, such as capturing laughs or smiles, or capturing sporting or weather events, can be improved. Thus, the systems and methods described herein avoid image capture operations that are less efficient, such as burst photography, or that require additional equipment, such as external sound / motion triggers.

[0121] Exemplary Methods 7 shows a flowchart diagram of an exemplary method for performing a rewind operation, according to an exemplary embodiment of the present disclosure. While FIG. 7 shows steps performed in a particular order for purposes of illustration and explanation, the methods of the present disclosure are not limited to the specifically shown order or arrangement. Various steps of method 700 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0122] At 702, method 700 may include providing a live video stream for display within a viewfinder portion of a user interface, for example, as described above with reference to frame 552 of Figure 5C and Figure 6. The live video stream may show at least a portion of a current field of view of an image capture system of a mobile image capture device.

[0123] At 704, the method 700 may include storing a video segment from the live video stream in a temporary image buffer, for example, as described above with reference to Figures 1-5C. The video segment may include multiple image frames captured by an image capture system.

[0124] At 706, the method 700 may include receiving a user input directed at a viewfinder portion of the user interface requesting a rewind operation, for example, as described above with reference to frame 554 of FIGS. 5B and 5C.

[0125] At 708, the method 700 may include performing a rewind operation within a viewfinder portion of the user interface on the video segment in response to receiving a user input requesting a rewind operation, for example, as described above with reference to Figures 5A-6. During the rewind operation, at least two of the image frames of the video segment may be provided for display within the user interface in reverse chronological order, for example, as described above with reference to frame 558 of Figure 5C and Figure 6.

[0126] Additional Disclosures The technology described herein refers to servers, databases, software applications, and other computer-based systems, and actions taken to, and information sent to, such systems. The flexibility inherent in computer-based systems allows for a wide variety of possible configurations, combinations, and division of tasks and functions among components. For example, the processes described herein may be implemented using a single device or component, or multiple devices or components acting in combination. Databases and applications may be implemented on a single system or distributed across multiple systems. Distributed components may operate serially or in parallel.

[0127] While the present subject matter has been described in detail with reference to various specific embodiments thereof, each example is provided by way of illustration and not limitation of the present disclosure. Those skilled in the art, upon gaining an understanding of the above, will be able to readily create modifications, variations, and equivalents of such embodiments. Accordingly, the present disclosure does not exclude the inclusion of such changes, variations, and / or additions to the present subject matter as would be readily apparent to one skilled in the art. For example, features illustrated or described as part of one embodiment may be used with another embodiment to yield yet a further embodiment. Accordingly, the present disclosure is intended to encompass such modifications, variations, and equivalents. [Explanation of symbols]

[0128] 10, 50 computing devices 100 Computing Systems, Systems 102 User Computing Devices, Devices 112, 132, 152 processors 114 Memory, User Computing Device Memory 116, 136, 156 data 118, 138, 158 instructions 120 non-temporary memory locations 122 Temporary Image Buffer 123 Machine Learning Image Selection Model, Machine Learning Model, Model 124 Machine Learning Image Capture Model, Machine Learning Model, Model 125 Machine Learning Source Selection Model, Machine Learning Model, Model 126 User Input Components 127 Image Capture System 128 cameras, forward and rear facing cameras 130 Server Computing System 134, 154 memory 140, 142, 144 Models, Machine Learning Models 150 Training Computing System 160 Model Trainer 162 training data, training dataset 180 Network 200 Machine Learning Image Selection Model 202, 302, 596 image frames 204 Frame Selection Set 300 Machine Learning Image Capture Model, Machine Learning Capture Model 304 Captcha Judgment 400 Machine Learning Source Selection Model 402 Primary Video Source 404 Secondary Video Source 406 Source Select Output 502 Handheld Devices, Devices 504 Viewfinder part 505 Display, display screen 506 hands 508 Capture button, virtual capture button 510 Thumbnail Strip 518 Current Image 552 Frames in chronological order, first frame, frame 554, 558, 562 Frames in chronological order, frames 556 Arrow 560 Arrow Set, Arrows 568 "Save All" button 570 notifications 580, 586, 588, 590, 592, 612 frames 582 Series of Circles 594 Pop-up Windows 598 "X" 602 First Frame 604 Primary Viewing Area 606 Secondary Viewing Area 608 Second Frame, Frame 610 Dotted outline of user's hand

Claims

1. A mobile image capture device for acquiring a desired still image, comprising: an image capture system operable to capture image frames; one or more processors; one or more non-transitory computer-readable media having instructions recorded thereon that, when executed by the one or more processors, cause the one or more processors to perform a plurality of operations; and wherein the plurality of operations comprises: providing for display within a viewfinder portion of a user interface a live video stream illustrating at least a portion of a current field of view of the image capture system of the mobile image capture device; an operation of automatically storing video segments from the live video stream in a temporary image buffer as the live video stream is provided, the operation of storing being a continuous background operation initiated without specific user input for storing the video segments, the video segments comprising a plurality of image frames captured by the image capture system; receiving a user input directed at the viewfinder portion of the user interface requesting a rewind operation; in response to receiving the user input requesting the rewind operation, performing the rewind operation within the viewfinder portion of the user interface on the video segment retrieved from the temporary image buffer, the rewind operation including performing a scrubbing operation in response to continued user input movement, wherein during the rewind operation at least two of the image frames of the video segment are provided for display within the user interface in reverse chronological order; receiving a user selection of one of the image frames, the selection by the user being based on an end of the continuous user input; storing the selected image frames in a non-transitory memory location for saving to a camera roll; mobile image capture devices, including

2. the user input having a velocity; 2. The mobile image capture device of claim 1, wherein performing the rewind operation includes transitioning the user interface between displaying a first image frame of the video segment and displaying at least a second image frame of the video segment at a rate that is positively correlated to the rate of the user input.

3. 10. The mobile image capture device of claim 1, wherein the video segments from the live video stream are stored in a temporary image buffer in a first-in, first-out (FIFO) configuration associated with a moving time window.

4. a touch-sensitive display screen for displaying the user interface; The mobile image capture device of claim 1 , wherein the user input comprises a user touch action on the touch-sensitive display screen.

5. The mobile image capture device of claim 1 , wherein the plurality of operations includes an operation of displaying a rewind strip during the rewind operation.

6. The mobile image capture device of claim 1 , wherein the live video stream comprises a composite environmental stream generated from multiple live video streams.

7. The mobile image capture device of claim 1 , wherein performing the rewind operation includes providing the video segment for display in the user interface in reverse chronological order.

8. a machine learning image selection model configured to receive the plurality of image frames of the video segment and, in response to receiving the plurality of image frames, output a frame selection set representing a selection of at least one of the plurality of image frames; The plurality of operations: inputting the plurality of image frames of the video segment into the machine learning image selection model; receiving the frame selection set as an output of the machine learning image selection model; providing a selection suggestion for display at the user interface, the selection suggestion including at least one of the image frames represented by the frame selection set; 10. The mobile image capture device of claim 1, comprising:

9. The plurality of operations: receiving a user input requesting switching the live video stream from a first source to a second source; switching the live video stream from the first source to the second source in response to receiving the user input requesting switching the live video stream from the first source to the second source; The mobile image capture device of claim 1 further comprising:

10. The mobile image capture device of claim 1 , wherein the video segments are automatically recorded from the live video stream into the temporary image buffer as soon as a camera application is opened.

11. 11. The mobile image capture device of claim 10, wherein the camera application is operable in at least an image capture mode and a video capture mode, and video segments are automatically stored from the live video stream when the camera application is in the image capture mode or the video capture mode.

12. A method for performing a rewind operation to obtain a desired still image, comprising: providing, by one or more computing devices, a live video stream showing at least a portion of a current field of view of the image capture system for display within a viewfinder portion of the user interface; automatically storing video segments from the live video stream in a temporary image buffer by the one or more computing devices as the live video stream is provided, the storing being a continuous background operation initiated without specific user input for storing the video segments, the video segments comprising a plurality of image frames captured by the image capture system; receiving, by the one or more computing devices, a user input directed at the viewfinder portion of the user interface requesting a rewind operation; performing, by the one or more computing devices, a rewind operation within the viewfinder portion of the user interface on the video segment retrieved from the temporary image buffer in response to receiving the user input requesting the rewind operation, wherein performing the rewind operation includes performing a scrubbing operation in response to continued user input movement, and during the rewind operation, at least two of the image frames of the video segment are provided for display within the user interface in reverse chronological order; receiving a user selection of one of the image frames, the selection by the user being based on an end of the continuous user input; storing, by the one or more computing devices, the selected image frames in a non-transitory memory location for saving to a camera roll; A method comprising:

13. 13. The method of claim 12, wherein performing the rewind operation includes transitioning the user interface between displaying a first image frame of the video segment and displaying at least a second image frame of the video segment at a rate that is positively correlated to a rate of the user input.

14. The method of claim 12 , wherein the video segments are automatically recorded from the live video stream into the temporary image buffer as soon as a camera application is opened.

15. 15. The method of claim 14, wherein the camera application is operable in at least an image capture mode and a video capture mode, and wherein video segments are automatically stored from the live video stream when the camera application is in the image capture mode or the video capture mode.

16. The method of claim 12 , wherein the video segments are automatically stored in the temporary image buffer from the live video stream regardless of the absence of user input on a shutter button.

17. one or more non-transitory computer-readable media having instructions recorded thereon, The instructions, when executed by one or more processors, cause the one or more processors to perform a plurality of operations to obtain a desired still image, the plurality of operations including: providing a live video stream showing at least a portion of a current field of view of the image capture system for display within a viewfinder portion of the user interface; an operation of automatically storing video segments from the live video stream in a temporary image buffer as the live video stream is provided, the operation of storing being a continuous background operation initiated without specific user input for storing the video segments, the video segments comprising a plurality of image frames captured by the image capture system; receiving a user input directed at the viewfinder portion of the user interface requesting a rewind operation; in response to receiving the user input requesting the rewind operation, performing the rewind operation within the viewfinder portion of the user interface on the video segment retrieved from the temporary image buffer, the rewind operation including performing a scrubbing operation in response to continued user input movement, wherein during the rewind operation at least two of the image frames of the video segment are provided for display within the user interface in reverse chronological order; receiving a user selection of one of the image frames, the selection by the user being based on an end of the continuous user input; storing the selected image frames in a non-transitory memory location for saving to a camera roll; [0023] 1. One or more non-transitory computer-readable media,

Images