User Interface for Image Capture

Abstract

The present disclosure provides a system, apparatus, method, and computer-readable medium that supports linking paused video recordings captured over a period of time. In a first aspect, a method of image processing may include determining a match between first image data and second image data, presenting an indication of the match to a user, receiving user input responsive to the indication, and determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data. Other aspects and features are also recited and described in the claims.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. patent application Ser. No. 17 / 648,523, entitled "USER INTERFACE FOR IMAGE CAPTURE," filed Jan. 20, 2022, which is expressly incorporated by reference in its entirety herein.

[0002] Aspects of the present disclosure relate generally to image capture. Several features may enable and provide an improved interface for capturing images.

[0003] introduction An image capture device is a device capable of capturing one or more digital images, whether a still image for a photograph or a sequence of images for a video. Capture devices can be incorporated into a wide variety of devices. By way of example, image capture devices may include stand-alone digital cameras or digital video camcorders; camera-equipped wireless communication device handsets, such as mobile phones, cellular or satellite radio phones, personal digital assistants (PDAs), panels or tablets, gaming devices, etc.; computing devices, such as webcams, video surveillance cameras, etc.; or other devices with digital imaging or video capabilities.

[0004] Sharing digital videos captured by image capture devices, such as videos of a user's life experiences, allows a user to stay in touch with many friends and family over great distances. Sharing such videos also allows others to experience what the user has experienced. Such sharing may be for personal enjoyment and / or may be used for marketing purposes. Summary of the Invention

[0005] The following summarizes some aspects of the present disclosure in order to provide a basic understanding of the discussed technology. This summary is not an extensive overview of all of the contemplated features of the present disclosure, and is not intended to identify key or critical elements of all aspects of the present disclosure or to delineate the scope of any or all aspects of the present disclosure. Its sole purpose is to present some concepts of one or more aspects of the present disclosure in summary form as a prelude to the more detailed description that is presented later.

[0006] In some aspects, video sharing and other video uses can be supported using a pause feature for videos that allows video recording to be resumed at a later time based on matching features of the stored video and the new image frame. Artificial intelligence can be used to identify matches with a sufficient confidence level (e.g., above a threshold confidence level). The matched video can be used to extend the paused video, and a compressed and / or encoded video file can be exported once the paused video is complete.

[0007] In one aspect of the disclosure, a method for image processing includes determining a match between first image data and second image data, presenting an indication of the match to a user, receiving a user input responsive to the indication, and determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data. The first image data and the second image data can be image data, such as a photograph or a video sequence of a series of image frames. The image data can be obtained from a camera of an image capture device and / or other sources, such as cloud storage, text messages, local storage, Internet sources, remote video feeds (e.g., RTSP streams), etc. Thus, in one embodiment, the matching identifies a match between a current image data record and previously recorded image data recorded by the same local storage device and stored in a buffer and / or storage device. In another embodiment, the matching identifies a match between the current image data and a previously recorded image received from another wireless device, such as via an attachment to a text message (e.g., an MMS and / or RCS chat message).

[0008] In an additional aspect of the disclosure, an apparatus, such as an image capture device, includes at least one processor and a memory coupled to the at least one processor, the at least one processor configured to perform operations including determining a match between first image data and second image data, presenting an indication of the match to a user, receiving user input responsive to the indication, and determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data.

[0009] In an additional aspect of the present disclosure, an apparatus includes means for determining a match between the first image data and the second image data, means for presenting an indication of the match to a user, means for receiving user input responsive to the indication, and means for determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data.

[0010] In an additional aspect of the present disclosure, a non-transitory computer readable medium has instructions stored therein that, when executed by a processor, cause the processor to perform operations including determining a match between the first image data and the second image data, presenting an indication of the match to a user, receiving user input responsive to the indication, and / or determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data.

[0011] An image capture device is a device capable of capturing one or more digital images, whether still photographs or a sequence of images for a video, and may be incorporated into a wide variety of devices. By way of example, an image capture device may include a stand-alone digital camera or digital video camcorder; a wireless communication device handset with a camera, such as a mobile phone, cellular or satellite radio phone, personal digital assistants (PDAs), panels or tablets, gaming devices, etc.; a computing device, such as a webcam, video surveillance camera, etc.; or other device with digital imaging or video capabilities.

[0012] Generally, this disclosure describes image processing techniques for digital cameras having image sensors and image signal processors (ISPs). The ISP can be configured to control the capture of image frames from one or more image sensors and to process one or more image frames from the one or more image sensors to generate a view of a scene in a corrected image frame. The corrected image frame can be part of a sequence of image frames forming a video. The video can include other image frames received from the image sensor or other image sensors and / or other corrected image frames based on input from the image sensor or another image sensor. In some embodiments, the processing of one or more image frames can be performed within the image sensor. The image processing techniques described in the embodiments disclosed herein can be performed by circuitry within the image sensor, circuitry within an image signal processor (ISP), circuitry within an application processor (AP), or a combination of these components, or two or all of these components.

[0013] In one embodiment, the image signal processor may receive instructions to capture a sequence of image frames in response to loading of software, such as a camera application, to generate a preview display from an image capture device. The image signal processor may be configured to generate a single flow of output frames based on image frames received from one or more image sensors. This single flow of output frames may include raw image data from the image sensor or corrected image frames processed by one or more algorithms in the image signal processor. For example, image frames acquired from an image sensor, where some processing may have been performed on the data before being output to the image signal processor, may be processed in the image signal processor by processing the image frames through an image post-processing engine (IPE) and / or other image processing circuitry to perform one or more of tone mapping, portrait lighting, contrast enhancement, gamma correction, etc.

[0014] After an output frame representative of a scene has been determined by the image signal processor using image corrections described in various embodiments herein, the output frame may be displayed on a device display as a single still image and / or as part of a video, may be saved to a storage device as a picture or video, may be transmitted over a network, and / or may be printed on an output medium. For example, the image signal processor may be configured to obtain input frames of image data (e.g., pixel values) from various image sensors and then generate corresponding output frames of image data (e.g., preview display frames, still image capture, frames for video, etc.). In other examples, the image signal processor may output frames of image data to various output devices and / or camera modules for further processing, such as for 3A parameter synchronization (e.g., auto focus (AF), auto white balance (AWB), and auto exposure control (AEC)), generating a video file via the output frames, composing frames for display, composing frames for storage, transmitting frames over a network connection, etc. That is, the image signal processor can acquire incoming frames from one or more image sensors, each coupled to one or more camera lenses, and then generate a flow of output frames to output to various destinations. In such an embodiment, the image signal processor can be configured to generate a flow of output frames that may have improved appearance in low light conditions.

[0015] In some aspects, the corrected image frame can be generated by combining aspects of the image correction of the present disclosure with other computational photography techniques, such as high dynamic range (HDR) photography or multi-frame noise reduction (MFNR). For HDR photography, the first and second image frames are captured using different exposure times, different apertures, different lenses, and / or other different characteristics, which may result in an improved dynamic range of the fused image when the two image frames are combined. In some aspects, the method can be performed with respect to MFNR photography, where the first and second image frames are captured using the same or different exposure times and are fused to generate a corrected first image frame having reduced noise compared to the captured first image frame.

[0016] In some aspects, the device may include an image signal processor or a processor (e.g., an application processor) that includes specific functionality related to camera control and / or processing, such as enabling or disabling image correction or otherwise controlling aspects of image correction by determining when deblurring should be applied, applying deblurring operations to blurred objects, and / or controlling the zoom level of the resulting image, etc. The methods and techniques described herein may be performed entirely by the image signal processor or processor, or various operations may be divided between the image signal processor and the processor, or in some aspects across additional processors.

[0017] The apparatus may include one, two, or more image sensors, such as including a first image sensor. If there are multiple image sensors, the first image sensor may have a larger field of view (FOV) than the second image sensor, or the first image sensor may have a different sensitivity or different dynamic range than the second image sensor. In one embodiment, the first image sensor may be a wide-angle image sensor and the second image sensor may be a telephoto image sensor. In another embodiment, the first sensor is configured to acquire images through a first lens having a first optical axis, and the second sensor is configured to acquire images through a second lens having a second optical axis different from the first optical axis. Additionally or alternatively, the first lens may have a first magnification and the second lens may have a second magnification different from the first magnification. This configuration may occur using a lens cluster on the mobile device, such as when multiple image sensors and associated lenses are located at offset locations on the front or back of the mobile device. Additional image sensors with larger, smaller, or the same field of view may also be included. The image correction techniques described herein may be applied to image frames captured from any of the image sensors in a multi-sensor device.

[0018] In an additional aspect of the present disclosure, a device configured for image processing and / or image capture is disclosed. The device includes a means for capturing an image frame. The device further includes one or more means for capturing data representative of a scene, such as image sensors (including charge-coupled devices (CCDs), Bayer filter sensors, infrared (IR) detectors, ultraviolet (UV) detectors, complementary metal oxide semiconductor (CMOS) sensors), time-of-flight detectors, etc. The device may further include one or more means for integrating and / or focusing light rays into the one or more image sensors (including simple lenses, compound lenses, spherical lenses, and aspherical lenses). These components can be controlled to capture a first image frame and / or a second image frame, which are input to the image processing techniques described herein.

[0019] Other aspects, features, and implementations will become apparent to those skilled in the art upon reviewing the following description of certain exemplary aspects in conjunction with the accompanying figures. Although features may be discussed in conjunction with certain aspects and figures below, various aspects may include one or more of the advantageous features discussed herein. In other words, although one or more aspects may be discussed as having certain advantageous features, one or more of such features may also be used in accordance with various aspects. Similarly, although exemplary aspects may be discussed below as device, system, or method aspects, the exemplary aspects may be implemented in a variety of devices, systems, and methods.

[0020] The methods described herein can be embedded in a computer-readable medium as computer program code including instructions that cause a processor to perform the operations of the methods. In some embodiments, the processor can be part of a mobile device including a first network adapter configured to transmit data, such as images or videos, as recorded data or streaming data over a first network connection of a plurality of network connections, a processor coupled to the first network adapter, and a memory. The processor can execute the instructions of the computer-readable medium to cause transmission of the corrected image frames described herein over a wireless communication network, such as a 5G NR communication network.

[0021] The foregoing has outlined rather broadly the features and technical advantages of the embodiments according to the present disclosure in order that the following Detailed Description may be better understood. Additional features and advantages are described hereinafter. The concepts and specific examples disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Such equivalent constructions do not depart from the scope of the appended claims. The concepts disclosed herein, both their organization and method of operation, characteristic of the concepts disclosed herein, together with associated advantages, will be better understood in consideration of the following description in conjunction with the accompanying figures. Each of the figures is provided for the purpose of illustration and description, and not as a definition of the limits of the claims.

[0022] Although aspects and implementations are described in this application by way of example for some embodiments, those skilled in the art will appreciate that additional implementations and use cases may occur in many different configurations and scenarios. The innovations described herein may be implemented across many different platform types, devices, systems, shapes, sizes, and packaging configurations. For example, aspects and / or applications may occur via integrated chip implementations and other non-modular component-based devices (e.g., end-user devices, vehicles, communication devices, computing devices, industrial equipment, retail / purchasing devices, medical devices, artificial intelligence (AI)-enabled devices, etc.). Some embodiments may or may not be specifically targeted to a use case or application, but a wide variety of combination applicability of the described innovations may occur. Implementations may range from chip-level or modular components to non-modular, non-chip-level implementations, and even aggregated, distributed, or original equipment manufacturer (OEM) devices or systems incorporating one or more aspects of the described innovations. In some practical settings, devices incorporating the described aspects and features may also necessarily include additional components and features for implementing and practicing the claimed and described aspects. For example, transmitting and receiving wireless signals necessarily includes a number of components for analog and digital purposes (e.g., hardware components including antennas, radio frequency (RF) chains, power amplifiers, modulators, buffers, processors, interleavers, summers / analog summers, etc.). It is contemplated that the innovations described herein may be practiced in a wide variety of devices, chip-level components, systems, distributed configurations, end-user devices, etc., of various sizes, shapes, and configurations.

[0023] A further understanding of the nature and advantages of the present disclosure can be realized by referring to the following drawings. In the accompanying figures, similar components or features may have the same reference label. Furthermore, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes the similar components. When only a first reference label is used in this specification, the description is applicable to any one of the similar components having the same first reference label, regardless of the second reference label. [Brief description of the drawings]

[0024] [Figure 1] 1 shows a block diagram of an example device for performing image capture from one or more image sensors. [Diagram 2] 1 shows a block diagram illustrating video matching using snapshots, according to some embodiments of the present disclosure. [Diagram 3] 1A-1C are diagrams illustrating detecting a pause command and taking a snapshot for video matching according to some embodiments of the present disclosure. [Figure 4] FIG. 1 illustrates detecting video matching for snapshots according to some embodiments of the present disclosure. [Diagram 5] 1 is a flowchart illustrating a method for assembling previously paused matched videos according to some embodiments of the present disclosure. [Figure 6] 1 is a flowchart illustrating a method for detecting a video match according to some embodiments of the present disclosure. [Figure 7] 1 shows a block diagram of an image capture device illustrating the saving of paused video to a buffer in accordance with some embodiments of the present disclosure.

[0025] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] The detailed description of the present invention, set forth below in conjunction with the accompanying drawings, is intended as an illustration of various configurations and is not intended to limit the scope of the present disclosure. Rather, the detailed description of the present invention includes specific details intended to provide a thorough understanding of the subject matter of the present invention. Those skilled in the art will appreciate that these specific details are not required in every instance and that in some instances, well-known structures and components are shown in block diagram form for clarity of presentation.

[0027] The present disclosure provides a system, apparatus, method, and computer-readable medium that provides a user interface to support image capture and linking between captured images and / or videos. In one embodiment, different image capture sessions can be matched based on an indicator of or within the content. A user can use such a system, apparatus, method, and computer-readable medium to effectively pause and continue a video recording across multiple image capture sessions without the user having to manually specify linking between the sessions. For example, the present disclosure provides a system, apparatus, method, and computer-readable medium that supports image processing to allow a user to record a first video, pause the video recording, and then continue. Linked snapshots can be used to match image data for a video recording or preview image to a previously paused video. If a match is identified between the image sensor's video or current scene and the previously paused video, the user can be prompted of the match and asked for the current video to continue the previously paused video.

[0028] Particular implementations of the subject matter described in this disclosure can be implemented to achieve one or more of the following potential advantages or benefits: In some aspects, the present disclosure provides techniques for improving the relevance of photos and / or videos. For example, matching videos to previously recorded videos can enable generating an automated combination of relevant videos, such as a journey through the first year of life or the first year of marriage. Using a newborn diary as an example, photos of the baby's associations can be used as snapshots to link the videos together. Aspects of the present invention can benefit users such as travelers, video bloggers, YouTubers, and / or videography enthusiasts.

[0029] An exemplary device for capturing image frames using one or more image sensors, such as a smartphone, may include a configuration of two, three, four, or more cameras on the back (e.g., opposite the user display) or front (e.g., on the same side as the user display) of the device. A device with multiple image sensors includes one or more image signal processors (ISPs), computer vision processors (CVPs) (e.g., AI engines), or other suitable circuitry for processing images captured by the image sensors. The one or more image signal processors can provide the processed image frames to a memory and / or processor (such as an application processor, an image front end (IFE), an image processing engine (IPE), or other suitable processing circuitry) for further processing, such as for encoding, storage, transmission, or other manipulation.

[0030] As used herein, an image sensor may refer to the image sensor itself as well as certain other components coupled to the image sensor that are used to generate an image frame for processing by an image signal processor or other logic circuitry, or for storage in memory, whether a short-term buffer or longer-term non-volatile memory. For example, an image sensor may include other components of a camera, including shutters, buffers, or other readout circuitry for accessing individual pixels of the image sensor. An image sensor may also refer to an analog front end (AFE) or other circuitry (e.g., an analog-to-digital converter) for converting analog signals into a digital representation of the image frame that can be further processed in digital circuitry coupled to the image sensor.

[0031] In the following description, numerous specific details are set forth, such as examples of specific components, circuits, and processes, to provide a thorough understanding of the present disclosure. The term "coupled" as used herein means directly connected or connected through one or more intervening components or circuits. In addition, in the following description, for the purpose of explanation, specific terminology is set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that these specific details may not be required to practice the teachings disclosed herein. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring the teachings of the present disclosure.

[0032] Some portions of the following Detailed Description are presented in terms of procedures, logic blocks, processes, and other symbolic representations of operations on data bits within a computer memory. In this disclosure, a procedure, logic block, process, etc., is conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system.

[0033] In the figures, a single block may be described as performing one or more functions. The functions performed by the block may be performed in a single component or across multiple components, and / or may be performed using hardware, software, or a combination of hardware and software. To clearly illustrate this interchangeability of hardware and software, various example components, blocks, modules, circuits, and steps are generally described below in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each specific application, and such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. Also, the example device may include components other than those shown, including well-known components such as a processor, memory, etc.

[0034] Aspects of the present disclosure are applicable to any electronic device that includes or is coupled to two or more image sensors capable of capturing image frames (or "frames"). Moreover, aspects of the present disclosure can be implemented in devices having or coupled to image sensors of the same or different capabilities and characteristics (resolution, shutter speed, sensor type, etc.). Furthermore, aspects of the present disclosure can be implemented in devices for processing image frames, such as processing devices capable of retrieving stored images for processing, including processing devices present in cloud computing systems, regardless of whether the device includes or is coupled to an image sensor.

[0035] Unless otherwise indicated, and as will be apparent from the discussion that follows, discussions utilizing terms such as "accessing," "receiving," "sending," "using," "selecting," "determining," "normalizing," "multiplying," "averaging," "monitoring," "comparing," "applying," "updating," "measuring," "deriving," "solving," "generating," and the like throughout this application will be understood to refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities in the computer system's registers and memory, and converts such data to other data that is similarly represented as physical quantities in the computer system's registers, memory, or other such information storage, transmission, or display devices.

[0036] The terms "device" and "apparatus" are not limited to one physical object (such as one smartphone, one camera controller, one processing system, etc.) or a particular number of physical objects. As used herein, a device can be any electronic device having one or more parts capable of implementing at least some parts of the present disclosure. Although the following description and examples use the term "device" to describe various aspects of the present disclosure, the term "device" is not limited to a particular configuration, type, or number of objects. As used herein, an apparatus may include a device for performing the described operations, or a portion of such a device.

[0037] 1 illustrates a block diagram of an exemplary device 100 for performing image capture from one or more image sensors. The device 100 may include or be otherwise coupled to an image signal processor 112 for processing image frames from one or more image sensors, such as a first image sensor 101, a second image sensor 102, and a depth sensor 140. In some implementations, the device 100 also includes or is coupled to a processor 104 and a memory 106 that stores instructions 108. The device 100 may also include or be coupled to a display 114 and input / output (I / O) components 116. The I / O components 116, such as a touch screen interface and / or physical buttons, may be used to interact with a user. The I / O components 116 may also include network interfaces for communicating with other devices, including a wide area network (WAN) adapter 152, a local area network (LAN) adapter 153, and / or a personal area network (PAN) adapter 154. An exemplary WAN adapter is a 4G LTE or 5G NR wireless network adapter. An exemplary LAN adapter 153 is an IEEE 802.11 WiFi wireless network adapter. An exemplary PAN adapter 154 is a Bluetooth wireless network adapter. Each of the adapters 152, 153, and / or 154 may be coupled to an antenna, including multiple antennas configured for primary and diversity reception and / or configured to receive a particular frequency band. The device 100 may further include or be coupled to a power source 118 for the device 100, such as a battery or components for coupling the device 100 to an energy source. The device 100 may also include or be coupled to additional features or components not shown in FIG. 1 .In one embodiment, a wireless interface, which may include a transceiver and a baseband processor, may be coupled to or included within WAN adapter 152 for the wireless communication device. In a further embodiment, an analog front end (AFE) for converting analog image frame data to digital image frame data may be coupled between image sensor 101 and image sensor 102 and image signal processor 112.

[0038] The device may include or be coupled to a sensor hub 150 for interfacing with sensors for receiving data regarding the movement of the device 100, data regarding the environment surrounding the device 100, and / or other non-camera sensor data. One exemplary non-camera sensor is a gyroscope, which is a device configured to measure rotation, orientation, and / or angular velocity to generate motion data. Another exemplary non-camera sensor is an accelerometer, which is a device configured to measure acceleration, which can also be used to determine the speed and distance of movement by appropriately integrating the measured acceleration, and one or more of the acceleration, speed, and / or distance can be included in the generated motion data. In some aspects, a gyroscope in an electronic image stabilization system (EIS) can be coupled to the sensor hub or directly to the image signal processor 112. In another example, the non-camera sensor can be a global positioning system (GPS) receiver. The image signal processor 112 may also receive contextual information from the processor 104 regarding the movement of the device 100, such as an indication regarding an active application or service running on the processor 104. For example, an indication of an active run-tracking application may indicate that the movement of the device 100 is a running activity.

[0039] The image signal processor 112 can receive image data, such as that used to form an image frame. In one embodiment, a local bus connection couples the image signal processor 112 to the first camera image sensor 101 and the second camera image sensor 102. In another embodiment, a wired interface couples the image signal processor 112 to an external image sensor. In a further embodiment, a wireless interface couples the image signal processor 112 to the image sensors 101, 102.

[0040] The first camera may include a first image sensor 101 and a corresponding first lens 131. The second camera may include a second image sensor 102 and a corresponding second lens 132. Each of the lenses 131 and 132 may be controlled by an associated auto-focus (AF) algorithm 133 running in the ISP 112, which adjusts the lenses 131 and 132 to focus on a particular focal plane at a particular scene depth from the image sensors 101 and 102. The AF algorithm 133 may be assisted by a depth sensor 140.

[0041] The first image sensor 101 and the second image sensor 102 are configured to capture one or more image frames. The lenses 131 and 132 focus light onto the image sensor 101 and the image sensor 102, respectively, through one or more apertures for receiving light, one or more shutters for blocking light when outside an exposure window, one or more color filter arrays (CFAs) for filtering light other than a certain frequency range, one or more analog front ends for converting analog measurements to digital information, and / or other suitable components for imaging. The first lens 131 and the second lens 132 may have different fields of view for capturing different representations of a scene. For example, the first lens 131 may be an ultra-wide angle (UW) lens and the second lens 132 may be a wide angle (W) lens. The multiple image sensors may include a combination of ultra-wide angle (high field of view (FOV)), wide angle, telephoto, and super telephoto (low FOV) sensors. That is, each image sensor can be configured via hardware configuration and / or software settings to provide different but overlapping fields of view. In one configuration, the image sensors are configured using different lenses with different magnifications resulting in different fields of view. The sensors can be configured such that the UW sensor has a larger FOV than the W sensor, which has a larger FOV than the T sensor, which has a larger FOV than the UT sensor. For example, a sensor configured for a wide FOV can capture a field of view ranging from 64 to 84 degrees, a sensor configured for an ultra-wide FOV can capture a field of view ranging from 100 to 140 degrees, a sensor configured for a telephoto FOV can capture a field of view ranging from 10 to 30 degrees, and a sensor configured for a super telephoto FOV can capture a field of view ranging from 1 to 8 degrees.

[0042] Image signal processor 112 processes image frames captured by image sensor 101 and image sensor 102. Although FIG. 1 illustrates device 100 as including two image sensors 101 and 102 coupled to image signal processor 112, any number of image sensors (e.g., one, two, three, four, five, six, etc.) may be coupled to image signal processor 112. In some aspects, a depth sensor, such as depth sensor 140, may be coupled to image signal processor 112, and output from the depth sensor may be processed in a similar manner as the output of image sensor 101 and image sensor 102. Furthermore, any number of additional image sensors or image signal processors may be present for device 100.

[0043] In some embodiments, the image signal processor 112 may execute instructions from a memory, such as instructions 108 from memory 106, instructions stored in a separate memory coupled to or included within the image signal processor 112, or instructions provided by the processor 104. Additionally or alternatively, the image signal processor 112 may include specific hardware (such as one or more integrated circuits (ICs)) configured to perform one or more operations described in this disclosure. For example, the image signal processor 112 may include one or more image front ends (IFEs) 135, one or more image post-processing engines 136 (IPEs), and / or one or more automatic exposure correction (AEC) 134 engines. The AF 133, AEC 134, AFE 135, and APE 136 may each include application specific circuitry and may be embodied as software code executed by the ISP 112 and / or as a combination of hardware and software code.

[0044] In some implementations, memory 106 may include a non-transient or non-transitory computer-readable medium having stored thereon computer-executable instructions 108 for performing all or a portion of one or more operations described in this disclosure. In some implementations, instructions 108 include a camera application (or other suitable application) to be executed by device 100 to generate images or videos. Instructions 108 may also include other applications or programs executed by device 100, such as an operating system and specific applications other than for image or video generation. Execution of the camera application, such as by processor 104, may cause device 100 to generate images using image sensors 101 and 102 and image signal processor 112. Memory 106 may also be accessed by image signal processor 112 to store processed frames or by processor 104 to obtain processed frames. In some embodiments, device 100 does not include memory 106. For example, device 100 can be a circuit that includes image signal processor 112, and the memory can be external to device 100. Device 100 can be coupled to external memory and configured to access the memory to write output frames for display or long-term storage. In some embodiments, device 100 is a system-on-chip (SoC) that incorporates image signal processor 112, processor 104, sensor hub 150, memory 106, and input / output components 116 in a single package.

[0045] In some embodiments, at least one of the image signal processor 112 or the processor 104 executes instructions to perform various operations described herein, including noise reduction operations. For example, execution of the instructions may instruct the image signal processor 112 to begin or end capturing an image frame or a sequence of image frames, including noise reduction as described in the embodiments herein. In some embodiments, the processor 104 may include one or more general-purpose processor cores 104A capable of executing scripts or instructions of one or more software programs, such as instructions 108 stored in the memory 106. For example, the processor 104 may include one or more application processors configured to execute a camera application (or other suitable application for generating images or videos) stored in the memory 106.

[0046] When executing the camera application, the processor 104 can be configured to instruct the image signal processor 112 to perform one or more operations related to the image sensor 101 or image sensor 102. For example, the camera application can receive a command to initiate a video preview display in which a video including a sequence of image frames from one or more image sensors 101 or image sensor 102 is captured and processed. Image correction, such as by cascaded IPE, can be applied to one or more image frames in the sequence. Execution of instructions 108 outside of the camera application by the processor 104 can also cause the device 100 to perform any number of functions or operations. In some embodiments, the processor 104 can include ICs or other hardware (e.g., an artificial intelligence (AI) engine 124 having one or more circuits configured to execute machine learning (ML) algorithms) in addition to the ability to execute software to cause the device 100 to perform any number of functions or operations, such as those described herein. In some other embodiments, the device 100 does not include the processor 104, such as when all of the described functionality is configured in the image signal processor 112.

[0047] In some embodiments, display 114 may include one or more suitable displays or screens that allow for user interaction and / or allow for presenting items to a user, such as previews of image frames being captured by image sensor 101 and image sensor 102. In some embodiments, display 114 is a touch-sensitive display. I / O components 116 may be or include any suitable mechanisms, interfaces, or devices for receiving input (e.g., commands) from a user and for providing output to a user via display 114. For example, I / O components 116 may include (without limitation) a graphical user interface (GUI), a keyboard, a mouse, a microphone, a speaker, a compressible bezel, one or more buttons (e.g., a power button), sliders, switches, etc.

[0048] Although shown coupled together via the processor 104, the components (such as the processor 104, memory 106, image signal processor 112, display 114, and I / O components 116) can be coupled together in various other configurations, such as via one or more local buses, not shown for simplicity. Although the image signal processor 112 is shown as separate from the processor 104, the image signal processor 112 can also be a core of the processor 104, an application processor unit (APU), included in a system on chip (SoC) or otherwise included with the processor 104. Although the device 100 is referred to in the examples herein for carrying out aspects of the present disclosure, some device components may not be shown in FIG. 1 to avoid obscuring aspects of the present disclosure. Moreover, other components, multiple components, or combinations of components can be included in a device suitable for carrying out aspects of the present disclosure. Thus, the present disclosure is not limited to any particular device or configuration of components, including the device 100.

[0049] The image capture device 100 can be used to record video by capturing changes in a scene over a period of time, such as 10 seconds, 30 seconds, 3 minutes, or 30 minutes. The video is recorded as one or a series of image frames that occur one after the other in continuous time. The recording continues until the user stops the recording, after which the video is saved as a file in the image capture device 100, such as in non-volatile memory, by encoding the image frames into an appropriate format. Past image capture sessions (including one or more image frames and / or videos) can be made available in a gallery application that displays previously captured videos and photos. The image frames are not all contiguous in time, since a user may want to add content to a previously captured video after a long period of time has elapsed. For example, a user may want to record a video capturing a short portion, such as 30 seconds, every day for a longer period of time, such as a year. Each day over that period, the video recording can be stopped and started as a new file. However, this can be an inconvenience to the user in that these separate videos must then be combined by the user using software into an output video.

[0050] The user can stop the recording, and when a subsequent image capture session captures related content, the user can be provided with the option to effectively continue the video recording at that future point in time. This option can be supported by an image matching function where similarities are identified between the current scene and a previously paused video recording. When similarities are detected, the user can be prompted with the option to append to the previous video recording and continue the video recording. Appending can include linking files together, such as in a playlist, and / or merging the content into a single file.

[0051] An example of video pausing and linking is shown in Figure 2. Figure 2 shows a block diagram illustrating video matching using snapshots, according to some embodiments of the present disclosure. Several videos 210, 220, 230, and 240 can be stored in a memory, such as a buffer. Each of the videos 210, 220, 230, and 240 can have an associated snapshot 212, 222, 232, and 242, respectively. In some embodiments, the snapshots 212, 222, 232, and 242 can be the last image frames in a sequence of image frames that make up the videos 210, 220, 230, and 240. The snapshots 212, 222, 232, and 242 can be analyzed for similarities and matches identified to combine some of the videos 210, 220, 230, and 240. Other aspects of the videos 210, 220, 230, and 240 can be used in determining a match as well. For example, snapshot 212 corresponding to video 210 may be compared against a first image frame of each of videos 220, 230, and 240 to determine a matching video. As another example, snapshot 212 corresponding to video 210 may be compared against multiple image frames in each of videos 220, 230, and 240 to determine a matching video. Video matching may be performed based on image recognition, such as facial recognition, to match scenes. Video matching may also or alternatively be performed based on detecting a particular scene, such as by photographing a user's hands at the end of video 210, such that a particular hand symbol (e.g., a hand with fingers spread) appears in snapshot 212.

[0052] 2, snapshot 212, by matching with snapshot 222, indicates that video 220 is a continuation of paused (e.g., previously recorded) video 210. Similarly, snapshot 222, by matching with snapshot 242, indicates that video 220 is a continuation of paused video 240. The user may be prompted that a match has been identified for paused video 210 when recording video 230, such as at the beginning of video 230, and may be prompted with an option to continue recording for paused video 210. In some embodiments, paused video 210 may be unpaused automatically (e.g., without user input) once a match between video 230 and snapshot 212 is determined.

[0053] The snapshot 212 may include a symbol, gesture, object, or scene that is predefined to indicate a video pause command. One example of a gesture is a user's open hand in the foreground of an image frame. If detected at the end of a video, such a gesture may indicate that the user is pausing the video rather than ending it. If the gesture is subsequently detected, the device may determine a match between the paused video and the current image frame and record an additional video as a continuation of the paused video. The matched videos may be combined to determine an output video that includes some or all image frames from the matched videos. The output video 250 may include image frames from the video 230 and the video 210 based on matching the video 230 with the snapshot 212 corresponding to the video 210. The output video 252 may include image frames from the video 240 and the video 220 based on matching the video 240 with the snapshot 222 corresponding to the video 220. Snapshot 212 may be different from snapshot 222 such that output video 250 is determined not to include video 220. For example, snapshot 212 may be of a user's hand with one finger outstretched and snapshot 222 may be of a user's hand with two fingers outstretched. As another example, snapshot 212 may have a first recognized face and snapshot 222 may have a different second recognized face. In some embodiments, combining the videos may involve processing such as scaling the video resolution of paused videos 210, 220, 230, and / or 240 to have a similar or the same resolution as other paused videos being combined in forming output video 250 and / or output video 252.

[0054] After combining the image frames from the matched videos, the output video 250 and / or the output video 252 may be further processed. In some embodiments, the output video 250 and / or the output video 252 may be encoded, such as by compressing the image frames to reduce image size. In some embodiments, the output video 250 and / or the output video 252 may have been audio processed to normalize audio levels between the combined videos. In some embodiments, the output video 250 and / or the output video 252 may have been video processed to normalize luminance or gamma levels between the combined videos.

[0055] The paused videos 210, 220, 230, and / or 240 can be recorded and paused based on user input. The user input to pause the video can be the user presenting a particular object or scene to the camera. An example is shown in FIG. 3, which illustrates detecting a pause command and taking a snapshot for video matching, according to some embodiments of the present disclosure. The user can place a "stop" hand symbol in the camera's view, which causes the image capture device 100 to capture 300 a snapshot 212 of the scene at the pause time. The device 100 stores the image frames captured during the video recording up to the time it receives this user input as the paused video 210 with the linked snapshot 212.

[0056] After detecting a pause command, device 100 stores the paused video until a match with a new video recording is found. Once a match is determined, the user may be prompted to continue the paused video, as shown in the example of FIG. 4. FIG. 4 is a diagram of detecting a video match to a snapshot, according to some embodiments of the disclosure. Device 100 may detect a matching scene to snapshot 212, such as by determining that the same object is present, that the same gesture is made, and / or that the same face is detected. The matching scene may be determined from a preview stream of image frames from the camera, or may be determined during ongoing video recording. Upon detecting a match, device 100 prompts the user (402) as to whether to continue the previously paused video. In some embodiments, prompt 402 may include a preview of the previously paused video. In some embodiments, prompt 402 may include an option to select one of several previously paused videos when multiple paused videos match snapshot 212.

[0057] Exemplary process steps for pausing and subsequently continuing a video are shown in FIG. 5. FIG. 5 is a flow chart illustrating a method for assembling previously paused matched videos according to some embodiments of the present disclosure. The method 500 includes, at block 502, recording a first video. The video may be encoded and stored in a gallery of the mobile device, or may be stored uncompressed in a buffer. In some embodiments, the first recording of the video may be received from another wireless device, from a storage device, from cloud storage, etc., as an attachment to an email message, and / or as an attachment to a text or chat message. At block 504, the video recording of the first video is paused. The pause may be based on user input, such as by a user controlling input to an image capture device to capture a particular object, scene, or gesture. The pause may be based on user input directly to the device, such as by pressing a pause button in a camera application that is recording the first video. The pausing may be based on a predetermined rule, such as by starting the recording of the first video at block 502 using a rule to capture 30 seconds of video. Upon deciding to pause the recording at block 504, a snapshot image frame may be stored and linked with the first video recorded at block 502. This snapshot may include the scene, object, or gesture that caused the pause at block 504, so that the same scene, object, or gesture can later be used to match a new recording to the first video paused at block 504.

[0058] A new video may be recorded at block 508, and the new video may be matched at block 510 with the previously paused video via their linked snapshots. In some embodiments, the new video may be received from another wireless device, from a storage device, from cloud storage, etc., as an attachment to an email message, and / or as an attachment to a text or chat message. The matching may use artificial intelligence (AI) algorithms to determine similarity between image frames of the new video and the snapshots. If the similarity is higher than a threshold percentage or score, the user may be prompted with the identity of the one or more matched videos. For example, the user may be prompted if a match between the first video recorded at block 502 and the new video recorded at block 508 is determined via the snapshots stored at block 506. The matching process at block 510 may include comparing one or more image frames of the new video recorded at block 508 with one or more previously captured snapshot image frames and / or one or more previously paused video recordings. For example, the buffer may contain multiple paused unencoded videos that can be matched to a new video in block 510. In some embodiments, the matching in block 510 may be triggered based on a user input, such as a user tapping on a display screen of an image capture device.

[0059] Once a match is determined in block 510, a user prompt may be provided to the user to complete the video by combining the previously paused video with the new view. In block 512, the user is prompted whether to output the combined video. The prompt may include an indication of the match along with a preview of the new video. The prompt may also, or alternatively, include an indication of the match along with a list of previously recorded image data that are part of the match (including image data previously recorded by the same device or another device) to allow the user to select one or more of the previously recorded image data to combine with the new video in block 508. If the user chooses not to combine in block 512, the camera application returns to an idle state from which the user may record a new video by returning to block 502. If the user indicates that the paused video is now complete or otherwise desires to output the video, an output video is determined in block 514, including image frames from all matched videos.

[0060] Pausing and matching of the video can be determined based on image content in image frames received from an image sensor. Figure 6 is a flow chart illustrating a method of detecting a video match according to some embodiments of the present disclosure. The method 600 includes, at block 602, a camera application running on an image capture device receiving a preview stream of image frames. At block 604, the device detects an object in the image frames from the preview stream. At block 606, the device determines whether a matching object exists between the image frame in the preview stream detected at block 604 and an object linked to the previously paused video. The matching process may identify the match by comparing the videos to identify one or more criteria including at least one of: identifying a partial scene match between the first image data and the second image data; using an artificial intelligence (AI) algorithm to identify common features between the first image frame and the second image frame; identifying a face match between the first image data and the second image data; identifying a location match between the first image data and the second image data; identifying a camera settings match between the first image data and the second image data; identifying an object match between the first image data and the second image data; and / or identifying an audio match between the first image data and the second image data.

[0061] If no match is determined at block 608, the method 600 will record a new video as a normal video recording operation at block 616. If a match is determined at block 608, the user is prompted at block 610 whether to continue the video corresponding to the matched snapshot image frame. If the user does not request to unpause the matched video, the method 600 will record a new video as a normal video recording operation at block 616. If the user requests to unpause the matched video, the method 600 will record a new video with a link to the previously paused matched video at block 614.

[0062] In some embodiments, the paused video may be stored in a buffer from the processing pipeline via an image signal processor, as shown in the example data flow of FIG. 7. FIG. 7 illustrates a block diagram of an image capture device illustrating the saving of the paused video to a buffer, according to some embodiments of the present disclosure. The device 100 may include an image signal processor (ISP) 112 having an IFE 135 and an IPE 136. The IFE 135 may receive image data from the first image sensor 101 and form image frames in a buffer portion of the memory 106, which may be non-volatile memory, volatile memory, or a combination of non-volatile and volatile memory. When the video recording is paused, the image frames may be stored in the memory 106 as a paused video 702. The second video 704 may be stored in the memory 106 as well. When the video 702 and the video 704 are matched, such as via a snapshot frame linked to the video 702, the IPE 136 may retrieve the paused video 702 and the video 704 from the memory 106. The IPE 136 may perform processing on the videos 702 and 704, such as combining image frames from the videos 702 and 704 and encoding the combined image frames as an output video 706. The output video 706 may be stored in the memory 106 and / or transmitted to remote storage over a wireless network, such as a 5G NR wireless network.

[0063] In one or more aspects, techniques for supporting image processing may include additional aspects, such as any single aspect or any combination of aspects described below or in connection with one or more other processes or devices described elsewhere herein. In a first aspect, supporting image processing may include an apparatus, such as an image capture device, configured to perform operations including determining a match between first image data and second image data, presenting an indication of the match to a user, receiving user input responsive to the indication, and determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data. Furthermore, the apparatus may perform or operate according to one or more aspects as described below. In some implementations, the apparatus includes a wireless device, such as a UE or a BS. In some implementations, the apparatus may include at least one processor and a memory coupled to the processor. The processor may be configured to perform the operations described herein with respect to the apparatus. In some other implementations, the apparatus may include a non-transitory computer-readable medium having program code recorded thereon, the program code being executable by a computer to cause the computer to perform the operations described herein with respect to the apparatus. In some implementations, the apparatus may include one or more means configured to perform the operations described herein. In some implementations, a method of wireless communication may include one or more operations described herein with respect to the apparatus.

[0064] In a second aspect, in combination with the first aspect, receiving the first image data includes receiving a series of image frames comprising a video sequence.

[0065] In a third aspect, in combination with one or more of the first aspect or the second aspect, the apparatus may be further configured to perform an operation including displaying the second image data in the preview, and presenting the indication includes presenting the indication together with the preview.

[0066] In a fourth aspect, in combination with one or more of the first to third aspects, determining the match includes at least one of: identifying a partial scene match between the first image data and the second image data; identifying common features between the first image frame and the second image frame using an artificial intelligence (AI) algorithm; identifying a facial match between the first image data and the second image data; identifying a location match between the first image data and the second image data; identifying a camera setting match between the first image data and the second image data; identifying an object match between the first image data and the second image data; or identifying an audio match between the first image data and the second image data.

[0067] In a fifth aspect, in combination with one or more of the first to fourth aspects, presenting an indication of a match to a user includes presenting a representation of the first image data.

[0068] In a sixth aspect, in combination with one or more of the first to fifth aspects, presenting the indication of the match to the user includes presenting a representation of the plurality of previously recorded image data including at least a portion of the first image data, receiving user input includes receiving a selection of at least one of the plurality of previously recorded image data for appending to the second image data, and determining the output image data includes appending at least a portion of image data of the selection of at least one of the plurality of previously recorded image data to at least a portion of the second image data.

[0069] In a seventh aspect, in combination with one or more of the first to sixth aspects, determining the match includes matching a first user gesture appearing in the first image data with a second user gesture appearing in the second image data.

[0070] In an eighth aspect, in combination with one or more of the first to seventh aspects, the apparatus may be further configured to perform an operation including storing the first image data by recording the first image data in a buffer, and determining the output image data includes encoding a video file including at least a portion of the first image data and at least a portion of the second video by retrieving at least a portion of the first image data from the buffer.

[0071] In a ninth aspect, in combination with one or more of the first to eighth aspects, determining the output image data includes encoding a video file by receiving at least a portion of the second image data.

[0072] In a tenth aspect, in combination with one or more of the first to ninth aspects, an image capture device includes an image sensor, a memory storing processor-readable code, and at least one processor coupled to the memory and the image sensor, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform steps including determining a match between the first image data and the second image data, presenting an indication of the match to a user, receiving user input responsive to the indication, and / or determining output image data including at least a portion of the first image data and at least a portion of the second image data based on the determining the match and the user input indicating combining the first image data and the second image data.

[0073] In an eleventh aspect, in combination with one or more of the first to tenth aspects, the image capture device may include an artificial intelligence (AI) engine coupled to the processor, and determining a match by the processor includes identifying common features between the first image data and the second image data using the artificial intelligence (AI) engine.

[0074] In a twelfth aspect, in combination with one or more of the first to eleventh aspects, the processor includes an image signal processor including an image front end (IFE) and an image processing engine (IPE) coupled to the IFE.

[0075] In a thirteenth aspect, in combination with one or more of the first to twelfth aspects, the image signal processor is configured to perform operations including recording, by the IFE, first image data from the image sensor into a buffer portion of the memory and recording, by the IFE, second image data from the image sensor into a buffer portion of the memory, and determining the output video includes encoding a video file including the first video and the second video by retrieving, by the IPE, the first video from the buffer and retrieving the second video from the buffer.

[0076] In a fourteenth aspect, in combination with one or more of the first aspect to the thirteenth aspect, the image capture device includes a display coupled to the at least one processor, the at least one processor configured to display the second image data in a preview on the display, and presenting the indication includes presenting the indication together with the preview.

[0077] Those skilled in the art will appreciate that information and signals may be represented using any of a wide variety of technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0078] The components, functional blocks, and modules described herein with respect to Figures 1-7 include, among many examples, processors, electronic devices, hardware devices, electronic components, logic circuits, memories, software code, firmware code, or any combination thereof. Software should be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, and / or functions, among many examples, whether referred to as software, firmware, middleware, microcode, hardware description language, or the like. Furthermore, features discussed herein may be implemented via dedicated processor circuitry, via executable instructions, or combinations thereof.

[0079] Those skilled in the art will further appreciate that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosure. Those skilled in the art will also readily recognize that the order or combination of components, methods, or interactions described herein are merely examples, and that the components, methods, or interactions of various aspects of the disclosure can be combined or performed in ways other than those shown and described herein.

[0080] The various example logic, logic blocks, modules, circuits, and algorithmic processes described in connection with the implementations disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. Interoperability between hardware and software has been described generally in terms of functionality and is illustrated in the various example components, blocks, modules, circuits, and processes described above. Whether such functionality is implemented in hardware or software depends on the particular application and design constraints imposed on the overall system.

[0081] The hardware and data processing devices used to implement the various example logic, logic blocks, modules, and circuits described in connection with the aspects disclosed herein may be implemented or performed using general purpose single-chip or multi-chip processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. A general purpose processor may be a microprocessor, or any conventional processor, controller, microcontroller, or state machine. In some implementations, a processor may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In some implementations, certain processes and methods may be performed by circuitry that is specialized for a given function.

[0082] In one or more aspects, the functions described may be implemented in hardware, digital electronic circuitry, computer software, firmware, or any combination thereof, including the structures disclosed herein and their structural equivalents. Implementations of the subject matter described herein may also be implemented as one or more computer programs, i.e., as one or more modules of computer program instructions encoded on a computer storage medium for execution by or for controlling the operation of a data processing apparatus.

[0083] If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code. The processes of the methods or algorithms disclosed herein may be implemented in processor-executable software modules that may reside on a computer-readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that can be enabled to transfer a computer program from one place to another. A storage medium may be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection may be properly termed a computer-readable medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray Disc, where disks typically reproduce data magnetically while discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media. Furthermore, operations of a method or algorithm may reside as one or any combination or set of codes and instructions on machine-readable and computer-readable media, which may be embodied in a computer program product.

[0084] Various modifications to the implementations described in this disclosure may be readily apparent to those skilled in the art, and the general principles defined herein may be applied to several other implementations without departing from the spirit or scope of the present disclosure. Thus, the claims are not intended to be limited to the implementations shown herein, but are to be accorded the widest scope consistent with the present disclosure, the principles and novel features disclosed herein.

[0085] Moreover, those skilled in the art will readily appreciate that the terms "upper" and "lower" may be used to facilitate description of the figures and are intended to indicate relative positions corresponding to the orientation of the figures on a suitably oriented page, and may not reflect the proper orientation of any device in which they may be implemented.

[0086] Certain features that are described herein in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination, and may even be initially claimed as such, one or more features from a claimed combination may in some cases be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0087] Similarly, although operations are shown in the figures in a particular order, this should not be understood as requiring such operations to be performed in the particular order shown, or in sequential order, or that all of the operations shown be performed in order to achieve a desired result. Furthermore, the figures may generally depict one or more exemplary processes in the form of a flow diagram. However, other operations not shown may be incorporated into the exemplary process depicted in the schematic. For example, one or more additional operations may be performed before, after, simultaneously with, or between any of the depicted operations. In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the program components and systems described may generally be integrated together in a single software product or packaged in multiple software products. Furthermore, several other implementations are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired results.

[0088] As used herein, including the claims, the term "or", when used in a list of two or more items, means that any one of the listed items can be employed alone, or any combination of two or more of the listed items can be employed. For example, if a composition is described as containing components A, B, or C, the composition may contain only A, only B, only C, a combination of A and B, a combination of A and C, a combination of B and C, or a combination of A, B, and C. Also, as used herein, including the claims, "or" as used in a list of items ending with "at least one of" indicates a disjunctive list, such as, for example, a list of "at least one of A, B, or C" means any of A, B, or C, or AB, or AC, or BC, or ABC (i.e., A and B and C), or any of them in any combination thereof. The term "substantially" is defined as a majority, but not necessarily the entirety, of what is specified, as will be understood by one of ordinary skill in the art (and is inclusive of what is specified, e.g., substantially 90 degrees includes 90 degrees and substantially parallel includes parallel). In any of the disclosed implementations, the term "substantially" can be replaced with "within a [percentage] of" what is specified, where percentage includes 0.1, 1, 5, or 10 percent.

[0089] The above description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the embodiments and designs described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. 1. A method comprising: determining a match between the first image data and the second image data; presenting an indication of the match to a user; and receiving a user input responsive to the indication; determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data; A method comprising:

2. The method of claim 1 , wherein receiving the first image data comprises receiving a series of image frames comprising a video sequence.

3. The method of claim 1 , further comprising displaying the second image data in a preview, and wherein presenting the indication comprises presenting the indication together with the preview.

4. determining the match, identifying a partial scene match between the first image data and the second image data; using an artificial intelligence (AI) algorithm to identify common features between the first image data and the second image data; identifying a facial match between the first image data and the second image data; identifying a location match between the first image data and the second image data; identifying a match of camera settings between the first image data and the second image data; identifying an object match between the first image data and the second image data; or Identifying an audio match between the first image data and the second image data. The method of claim 1 , comprising at least one of:

5. The method of claim 1 , wherein presenting the indication of the match to the user comprises presenting a representation of the first image data.

6. presenting the indication of the match to the user includes presenting a representation of a plurality of previously recorded image data including at least a portion of the first image data; receiving the user input includes receiving a selection of at least one of the plurality of previously recorded image data for appending to the second image data; determining the output image data includes appending at least a portion of image data of the at least one of the plurality of previously recorded image data to at least a portion of the second image data. The method of claim 1.

7. The method of claim 1 , wherein determining the match comprises matching a first user gesture appearing in the first image data with a second user gesture appearing in the second image data.

8. storing the first image data by recording the first image data in a buffer; determining the output image data includes retrieving at least a portion of the first image data from the buffer to encode a video file including at least a portion of the first image data and at least a portion of the second image data. The method of claim 1.

9. 2. The method of claim 1, wherein determining the match between the first image data and the second image data comprises determining a match between a first snapshot associated with the first image data and the second image data.

10. An apparatus comprising: a memory storing processor readable code; at least one processor coupled to the memory and configured to execute the processor readable code, the processor readable code causing the at least one processor to: determining a match between the first image data and the second image data; presenting an indication of the match to a user; and receiving a user input responsive to the indication; determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data; At least one processor that executes steps including: An apparatus comprising:

11. The apparatus of claim 10 , wherein receiving the first image data comprises receiving a series of image frames comprising a video sequence.

12. The processor readable code further comprises: displaying the second image data in a preview, and presenting the indication includes presenting the indication together with the preview.

11. The apparatus of claim 10.

13. determining the match, identifying a partial scene match between the first image data and the second image data; using an artificial intelligence (AI) algorithm to identify common features between the first image data and the second image data; identifying a facial match between the first image data and the second image data; identifying a location match between the first image data and the second image data; identifying a match of camera settings between the first image data and the second image data; identifying an object match between the first image data and the second image data; or The apparatus of claim 10 further comprising at least one of: identifying an audio match between the first image data and the second image data.

14. The apparatus of claim 10 , wherein presenting the indication of the match to the user comprises presenting a representation of the first image data.

15. presenting the indication of the match to the user includes presenting a representation of a plurality of previously recorded image data including at least a portion of the first image data; receiving the user input includes receiving a selection of at least one of the plurality of previously recorded image data for appending to the second image data; determining the output image data includes appending at least a portion of image data of the at least one of the plurality of previously recorded image data to at least a portion of the second image data.

11. The apparatus of claim 10.

16. 11. The apparatus of claim 10, wherein determining the match comprises matching a first user gesture appearing in the first image data with a second user gesture appearing in the second image data.

17. The processor readable code further comprises: storing the first image data by recording the first image data in a buffer; 11. The apparatus of claim 10, wherein determining the output image data includes encoding a video file including at least a portion of the first image data and at least a portion of the second image data by retrieving at least a portion of the first image data from the buffer.

18. The apparatus of claim 10 , wherein determining the output image data comprises encoding a video file by receiving the at least a portion of the second image data.

19. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to: determining a match between the first image data and the second image data; presenting an indication of the match to a user; and receiving a user input responsive to the indication; determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data; A non-transitory computer-readable medium for causing a computer to perform operations including:

20. 20. The non-transitory computer-readable medium of claim 19, wherein receiving the first image data comprises receiving a series of image frames comprising a video sequence.

21. 20. The non-transitory computer-readable medium of claim 19, wherein the instructions further cause the processor to perform operations including displaying the second image data in a preview, and wherein presenting the indication includes presenting the indication together with the preview.

22. 20. The non-transitory computer-readable medium of claim 19, wherein determining the match includes matching a first user gesture appearing in the first image data with a second user gesture appearing in the second image data.

23. The instructions further cause the processor to: performing operations including storing the first image data by recording the first image data in a buffer; determining the output image data includes retrieving at least a portion of the first image data from the buffer to encode a video file including at least a portion of the first image data and at least a portion of the second image data.

20. The non-transitory computer readable medium of claim 19.

24. presenting the indication of the match to the user includes presenting a representation of a plurality of previously recorded image data including at least a portion of the first image data; receiving the user input includes receiving a selection of at least one of the plurality of previously recorded image data for appending to the second image data; determining the output image data includes appending at least a portion of image data of the at least one of the plurality of previously recorded image data to at least a portion of the second image data.

20. The non-transitory computer readable medium of claim 19.

25. The instructions further cause the processor to: storing the first image data by recording the first image data in a buffer; determining the output image data includes retrieving at least a portion of the first image data from the buffer to encode a video file including at least a portion of the first image data and at least a portion of the second image data.

20. The non-transitory computer readable medium of claim 19.

26. 1. An image capture device, comprising: An image sensor; a memory storing processor readable code; at least one processor coupled to the memory and the image sensor and configured to execute the processor readable code, the processor readable code causing the at least one processor to: determining a match between the first image data and the second image data; presenting an indication of the match to a user; and receiving a user input responsive to the indication; determining output image data including at least a portion of the first image data and at least a portion of the second image data based on determining the match and the user input indicating combining the first image data and the second image data; At least one processor that executes steps including: An image capture device comprising:

27. determining the match includes matching a second user gesture in the second image data with a first user gesture in the first image data.

27. The image capture device of claim 26.

28. an artificial intelligence (AI) engine coupled to the processor; determining the match by the processor includes identifying common features between the first image data and the second image data using the artificial intelligence (AI) engine.

27. The image capture device of claim 26.

29. The at least one processor includes an image signal processor, the image signal processor comprising: an imaging front end (IFE); an image processing engine (IPE) coupled to the IFE; Including, said image signal processor comprising: recording, by the IFE, the first image data from the image sensor into a buffer portion of the memory; recording, by the IFE, the second image data from the image sensor into the buffer portion of the memory; configured to perform operations including: determining the output image data includes encoding, by the IPE, a video file including the first image data and the second image data by retrieving, by the IPE, the first image data from the buffer and retrieving, by the IPE, the second image data from the buffer, 27. The image capture device of claim 26.

30. a display coupled to the at least one processor; the at least one processor is configured to display the second image data in a preview on the display, and presenting the indication includes presenting the indication together with the preview.

27. The image capture device of claim 26.

Images