Zoom in or out with slow-motion video capture

Abstract

An aspect is described for a device for generating video content, the device including a first camera and one or more processors coupled to the first camera, the one or more processors configured to receive a first set of frames captured by the first camera at a first rate, process the first set of frames to generate video content for display, receive a selection to zoom in or out, receive a second set of frames captured by the first camera at a second rate less than the first rate during the zoom in or out to a zoom threshold for the first camera, process the second set of frames to generate video content for display, and generate video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

Description

Claiming priority

[0001]

[0001] This application claims priority to U.S. Application No. 17 / 156,219, filed January 22, 2021, the entire contents of which are incorporated herein by reference. [Technical field]

[0002]

[0002] This disclosure relates to image processing. [Background technology]

[0003]

[0003] A camera device includes one or more cameras that capture frames (e.g., images). Examples of camera devices include standalone digital cameras or digital video camcorders, camera-equipped wireless communication device handsets such as mobile phones, cellular or satellite radio telephones having one or more cameras, camera-equipped personal digital assistants (PDAs), panels or tablets, gaming devices, computing devices, or any device with digital imaging or video capabilities, including cameras such as so-called "webcams."

[0004]

[0004] The camera device processes the captured frames and outputs the frames for display. In some examples, the camera device captures frames at a first rate and displays the frames at a second rate that is less than the first rate. A result of displaying the frames at the second rate that is less than the first rate at which the frames are captured is that the image content appears to move in slow motion. Summary of the Invention

[0005]

[0005] Generally, this disclosure describes techniques for slow motion video capture with zoom-in or zoom-out capabilities in a multi-camera device. Each of the cameras on the camera device may be associated with a zoom factor range between respective zoom thresholds. For example, the zoom factor for a first camera may be between 0.5x and 1x magnification of the captured image content. In this example, the zoom factor range is from 0.5x magnification to 1x magnification, with the first zoom threshold being 0.5x and the second zoom threshold being 1x. Other cameras of the device may have other zoom factor ranges (e.g., magnifications from 1x to 2x, etc.). Also, each of the cameras may be configured to capture frames according to respective camera parameters. For zooming in (e.g., increasing magnification) or zooming out (e.g., decreasing magnification), the camera processor processes frames captured by the first camera based on the camera parameters for the first camera until a zoom threshold for the first camera is reached, at which point the camera processor processes frames captured by the second camera (e.g., the camera processor switches from the wide-angle camera to the telephoto camera).

[0006] In one or more examples, during zooming in or out (e.g., during the time that frames captured by the first camera are zoomed in or out from a current zoom factor to a zoom threshold), the first camera may be configured to capture frames at a lower frame rate than the rate at which the first camera may capture frames when not zooming in or out. As will be described in more detail, by reducing the capture rate during zooming in or out, example techniques may reduce the number of inferior quality frames captured and displayed by the first camera before frames captured by the second camera are displayed.

[0007]

[0007] For example, in slow motion, captured frames are displayed at a rate lower than the rate at which the frames were captured. Also, during zooming in or out, the camera parameters may not be optimized, which results in poorer image quality of the frames captured during the time of zooming in or out from the current zoom factor to the zoom threshold. In slow motion, if there are many frames with poor image quality, the viewer experience is adversely affected. By reducing the number of poor quality frames captured and then displayed, the exemplary technique may improve the overall operation of the camera device, and thus the resulting device may provide the ability to zoom in or out even when slow motion video capture is enabled.

[0008]

[0008] In one example, the disclosure describes a device for generating video content, the device comprising a first camera and one or more processors coupled to the first camera, the one or more processors configured to receive a first set of frames captured by the first camera at a first rate, process the first set of frames to generate video content for display, receive a selection to zoom in or zoom out, receive a second set of frames captured by the first camera at a second rate less than the first rate during zooming in or out up to a zoom threshold for the first camera, process the second set of frames to generate video content for display, and generate video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

[0009]

[0009] In one example, the disclosure describes a method for generating video content comprising receiving, using one or more processors, a first set of frames captured by a first camera at a first rate; processing, using the one or more processors, the first set of frames to generate video content for display; receiving, using the one or more processors, a selection to zoom in or out; receiving, using the one or more processors, during the zooming in or out to a zoom threshold for the first camera, a second set of frames captured by the first camera at a second rate less than the first rate, using the one or more processors; processing, using the one or more processors, the second set of frames to generate video content for display; and generating, using the one or more processors, video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

[0010]

[0010] In one example, the disclosure describes a computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to receive a first set of frames captured by a first camera at a first rate, process the first set of frames to generate video content for display, receive a selection to zoom in or out, receive a second set of frames captured by the first camera at a second rate less than the first rate while zooming in or out to a zoom threshold for the first camera, process the second set of frames to generate video content for display, and generate video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

[0011]

[0011] In one example, the disclosure describes a device for generating video content comprising means for receiving a first set of frames captured by a first camera at a first rate, means for processing the first set of frames to generate video content for display, means for receiving a selection to zoom in or zoom out, means for receiving a second set of frames captured by the first camera at a second rate less than the first rate during zooming in or out to a zoom threshold for the first camera, means for processing the second set of frames to generate video content for display, and means for generating video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

[0012] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0013] [Figure 1]

[0013] A block diagram of a device configured to implement one or more of the example techniques described in this disclosure. [Diagram 2]

[0014] 2 is a block diagram illustrating the camera processor of the device of FIG. 1 in further detail. [Diagram 3]

[0015] 1 is a conceptual diagram illustrating an example method for effecting zooming in or out. [Figure 4]

[0016] 4 is a timing diagram illustrating an example of zooming in or out. [Diagram 5]

[0017] 6 is a flowchart illustrating an example method of operation according to one or more examples described in this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014]

[0018] Exemplary techniques described in this disclosure relate to slow-motion video capture with zoom-in or zoom-out capabilities in a multi-camera device. The camera device may include multiple cameras to support zooming in or out of video content. For example, the camera device may include an ultra-wide camera (configured to capture a field of view angle greater than 120°, e.g., 120° to 180°), a wide camera (configured to capture a field of view angle greater than 60°, e.g., 60° to 120°), and a telephoto camera (configured to capture a field of view angle less than 60°, e.g., 10° to 60°). Each of the multiple cameras may be associated with a respective zoom factor range and zoom threshold. For example, the ultra-wide camera may provide a zoom factor range of 0.5x to 1x, where 0.5x and 1x are zoom thresholds. The wide camera may provide a zoom factor range of 1x to 2x, where 1x and 2x are zoom thresholds. A telephoto camera may offer a zoom factor greater than 2x, where 2x is the zoom threshold. The zoom factors of 0.5x, 1x, and 2x may refer to the reproduction ratio of an object on the respective camera's sensor (e.g., at a 0.5x zoom factor for an ultra-wide camera, a ½ inch object is reproduced as 0.25 of the object's size on the ultra-wide camera's sensor).

[0015]

[0019] Each camera may be associated with a respective zoom factor based on optical and / or digital zoom capabilities. That is, each camera may provide zooming in or out over a zoom factor range based on one or a combination of optical and digital zoom. Examples of optical and digital zoom are described below.

[0016]

[0020] In the above example, there is no overlap in the zoom factor ranges for each of the cameras. However, in some examples, there may be overlap in the zoom factor ranges. For example, the zoom factor range for the ultra-wide camera may be 0.5x to 1.3x, the zoom factor range for the wide camera may be 1x to 2.3x, and the zoom factor range for the telephoto camera may be greater than 2x. In this example, the zoom thresholds for the ultra-wide camera are 0.5x and 1.3x, the zoom thresholds for the wide camera are 1x and 2.3x, and the zoom threshold for the telephoto camera is 2x.

[0017]

[0021] The zoom factor ranges and zoom thresholds are provided by way of example only and should not be considered limiting. The example techniques described in this disclosure are not limited to any particular zoom factor ranges or zoom thresholds.

[0018]

[0022] In some examples, when zooming in or out, there is a handoff between different cameras once the respective zoom thresholds are reached. For example, a user of a camera device may first capture video content using an ultra-wide camera with a zoom factor of 0.5x, and a camera processor of the camera device may process frames captured by the ultra-wide camera. The user may then zoom in (e.g., using a button or pinch motion). In this example, while zooming in, the camera processor may process frames captured by the ultra-wide camera until the zoom threshold of the ultra-wide camera is reached (e.g., 1x). The camera processor may then process frames captured by the wide camera.

[0019]

[0023] One exemplary method for maintaining the quality of the captured video content is for each camera to determine its respective camera parameters prior to video capture. Examples of camera parameters include 3A parameters (autofocus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters), as described in more detail. Even if the camera processor is not processing frames from a particular camera, the camera parameters are already known. In that case, when the camera processor should switch from processing frames from a current camera to a subsequent camera, there is no delay in determining the camera parameters for the subsequent camera, since the camera parameters for the subsequent camera have been pre-determined. In this way, a handoff from a current camera to a subsequent camera (e.g., a switch in processing frames) can be performed with limited impact on quality and limited delay.

[0020]

[0024] In addition to the camera device including multiple cameras to support zooming in or out, the camera device also provides an option for slow-motion video capture. In slow-motion video capture, the camera of the camera device captures frames at a first rate that is greater than a second rate at which the captured frames are displayed. As an example, the camera may capture 10 seconds of video content at a capture rate of 120 frames per second (fps). In this example, there are 1200 frames (e.g., 120fps*10 seconds). The camera device, or some other device, may display 1200 frames at a display rate of 30fps. In this example, the 1200 frames are displayed over 40 seconds (e.g., 1200 frames / 30fps). Thus, in this example, the 10 seconds of video content is displayed over 40 seconds, and the viewer perceives the video content as being in slow motion (e.g., the rate of movement of objects in the video content is slower than the rate at which the objects move in reality).

[0021]

[0025] With slow motion video capture enabled, there may be technical difficulties in enabling zooming in or out. For example, in response to a request to zoom in or out, the camera processor performs a digital zoom in or out on frames captured by the current camera until a zoom threshold is reached. As an example, the current camera may be an ultra-wide camera, and in response to a zoom in, the camera processor may perform a digital zoom in on frames captured by the current camera until a zoom threshold of 1x is reached.

[0022]

[0026] Digital zoom in or out involves cropping and resizing of the frame during a linear increase or decrease in the zoom factor. For example, a camera processor may receive a frame from an ultra-wide camera, and to perform a 0.6x zoom, the camera processor may crop 10% of the perimeter of the frame, resize the result of the cropping, etc. The camera processor may perform digital zoom in or out by increasing the zoom factor over a period of time (e.g., to 0.6x, 0.7x, 0.8x, 0.9x, 1x) until a zoom threshold is reached. Although examples are described with respect to an ultra-wide camera, exemplary techniques for zooming in or out may be similar for other cameras.

[0023]

[0027] In some examples, digital zooming in or out causes a decrease in image quality due to resizing. For example, camera parameters may not be precise for frames captured during the time the camera processor is performing zooming in or out. For example, camera parameters may be set based on a particular zoom factor (e.g., 0.5x for an ultra-wide camera, 1x for a wide camera, and 2x for a telephoto camera). A frame captured during zooming in or out may be considered as a frame captured with a different video content compared to a video content without zooming in or out. Camera parameters, such as autofocus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters, determined for a particular zoom factor, may not be precise for frames captured with a different video content, where a frame captured during zooming in or out has a different video content.

[0024]

[0028] The above describes an example of digital zoom. However, for capturing using optical zoom, the same problem of reduced image quality exists. In optical zoom, the camera lens and / or sensor moves to provide zooming in or out. The camera parameters may be set for the lens or sensor at a particular location, and the movement of the lens or sensor may cause the camera parameters to become inaccurate.

[0025]

[0029] After the zoom threshold for the current camera is reached, the camera processor outputs the frame captured by the subsequent camera. In this case, the frame captured by the subsequent camera is captured using the already determined camera parameters for the subsequent camera. Therefore, the image quality is relatively high for the frame captured by the subsequent camera. The above exemplary process of generating a frame with a lower image quality is repeated if there is another request for zooming in or out using the subsequent camera.

[0026]

[0030] As described above, in slow motion video capture, the rate at which frames are captured is greater than the rate at which they are displayed, so as to cause a slow motion effect. Also, as described, during zooming in or out, the image quality of the captured frames may be relatively low. In some examples, these frames with lower image quality are displayed for a longer amount of time than the amount of time to capture, so there may be an undesirably long period of time during which low quality frames are displayed. For example, the current camera may be capturing frames at 120 fps, and the amount of time to zoom in or out to the zoom threshold may be 0.5 seconds. In this example, during zooming in or out, the current camera may capture 60 frames (e.g., 120 fps * 0.5 seconds), and the image quality of the 60 frames may be poorer than without zooming in or out. The display rate may be 30 fps. Thus, the 60 frames may be displayed for 2 seconds (e.g., 60 frames / 30 fps). In this example, there may only be 0.5 seconds worth of poor quality frames, but it may take 2 seconds to display the poor quality frames, which may be undesirable to a viewer.

[0027]

[0031] According to one or more examples described in this disclosure, during normal operation with slow motion enabled (e.g., without zooming in or out), the camera may capture a first set of frames at a first rate (e.g., 120 fps or more), and during zooming in or out with slow motion enabled, the camera may capture a second set of frames at a second rate lower than the first rate (e.g., 30 fps or less). The image quality of the second set of frames may be inferior to the image quality of the first set of frames.

[0028]

[0032] The camera device or some other device may display the first set for frames and the second set for frames at a third rate (e.g., 30 fps). In this example, if 9.5 seconds of video content is captured by the first set of frames and 0.5 seconds of video content is captured by the second set of frames, there will be 1140 frames with relatively high quality image content (e.g., 9.5 seconds * 120 fps) and 15 frames with relatively low quality image content (e.g., 0.5 seconds * 30 fps). In that case, during the display, there will be 38 seconds of playback of frames with relatively high quality image content (e.g., 9.5 seconds of video content is expanded to 38 seconds to create a slow motion effect). There will also be 0.5 seconds of playback of frames with relatively low quality image content (e.g., 0.5 seconds of video content is displayed within 0.5 seconds). The amount of frame time with relatively low quality image content is reduced from 2 seconds to 0.5 seconds when the capture rate is 120 fps, so there may be little impact on the viewer experience.

[0029]

[0033] 1 is a block diagram of a device configured to implement one or more of the example techniques described in this disclosure. Examples of camera devices 10 include standalone digital cameras or digital video camcorders, camera-equipped wireless communication device handsets such as mobile phones with multiple cameras, cellular or satellite radio telephones, camera-equipped personal digital assistants (PDAs), panels or tablets, gaming devices, computing devices, or any device with digital imaging or video capabilities, including cameras such as so-called "webcams."

[0030]

[0034] As shown in the example of Figure 1, camera device 10 includes cameras 12A-12C (e.g., each having an image sensor and a lens), a camera processor 14 and a memory 20 local to camera processor 14, a central processing unit (CPU) 16, a graphical processing unit (GPU) 18, a user interface 22, a memory controller 24 that provides access to a system memory 30, and a display interface 26 that outputs signals that cause graphical data to be displayed on a display 28. Although the example of Figure 1 shows one camera processor 14, in some examples there may be multiple camera processors (e.g., one for each of cameras 12A-12C). Techniques described in this disclosure with respect to camera processor 14 may include examples in which one or more camera processors perform example techniques.

[0031]

[0035] Also, although various components are shown as separate components, in some examples the components may be combined to form a system on a chip (SoC). As an example, the camera processor 14, the CPU 16, the GPU 18 and the display interface 26 may be formed on a common integrated circuit (IC) chip. In some examples, one or more of the camera processor 14, the CPU 16, the GPU 18 and the display interface 26 may be in separate IC chips. Various other permutations and combinations are possible and the techniques should not be considered limited to the examples shown in FIG. 1.

[0032]

[0036] 1 (whether formed on one device or different devices) may be formed as at least one of fixed function or programmable circuitry, such as in one or more microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), or other equivalent integrated or discrete logic circuitry. Examples of local memory 20 and system memory 30 include one or more volatile or non-volatile memory or storage devices, such as random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic data medium, or optical storage medium.

[0033]

[0037] The various units shown in Figure 1 communicate with each other using a bus 32. The bus 32 can be any of a variety of bus structures, such as a third generation bus (e.g., a HyperTransport bus or an InfiniBand bus), a second generation bus (e.g., an Advanced Graphics Port bus, a Peripheral Component Interconnect (PCI) Express bus, or an Advanced eXtensible Interface (AXI) bus), or another type of bus or device interconnect. The particular configuration of buses and communication interfaces between the different components shown in Figure 1 is merely exemplary, and other configurations of camera devices and / or other image processing systems having the same or different components may be used to implement the techniques of this disclosure.

[0034]

[0038] Camera device 10 includes cameras 12A-12C. Each of cameras 12A-12C may be configured to capture frames having image content according to respective camera parameters, as described below. Cameras 12A-12C may output their captured frames to camera processor 14 for image processing. As described in more detail below, in one or more examples, camera processor 14 may control the rate at which cameras 12A-12C capture frames, such as during zooming in or out, to reduce the number of frames having relatively poorer quality that are displayed.

[0035]

[0039] Camera processor 14 outputs the resulting frames having image content (e.g., pixel values ​​for each of the image pixels) to system memory 30 via memory controller 24. That is, camera processor 14 may generate video content for playback that includes the resulting frames. In one or more examples described in this disclosure, the frames may be further processed to generate one or more frames for display. In some examples, rather than camera processor 14 performing the blending, GPU 18 or some other circuitry of camera device 10 may be configured to perform the blending.

[0036]

[0040] This disclosure describes the exemplary techniques as implemented by camera processor 14. However, the exemplary techniques should not be considered limited to camera processor 14 implementing the exemplary techniques. For example, camera processor 14 may be configured to implement the exemplary techniques described in this disclosure in combination with CPU 16, GPU 18, and / or display interface 26. In some examples, each of cameras 12A-12C may include processing circuitry. Thus, examples of processors that may implement the exemplary techniques include processing circuitry that is part of cameras 12A-12C. For example, one or more processors may be configured to implement the exemplary techniques described in this disclosure. Examples of one or more processors include camera processor 14, CPU 16, GPU 18, display interface 26, one or more processing circuits of cameras 12A-12C, or any combination of one or more of camera processor 14, CPU 16, GPU 18, display interface 26, and one or more processing circuits of cameras 12A-12C.

[0037]

[0041] CPU 16 may comprise a general-purpose or special-purpose processor that controls the operation of camera device 10. A user may provide input to camera device 10 to cause CPU 16 to execute one or more software applications. The software applications executing on CPU 16 may include, for example, a media player application, a video game application, a graphical user interface application, or another program. A user may provide input to camera device 10 via one or more input devices (not shown), such as a keyboard, a mouse, a microphone, a touchpad, or another input device, that are coupled to camera device 10 via a user interface 22.

[0038]

[0042] An example of a software application is a camera application. CPU 16 executes the camera application, and in response, the camera application causes CPU 16 to generate content that display 28 outputs. GPU 18 may be configured to process the content generated by CPU 16 for rendering on display 28. For example, display 28 may output information such as light intensity, whether a flash is enabled, and other such information. A user of camera device 10 may interface with display 28 to configure the manner in which images are generated (e.g., with or without a flash, focus settings, exposure settings, and other parameters). As an example, a user of camera device 10 may select to take multiple frames (e.g., multiple pictures), where two or more of the multiple frames are blended together to generate one or more output frames (e.g., to reduce blur). However, taking multiple frames that are blended together may be a default option (e.g., no user selection is required). The camera application also causes CPU 16 to instruct camera processor 14 to capture and process frames of image content captured by camera 12 in a user-defined manner.

[0039]

[0043] Memory controller 24 enables the transfer of data into and out of system memory 30. For example, memory controller 24 may receive memory read and write commands and service such commands with respect to memory 30 in order to provide memory services to components in camera device 10. Memory controller 24 is communicatively coupled to system memory 30. Although memory controller 24 is shown in the example camera device 10 of FIG. 1 as being a processing circuit that is separate from both CPU 16 and system memory 30, in other examples, some or all of the functionality of memory controller 24 may be implemented on one or both of CPU 16 and system memory 30.

[0040]

[0044] System memory 30 may store program modules and / or instructions and / or data that are accessible by camera processor 14, CPU 16, and GPU 18. For example, system memory 30 may store user applications (e.g., instructions for a camera application), obtained frames from camera processor 14, etc. System memory 30 may further store information for use by and / or generated by other components of camera device 10. For example, system memory 30 may act as device memory for camera processor 14.

[0041]

[0045] As shown, camera device 10 includes cameras 12A-12C. Each of cameras 12A-12C may be configured with a different zoom factor and may capture a different amount of field of view. In some examples, each of cameras 12A-12C may be configured with a different lens geometry and different levels of optical zoom (e.g., different zoom factor ranges based on how much the lenses or sensors of cameras 12A-12C can move). The field of view may be a field of view (FOV). In some examples, cameras 12A-12C with the same zoom factor may be capturing the same FOV.

[0042]

[0046] Camera 12A may be an ultra-wide camera (e.g., configured to capture a field of view angle between 120° and 180°), camera 12B may be a wide camera (e.g., configured to capture a field of view angle between 60° and 120°), and camera 12C may be a telephoto camera (e.g., configured to capture a field of view angle between 10° and 60°). Camera 12A may be configured to provide a zoom factor from a first zoom threshold of 0.5X to a second zoom threshold of 1X. Camera 12B may be configured to provide a zoom factor from a third zoom threshold of 1X to a fourth zoom threshold of 2X. Camera 12C may be configured to provide a zoom factor greater than a fifth zoom threshold of 2X.

[0043]

[0047] In the above example, the second zoom threshold and the third zoom threshold are the same, and the fourth zoom threshold and the fifth zoom threshold are the same. However, the technique is not so limited. In some examples, there may be overlap in the zoom factor ranges of cameras 12A-12C. For example, the second zoom threshold for camera 12A may be 1.3X and the third zoom threshold for camera 12B may be 1X. In this example, from 1X to 1.3X zoom, both cameras 12A and 12B may provide such zoom factors.

[0044]

[0048] The examples of the number of cameras 12 on the camera device 10, the viewing angles, and the zoom factor ranges and zoom thresholds are provided merely to aid in understanding. The exemplary techniques should not be considered limited to such examples. There may be more or fewer cameras 12, the viewing angles may be different, and the zoom factor ranges and thresholds may be different.

[0045]

[0049] The zoom factor range associated with each of the cameras 12A-12C does not necessarily imply that the sensors or lenses of the cameras 12A-12C move to provide the desired zoom, although that may be the case. That is, optical zoom is one way to provide zoom, but the techniques are not limited to optical zoom. In some examples, the zoom factor range associated with the cameras 12A-12C may indicate from which of the cameras 12A-12C the camera processor 14 should receive frames to perform digital zoom in or out. Thus, cameras 12A-12C being configured for a particular zoom factor range may include examples where the sensors or lenses of the cameras 12A-12C are moving to provide optical zoom in or out and / or examples where the camera processor 14 receives frames from them to perform digital zoom in or out.

[0046]

[0050] Camera processor 14 is configured to receive image frames from cameras 12A-12C and process the image frames to generate output frames for display (e.g., generate video content for playback). CPU 16, GPU 18, camera processor 14, or some other circuitry may be configured to process the output frames including image content generated by camera processor 14 into images for display on display 28. In some examples, GPU 18 may be further configured to render graphics content on display 28.

[0047]

[0051] Cameras 12A-12C may be configured to capture frames according to their respective camera parameters, which may be determined by camera processor 14 and / or CPU 16. For example, before a user of camera device 10 captures a frame, cameras 12A-12C may determine camera parameters referred to as the 3A parameters: autofocus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters.

[0048]

[0052] In autofocus, CPU 16 may determine an area of ​​an image to be focused on, and therefore image content in the focused area is in focus. Other areas may be in focus, but are not necessarily in focus, depending on the aperture size of cameras 12A-12C. As an example, CPU 16 may utilize a phase detection autofocus (PDAF) technique. CPU 16 may then control cameras 12A-12C based on the area to be focused on (e.g., adjust focal length by adjusting the location of the lenses of cameras 12A-12C). In some examples, CPU 16 may determine objects that are in the foreground and control cameras 12A-12C to focus on the foreground objects. As another example, CPU 16 may utilize object detection techniques, such as face detection, to determine the area to be focused on.

[0049]

[0053] In automatic exposure control, CPU 16 may determine the aperture and shutter speed of cameras 12A-12C based on factors such as external lighting conditions. As an example, camera processor 14 may determine information indicative of light and adjust aperture and shutter speed accordingly to keep the brightness of the image constant (e.g., keep the brightness of desired areas of the frame in the mid-band of brightness values). For example, if there is too much light (i.e., overexposed), details in the image may be lost, and if there is too little light (i.e., underexposed), the image may be too dark and details may not be visible. There are various ways in which exposure is controlled using "metering," such as center-weighted metering maps in which the brightness in the center of the image is kept higher and is gradually reduced near the edges of the image. In some examples, center-weighted metering ensures that the center of the image has values ​​in the middle range of possible brightness levels (i.e., mid-tones). The outer areas may be lighter, darker, or the same, depending on the actual content of the scene being captured. The techniques described in this disclosure are not limited to the example of a center-weighted photometry map (eg, not limited to a photometry map biased towards exposure control for the center of the image).

[0050]

[0054] In auto white balance, the CPU 16 may determine the "color temperature" of the light source, where color temperature refers to the relative warmth or coolness of white. In some cases, the captured image may have an unrealistic color cast. In some examples, the auto white balance algorithm applies a heuristic that analyzes the ratio between red, green, and blue components and outputs suggested red and blue gain levels. When the R, B channels are multiplied by their multipliers, the image will look more balanced, gray areas will look gray, and will not have an unnatural color cast. However, for multi-illumination scenes (e.g., with shadows), it may be more difficult to determine the multipliers, as different multipliers are ideal for different areas.

[0051]

[0055] In one or more examples, camera processor 14 may store respective camera parameters (e.g., 3A camera parameters) for each of cameras 12A-12C. Cameras 12A-12C may utilize AF parameters to adjust focal length and AEC parameters to cause cameras 12A-12C to operate according to shutter and aperture size. Camera processor may utilize AWB as part of processing frames. In some examples, each of cameras 12A-12C may capture frames according to their respective camera parameters. However, camera processor 14 may process frames from only one of cameras 12A-12C, and the remaining frames may be "flushed" (e.g., not processed or stored for display). As described below, in some examples, there may be times when frames from two different cameras 12A-12C are processed, but generally, frames from only one of cameras 12A-12C are processed by camera processor 14.

[0052]

[0056] In some examples, camera processor 14 may be configured as an image processing pipeline. For example, camera processor 14 may include a camera interface that interfaces between camera 12 and camera processor 14. Camera processor 14 may include additional circuitry for processing image content.

[0053]

[0057] In some examples, camera processor 14 may be configured to perform digital zoom in or out. Although camera processor 14 has been described with respect to performing digital zoom in or out, in some examples, cameras 12A-12C may be configured to perform digital zoom in or out. For example, cameras 12A-12C may each include processing circuitry for performing digital zoom in or out.

[0054]

[0058] To zoom in, camera processor 14 may perform cropping and resizing of a frame received from one of cameras 12A-12C. As an example, in response to receiving a request to zoom in from 1X to 2X, camera processor 14 may crop 50% of the perimeter of the frame received from camera 12B and resize the cropped frame. In this example, the center 50% of the frame is now resized to the full size of the frame, and the image content appears to be twice as large.

[0055]

[0059] To avoid abrupt jumps when zooming, when zooming from one zoom threshold to another (e.g., from 1X to 2X), camera processor 14 and / or cameras 12A-12C may gradually increase or decrease the zoom factor. For example, if the zoom factor is suddenly changed from 1X to 2X, the viewer experience may be undesirable. Thus, the zoom factor may change gradually. As an example, the zoom factor (e.g., by camera processor 14 and / or cameras 12A-12C) may increase or decrease linearly, such that the zoom factor is, as an example, 1X, then 1.1X, then 1.2X, and so on up to 2X. Both the linear increase or decrease in zoom factor and the step size of 0.1X are provided as an example and should not be considered limiting. Also, as described above, there may be at least two ways in which the zoom factor changes: (1) based on digital zoom, in which camera processor 14 crops and resizes, and (2) based on optical zoom, which involves movement of the sensors or lenses of cameras 12A-12C. The two methods may be used individually or together.

[0056]

[0060] Again, a user of camera device 10 may zoom in or out on the video content that camera device 10 is recording. In response to a request to zoom in or out, camera processor 14 may process frames captured by one of cameras 12A-12C until the zoom factor reaches a zoom threshold for one of cameras 12A-12C. Because the zoom factor may change gradually from one zoom threshold to another, there may be an amount of time that elapses for the zoom factor to reach the zoom threshold. For example, it may take about 0.5 seconds to reach the zoom threshold. In some examples, the range of time it takes to reach the zoom threshold is between about 0.2 and 0.7 seconds.

[0057]

[0061] Thus, camera processor 14 may process frames captured by a first one of cameras 12A-12C based on camera parameters for the first one of cameras 12A-12C. In response to a request to zoom in or out, camera processor 14 may process frames captured by the first one of cameras 12A-12C during zooming in or out to a zoom threshold for the first one of cameras 12A-12C. For example, during zooming in or out to a zoom threshold for the first camera (e.g., 0.5 seconds), the first camera captures frames based on the initially determined camera parameters. For example, the zoom factor for the first one of cameras 12A-12C may initially be at 1X, and the camera parameters (e.g., AF, AWB, and AEC 3A parameters) may have been determined when the zoom factor for the first camera is 1X. The zoom factor for a second one of cameras 12A-12C may initially be 2X, and the camera parameters (e.g., 3A parameters) may have been determined when the zoom factor for the second camera is 2X.

[0058]

[0062] In that case, during the time (e.g., 0.5 seconds) that the zoom factor changes from 1X to the zoom threshold of 2X, the first camera may continue to capture frames based on the camera parameters determined at the 1X zoom factor. That is, while the zoom factor may be changing (e.g., 1.1X, 1.2X, 1.3X, and so on up to 2X), the first camera may capture frames based on the camera parameters determined at the 1X zoom factor. After reaching the 2X zoom factor, camera processor 14 may begin processing frames captured by the second one of cameras 12A-12C using the camera parameters for the second one of cameras 12A-12C that were determined based on the 2X zoom factor for the second one of cameras 12A-12C.

[0059]

[0063] In the above example, during the time (e.g., 0.5 seconds) that the zoom factor changes from 1X to a zoom threshold of 2X, the first camera may continue to capture frames based on the camera parameters determined at the 1X zoom factor. In general, the camera parameters for each of the cameras 12A-12C may be determined based on each of the cameras 12A-12C being at its respective first zoom threshold. During the time that the zoom factor changes from the first zoom threshold to the second zoom threshold for each of the cameras 12A-12C, each of the cameras 12A-12C may continue to capture frames based on the camera parameters determined at the first threshold.

[0060]

[0064] In some examples, during the time that the zoom factor is changing from the first zoom threshold to the second zoom threshold, the quality of the captured frames may be lower than the quality of the captured frames when the zoom factor is not changing. For example, because the camera parameters for a first camera of the cameras 12A-12C are based on the first zoom threshold for the first camera, the camera parameters may not be precisely accurate for the first camera when the zoom factor is not the first zoom threshold. As an example, the camera parameters for the first camera may be set based on a zoom factor of 1X. The camera parameters may not be accurate for a zoom factor of 1.2X. Thus, a frame captured at a zoom factor of 1.2X with camera parameters determined for a zoom factor of 1X may be of lower quality compared to a frame captured at a zoom factor of 1X.

[0061]

[0065] As an example, whenever there is a zoom, the field of view (FOV) may remain stable until the zoom ends (e.g., until the zoom threshold is reached). The camera parameters during the zoom are not reset or recalculated, but the FOV may change from zooming in or out. Thus, the camera parameters used for capture and processing during zooming are the same parameters that were already set before the zooming started. This results in the zooming FOV not having precise camera parameters. For example, if the focus object (e.g., AF) is somewhere other than the center FOV, zooming will result in an off focus preview, since the object to be focused is no longer in the FOV. The same applies for exposure (e.g., AEC) and white balance (AWB). For example, if the zoom is from a well-lit scene to a shadow, the zooming FOV will remain poorly lit since it uses the per-zoom exposure parameters (i.e., pre-zoom AEC). Thus, during zooming in or out, the captured image tends to have poorer quality than when not zooming in or out.

[0062]

[0066] Although the quality of the captured frames during the time when the zoom factor is changing from the first zoom threshold to the second zoom threshold may be relatively low, in most cases, this relatively low quality of the frames does not affect the viewer experience. For example, each of the cameras 12A-12C may be configured to capture frames at a particular frame rate (e.g., the capture rate is 30 frames per second (fps)). In some cases, the camera device 10 displays frames at a display rate that is the same as the capture rate (e.g., the display rate is 30 fps and the capture rate is 30 fps).

[0063]

[0067] As explained above, it may take 0.5 seconds to zoom from one zoom threshold to another. Thus, during this 0.5 seconds, assuming a capture rate of 30 fps, there may be approximately 15 frames captured with relatively low quality (e.g., 30 fps * 0.5 seconds = 15 frames). If the display rate is also 30 fps, it will take 0.5 seconds to display these frames with relatively low quality. The viewer may not perceive or be affected by 0.5 seconds worth of poor quality frames being displayed.

[0064]

[0068] However, in some cases, having frames of poor quality can impact the viewer experience. As an example, a user may configure the camera device 10 to capture frames for slow motion. In slow motion, the capture rate of frames is greater than the display rate of frames. Thus, it takes more time to display a frame than the amount of time it took to capture the frame. This creates a viewer perception of slow motion, as the image content appears to be moving slower.

[0065]

[0069] As an example, assume the capture rate is 240 fps and the display rate is 30 fps. In this example, one second of video capture results in 240 frames. At a display rate of 30 fps, it will take eight seconds to display that one second of video (e.g., 240 frames / 30 fps). In this example, from the viewer's perspective, objects in the captured frames will appear to move at 1 / 8 the rate at which the objects are moving in reality, thus creating the perception of slow motion.

[0066]

[0070] In some examples, a user of camera device 10 may begin video capture in normal mode. The user may then cause camera device 10 to begin capturing in slow motion. After a period of time, the user may cause camera device 10 to return to capturing in normal mode. For example, during normal mode, cameras 12A-12C may capture frames at 30 fps. During slow motion mode, cameras 12A-12C may capture frames at 120 fps or more.

[0067]

[0071] For a better visual experience, during slow motion, the user may zoom in or out. For example, a first camera of cameras 12A-12C may be set for a zoom factor of 0.5X, where the zoom threshold for the first camera is 0.5X to 1X. Initially, the first camera may capture at 30 fps at 0.5X. Then, the user may cause the camera device 10 to capture in slow motion mode. In this example, the CPU 16 and / or camera processor 14 may cause the first camera to capture at 240 fps. While the first camera is capturing frames, the user may zoom in, thus increasing the zoom factor from 0.5X to 1X (e.g., over about 0.5 seconds). Then, the camera processor 14 may start processing frames captured by a second camera of cameras 12A-12C, which has a zoom factor range of 1X to 2X. With slow motion still enabled, CPU 16 and / or camera processor 14 may cause the second camera to capture at 240 fps with a zoom factor of 1X. At some later point in time, the user may cause camera device 10 to end slow motion, and camera processor 14 may process frames captured by the second camera at 30 fps.

[0068]

[0072] In this manner, there is handoff and switching between the first and second cameras to enable slow motion video capture with zooming in or out. For example, camera processor 14 processes frames from a first one of cameras 12A-12C until a zoom threshold for the first camera is reached, and then camera processor 14 processes frames from a second one of cameras 12A-12C. The capture rate of cameras 12A-12C may be based on whether slow motion is enabled (e.g., a higher capture rate for slow motion and a lower capture rate for normal capture).

[0069]

[0073] However, there may be some problems with enabling zooming in or out with slow motion. As explained above, frames captured during zooming in or out tend to have poorer quality because the camera parameters are not precise with respect to the zoom factor, but the amount of time it takes to display these frames with poorer quality is not long (e.g., 0.5 seconds), so there is a low impact. With slow motion video capture enabled, the amount of time it takes to display these frames with poorer quality for zooming in or out is relatively long.

[0070]

[0074] For example, if the capture rate is 240 fps, during the 0.5 seconds it takes to go from a first zoom threshold to a second zoom threshold of one of the cameras 12A-12C, the number of frames captured is 120 frames (e.g., 240 fps * 0.5 seconds). If the display rate is 30 fps, it takes 4 seconds to display 120 frames (e.g., 120 frames / 30 fps). Thus, in this example, there will be 4 seconds during which frames with relatively poor quality are displayed. As explained above, in normal mode (e.g., not slow motion), there will be 0.5 seconds during which frames with relatively poor quality are displayed. Thus, in slow motion with zooming in or out, if the capture rate is 240 fps, the amount of time during which frames with poor quality are displayed may be 8 times longer than in normal mode with zooming in or out. At higher capture rates, the amount of time that frames with inferior quality are displayed can be much greater (e.g., 960fps will result in 16 seconds of inferior quality, since 960fps / 30fps*0.5 seconds equals 16 seconds).

[0071]

[0075] This disclosure describes example techniques for enabling zooming in or out with slow motion enabled. For example, during zooming in or out to a zoom threshold of one of cameras 12A-12C with slow motion enabled, camera processor 14 may receive frames at a lower rate than the rate at which camera processor 14 receives frames with slow motion enabled but without zooming in or out.

[0072]

[0076] As an example, camera processor 14 may receive a first set of frames captured by a first one of cameras 12A-12C at a first rate with slow-motion video capture enabled (e.g., the first rate is 120 fps or greater). Camera processor 14 may process the first set of frames to generate video content for display. CPU 16 and / or camera processor 14 may receive a selection to zoom in or out. During zooming in or out to a zoom threshold for the first camera, camera processor 14 may receive a second set of frames captured by the first camera at a second rate less than the first rate (e.g., the second rate is 30 fps). Camera processor 14 may process the second set of frames to generate video content for display. For example, camera processor 14 may generate video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate (e.g., 30 fps). As an example, camera processor 14 generating video content for playback may refer to an example in which camera processor 14 provides a first set of frames and a second set of frames for display interface 26 to output for display.

[0073]

[0077] In this example, display interface 26 may output the generated video content including the first set of frames and the second set of frames at a third rate less than the first rate (e.g., the third rate is 30 fps) for display. In some examples, display interface 26 may be configured to generate video content for playback. For example, to generate video content for playback, display interface 26 may receive the first set of frames and the second set of frames, output the first set of frames, and then output the second set of frames in a manner that causes a viewer to perceive a video playback of the frames. Generally, camera processor 14, display interface 26, or some other component generating video content for playback may refer to camera processor 14, display interface 26, or some other component composing, receiving, or processing a set of frames (e.g., the first and second sets of frames) in a manner in which the set of frames may be output for display.

[0074]

[0078] In an illustrative example, in normal mode, camera processor 14 may receive frames at a rate of 30 fps from camera 12A, which is currently at a zoom factor of 0.5X. Thus, camera 12A is configured to capture frames at 30 fps. In this example, assume that camera 12A captures 8 seconds of frames at a capture rate of 30 fps, and thus has captured 240 frames.

[0075]

[0079] The user may then enable slow motion. With slow motion enabled, camera processor 14 may receive frames at a rate of 120 fps from camera 12A, which is currently at a zoom factor of 0.5X. In this example, camera 12A is configured to capture frames at 120 fps with slow motion enabled. Assume camera 12A captures 4 seconds of frames at a capture rate of 120 fps, thus capturing 480 frames.

[0076]

[0080] The user may then enable zooming in (or there may be an automatic zoom, such as in the case of object or face tracking), and thus the zoom factor changes from 0.5X to 1X (e.g., the zoom threshold for camera 12A). Rather than capturing frames at 120 fps during the 0.5 seconds it takes to go from a zoom factor of 0.5X to 1X (e.g., due to a change in lens / sensor position or incremental zoom), camera 12A may capture frames at 30 fps, and camera processor 14 may receive frames at 30 fps, to reduce the number of frames with lower quality captured during the time it takes to go from a zoom factor of 0.5X to 1X. As explained above, frames captured during the 0.5 seconds it takes for the zoom factor to reach the zoom threshold tend to have lower quality. Thus, there will be 15 frames with lower quality.

[0077]

[0081] After reaching a zoom factor of 1X, camera processor 14 may receive frames from camera 12B. Since slow motion is still enabled and there is no further zooming in, camera processor 14 may receive frames from camera 12B at a rate of 120 fps. Assume there are 4 seconds of frames with a capture rate of 120 fps, and therefore camera 12B has captured 480 frames.

[0078]

[0082] Then, assume that the user turns off slow motion. Assume that there are another 10 seconds of frames captured by camera 12B at 30 fps, resulting in 300 frames.

[0079]

[0083] In this example, if the display rate is 30 fps, the amount of time it takes to display the frames is as follows: First, there were 8 seconds at a capture rate of 30 fps, resulting in 240 frames. It takes 8 seconds at 30 fps to display the 240 frames. Then, there were 4 seconds at a capture rate of 120 fps, resulting in 480 frames. It takes 16 seconds at 30 fps to display the 480 frames.

[0080]

[0084] Then zooming in is enabled, where 0.5 seconds of frames are captured at 30 fps, resulting in 15 frames. It takes 0.5 seconds at 30 fps to display the 15 frames. As explained above, the quality of these 15 frames may be relatively poor due to imprecision in the camera parameters during zooming in. By reducing the capture rate to 30 fps, there is only 0.5 seconds worth of time for poor quality frames to be displayed, compared to 2 seconds if the capture rate was kept at 120 fps.

[0081]

[0085] Then there are another 4 seconds of frames captured at 120 fps, resulting in 480 frames. It takes 16 seconds to display the 480 frames at 30 fps. Slow motion is then turned off and 10 seconds of frames are captured at 30 fps, resulting in 300 frames. It takes 10 seconds to display the 300 frames at 30 fps.

[0082]

[0086] In the above example, each of cameras 12A-12C is configured to capture frames at 120 fps during slow motion or 30 fps during normal video. However, in some examples, the capture rates for two or more of cameras 12A-12C may differ. As an example, during slow motion, camera 12A may capture at 960 fps, camera 12B may capture at 480 fps, and camera 12C may capture at 120 fps. The capture rates for each of cameras 12A-12C may be user-configurable options.

[0083]

[0087] Camera processor 14 may perform some initial image processing on the frames, although such initial image processing may not be necessary in all instances. Camera processor 14 may output the frames to system memory 30 for storage. In some instances, rather than or in addition to outputting the frames to system memory 30, camera processor 14 may output the frames to local memory 20. As another example, camera processor 14 may store each of the frames in local memory 20 for temporary storage as each frame is captured, and then move the frames from local memory 20 to system memory 30.

[0084]

[0088] Figure 2 is a block diagram illustrating the camera processor of the device of Figure 1 in further detail. As shown, cameras 12A-12C may each capture frames that are output to camera processor 14. As explained above, each of cameras 12A-12C may be associated with a different field of view angle and a different range for zoom factors between respective zoom thresholds. In this manner, there are parallel streams that are output to camera processor 14.

[0085]

[0089] In one or more examples, prior to frame capture or possibly the first few captured frames are utilized to determine camera parameters. For example, camera processor 14 and / or CPU 16 may determine auto focus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters, referred to as 3A parameters, for each of cameras 12A-12C. As shown, camera parameters 34A may be camera parameters for camera 12A, camera parameters 34B may be camera parameters for camera 12B, and 3A parameters 3A may be camera parameters for camera 12C. For ease of explanation, camera parameters 34A-34C are stored locally in cameras 12A-12C, respectively. However, camera parameters 34A-34C may be stored elsewhere, such as in local memory 20 and / or system memory 30, as two examples.

[0086]

[0090] In some examples, once the camera parameters 34A-34C are determined, the camera parameters 34A-34C may remain stable for the remainder of the recording. In some examples, the camera parameters 34A-34C may not be recalculated because such recalculation may take time and result in delays or poor quality. The camera processor 14 and / or CPU 16 may determine the camera parameters 34A-34C individually (e.g., different parameters for each), or the camera processor 14 and / or CPU 16 may determine one of the camera parameters 34A-34C and apply the determined one to the others of the camera parameters 34A-34C (e.g., the same values ​​of defocus distance, luma index, and color temperature for the camera parameters 34A-34C).

[0087]

[0091] However, in some instances, it may be possible to recalculate the camera parameters 34A-34C for cameras 12A-12C whose frames have not been processed. For example, when frames from camera 12A are being processed, frames captured by cameras 12B and 12C may be utilized to update camera parameters 34B and 34C, respectively. Such calculations may be able to compensate for field of view (FOV) changes when switching between cameras 12A-12C.

[0088]

[0092] 2, camera processor 14 includes a switch 36 configured to selectively select a stream from one of cameras 12A-12C. In some cases, as described below, switch 36 may be configured to select two of the three streams. In general, however, switch 36 may select one of the streams of cameras 12A-12C. The unselected stream may be bypassed from all processing and go directly to the sink / flash.

[0089]

[0093] As shown, camera processor 14 includes a rate controller 38. Rate controller 38 may be configured to select the rate at which the circuitry of camera processor 14 receives frames. As an example, rate controller 38 may be configured to set a capture rate at which cameras 12A-12C capture frames. For example, rate controller 38 may cause cameras 12A-12C to capture frames at 30 fps, 120 fps, 240 fps, 960 fps, etc. In such an example, sensors of cameras 12A-12C may detect light and output a current indicative of the light and color based on the capture rate (e.g., 30 times per second for 30 fps). Rate controller 38 may set the rates of cameras 12A-12C to be the same or different.

[0090]

[0094] There may be other ways in which rate controller 38 may select the rate at which the circuitry of camera processor 14 receives frames. For example, cameras 12A-12C may each capture frames at a relatively high rate (e.g., 960 fps) and rate controller 38 may be configured to drop frames to achieve the desired rate. For example, the rate controller may drop every frame and forecast every eighth frame, resulting in an effective rate of 120 fps (e.g., 960 fps / 8=120 fps).

[0091]

[0095] In this disclosure, camera processor 14 receiving frames at a particular capture rate refers to instances where the capture rates of cameras 12A-12C are set to a particular rate and where frames captured by cameras 12A-12C are dropped to effectively achieve the particular capture rate. There may be other ways in which camera processor 14 may receive frames at a particular rate (e.g., capture rate), and the techniques described in this disclosure are not limited to set techniques.

[0092]

[0096] Camera processor 14 includes digital zoom circuitry 40A, processing circuitry 42A, and buffer 44A. Camera processor 14 may optionally include digital zoom circuitry 40B, processing circuitry 42B, and buffer 44B. Exemplary uses of digital zoom circuitry 40B, processing circuitry 42B, and buffer 44B are described in more detail below. For ease of understanding, exemplary techniques are described with respect to digital zoom circuitry 40A, processing circuitry 42A, and buffer 44A.

[0093]

[0097] Digital zoom circuitry 40A or 40B may be part of cameras 12A-12C instead of or in addition to being part of camera processor 14. Also, digital zoom circuitry 40A and / or 40B may not be necessary in all instances. For example, in some instances, optical zoom may be sufficient. In some instances, zooming in or out may be based on both optical zoom and digital zoom provided by digital zoom circuitry 40A and / or 40B. To implement optical zoom, camera processor 14 may include circuitry that sets the location of sensors and / or lenses of cameras 12A-12C to achieve a desired zoom factor.

[0094]

[0098] Digital zoom circuitry 40A may receive frames from a selected one of cameras 12A-12C and pass the frames to processing circuitry 42A if zoom is not enabled. Processing circuitry 42A may be configured to perform any processing, such as filtering or any other processing required to prepare the output frames to be output. Examples of processing include spatial filtering (e.g., filtering across image content in the same frame) or edge enhancement, as some examples. Buffer 44A may be configured to store the frames until they are output to system memory 30. Display interface 26 may then output the frames for display. Thus, camera processor 14, buffer 44A, and display interface 26 may be considered to generate video content for playback, including the frames that are then output for display.

[0095]

[0099] In one or more examples, as part of capturing video content, a user may enable slow motion mode (e.g., video capture with slow motion enabled). Processing circuit 42A may receive a first set of frames captured by first camera 12A at a first rate. The first rate may be set by rate controller 38 and may be relatively high (e.g., at least 120 fps or more). Processing circuit 42A may process the first set of frames (e.g., store the frames in buffer 44A) to generate video content for display. First camera 12A may capture the first set of frames based on camera parameters 34A.

[0096]

[0100] CPU 16 may receive a selection to zoom in or out. For example, a user may utilize display 28 to provide input to indicate that the user desires to zoom in or out on the image content being captured. CPU 16 may then output information indicative of the selection to zoom in or out to camera processor 14 and / or cameras 12A-12C.

[0097]

[0101] In response to a selection to zoom in or out, camera processor 14 may effectuate the implementation of the zooming in or out. For example, as camera processor 14 was receiving frames from first camera 12A, camera processor 14 may cause the position of the lens and / or sensor of camera 12A to move to change focus and provide zooming in or out. As another example, digital zoom circuitry 40A may implement digital zoom on frames captured by camera 12A.

[0098]

[0102] During zooming in or out to a zoom threshold for the first camera 12A, the processing circuit 42A may receive a second set of frames captured by the first camera 12A at a second rate less than the first rate. For example, the rate controller 38 may cause the first camera 12A to capture frames at the second rate (e.g., 30 fps). As another example, the rate controller 38 may drop frames from the first camera 12A to achieve a valid second rate.

[0099]

[0103] Because the second set of frames are captured while zooming in or out and the camera parameters 34A are determined without zooming in or out, the image quality of the second set of frames is inferior to the image quality of the first set of frames. By reducing the rate at which the second set of frames are captured (e.g., by setting the capture rate or dropping frames to achieve an effective capture rate), the number of frames having inferior quality is minimized.

[0100]

[0104] The camera processor 14 may process the second set of frames to generate video content for display. As an example, the digital zoom circuitry 40A may perform digital zoom in or out until a zoom threshold for the first camera 12A is reached (e.g., from a 0.5X to 1X zoom factor). For example, performing digital zoom in or out until a zoom threshold for the first camera 12A is reached includes the digital zoom circuitry 40A performing cropping and resizing during the increase or decrease in the zoom factor on the second set of captured frames.

[0101]

[0105] Buffer 44A may store the first set of frames and the second set of frames. System memory 30 may receive the first set of frames and the second set of frames. Display interface 26 may retrieve the first set of frames and the second set of frames and output the first set of frames and the second set of frames at a third rate less than the first rate (e.g., 30 fps) for display on display 28. For example, camera processor 14, buffer 44A, and display interface 26, either alone or in combination, may be considered to generate video content for playback that includes the first set of frames and the second set of frames at a third rate less than the first rate (e.g., 30 fps).

[0102]

[0106] In the above example, receiving a selection to zoom in or out is received with slow motion video capture enabled, and camera processor 14 may receive a second set of frames with slow motion video capture enabled.

[0103]

[0107] After the zoom threshold for the first camera 12A is reached, the camera processor 14 may receive frames from another one of the cameras 12. Switching from one camera to another is called a hard switch. For example, the switch 36 may switch from the first camera 12A to the second camera 12B. In this example, the camera processor 14 may receive a third set of frames captured by the second camera 12B of the multiple cameras 12 at a fourth rate that is greater than the second rate. As an example, the fourth rate may be equal to or greater than 120 fps and greater than the second rate, which may be 30 fps. As above, the rate controller 38 may set the capture rate of the second camera 12B or drop frames to achieve that capture rate. The camera processor 14 may process the third frames (e.g., via the processing circuitry 42A, since there may be no zoom) to generate video content for display. The display interface 26 may output the third set of frames at the third rate for display on the display 28. For example, camera processor 14 and display interface 26 may be viewed as generating video content for playback including a first set of frames, a second set of frames, and a third set of frames at a third rate, in this example, the fourth rate at which camera processor 14 receives and processes frames is greater than the display rate of the third rate because slow motion video capture is still enabled.

[0104]

[0108] The second camera 12B may capture frames based on the camera parameters 34B. For example, while or prior to the first camera 12A capturing the first set of frames, the CPU 16 and / or camera processor 14 may determine one or more of the AF parameters, AWB parameters, and AEC parameters (e.g., determine the camera parameters 34A-34C) for each of the multiple cameras 12A-12C. In such an example, receiving the third set of frames captured by the second camera 12B includes receiving the third set of frames based on one or more of the AF parameters, AWB parameters, and AEC parameters (e.g., camera parameters 34B) determined for the second camera 12B while or prior to the first camera 12A capturing the first set of frames. For example, as described above, the camera parameters 34A-34C may be stable or may not be redetermined during the capturing of frames. Thus, although the 3A parameters 34B were determined at or before the first set of frames were captured by camera 12A, the camera parameters 34B may still be accurate for camera 12B.

[0105]

[0109] As described above, in some examples, camera processor 14 may optionally include digital zoom circuitry 40B, processing circuitry 42B, and buffer 44B. One or more of digital zoom circuitry 40B, processing circuitry 42B, and buffer 44B (e.g., processing circuitry 42B may be present, but digital zoom circuitry 40B is not present). Digital zoom circuitry 40B, processing circuitry 42B, and buffer 44B may be utilized in examples where there is overlap in the zoom factor ranges for different cameras 12A-12C. For example, in some examples, the zoom thresholds for camera 12A may be 0.5X and 1X, and the zoom thresholds for camera 12B may be 1X-2X. However, in some examples, the zoom thresholds for camera 12A may be 0.5X-1.3X, and the zoom thresholds for camera 12B may be 1X-2X. In this example, there is overlap from 1X to 1.3X in the zoom factor ranges for cameras 12A and 12B.

[0106]

[0110] The use of digital zoom circuitry 40B, processing circuitry 42B, and buffer 44B when there is overlap in the zoom factor ranges is by way of example and should not be considered limiting. Digital zoom circuitry 40B, processing circuitry 42B, and buffer 44B may still be utilized even in instances where there is no overlap in the zoom factor ranges.

[0107]

[0111] In such an example, if processing circuit 42A is processing frames from camera 12A in response to a zoom in or out, processing circuit 42A may continue to process frames from camera 12A until a zoom threshold for camera 12A is reached. In parallel, processing circuit 42B may begin processing frames from camera 12B. For example, switch 36 may be configured to route output from camera 12A (e.g., via digital zoom circuit 40A) to processing circuit 42A and route output from camera 12B (e.g., via digital zoom circuit 40B) to processing circuit 42B. Processing circuit 42B may output to buffer 44B.

[0108]

[0112] Camera processor 14 may also include a switch 46. Switch 46 may be configured to selectively output from buffer 44A or buffer 44B. In some examples, as the zoom factor for camera 12A passes through 1X toward 1.3X, switch 46 may be configured to output from buffer 44A until the zoom factor reaches 1.3X. In parallel, after the zoom factor passes through 1X, processing circuit 42B may be configured to output frames to buffer 44B. In this case, from the zoom factor of 1X to 1.3X, frames in buffer 44B may be overwritten as buffer 44B fills. However, after the zoom factor reaches 1.3X, switch 46 may switch to select output from buffer 44B.

[0109]

[0113] Having both buffer 44A and buffer 44B can be beneficial to avoid gaps in frames to be displayed when there is a switch from camera 12A to camera 12B. For example, there may already be frames in buffer 44B ready to be displayed. Otherwise, there may be a momentary blank frame during the switch over from camera 12A to camera 12B. By pre-filling buffer 44B with frames, there may be no blank frame.

[0110]

[0114] As described above, with slow-motion video capture enabled, camera processor 14 may be configured to receive a first set of frames captured by first camera 12 at a first rate and receive a second set of frames captured by first camera 12A at a second rate less than the first rate during zooming in or out to a zoom threshold for the first camera. In some examples, in parallel with processing the second set of frames, camera processor 14 may receive a third set of frames captured by a second camera 12B of multiple cameras 12A-12C. For example, during a time when camera 12A is zoomed in from 0.5X to 1.3X, camera processor 14 may receive in parallel a third set of frames from camera 12B, such as when the zoom factor is between 1X and 1.3X. Camera processor 14 may process the third set of frames.

[0111]

[0115] The switch 46 may be configured to select the buffer 44A to output a second set of frames for display until a second zoom threshold (e.g., 1.3X) for the first camera 12A is reached. The switch 46 may then be configured to select the buffer 44B to output a third set of frames for display after the second zoom threshold (e.g., 1.3X) for the first camera 12A is reached. In this manner, generating video content for playback may include generating video content for playback at a third rate including the first set of frames, the second set of frames until the second zoom threshold for the first camera is reached, and the third set of frames after the second zoom threshold for the first camera is reached. In this example, both the first camera 12A and the second camera 12B are configured to operate at a zoom factor greater than or equal to the first zoom threshold and less than or equal to the second zoom threshold. For example, both camera 12A and camera 12B are configured to operate at a zoom factor greater than or equal to 1X and less than or equal to 1.3X.

[0112]

[0116] 3 is a conceptual diagram illustrating an example method for enabling zooming in or out. In the example shown in FIG. 3, display 28 may display three icons 48A-48C, each associated with a different zoom threshold for a different one of cameras 12A-12C. For example, icon 48A may be associated with 0.5X for camera 12A, icon 48B may be associated with 1X for camera 12B, and icon 48C may be associated with 2X for camera 12C. Thus, in some examples, camera processor 14 and / or CPU 16 may receive information indicating a selection to zoom as a way of receiving a selection to zoom in or out (e.g., when video capture in slow motion is enabled, etc.).

[0113]

[0117] However, there may be other ways to receive a selection to zoom in or out, such as by receiving information indicating a pinch to zoom motion, in which a user may place two fingers on display 28 and pinch the two fingers together to zoom out or spread the two fingers apart to zoom in.

[0114]

[0118] In either case (e.g., select or pinch zoom), camera processor 14 may cause the zoom factor of one of cameras 12A-12C from which camera processor 14 is receiving frames to increase or decrease until a zoom threshold for one of cameras 12A-12C is reached. During the zooming in or out to the zoom threshold (e.g., the approximately 0.5 seconds it takes for the zoom factor to reach the zoom threshold), camera processor 14 may receive and process frames at a lower rate than the rate at which camera processor 14 may receive and process frames when the user is not zooming in or out.

[0115]

[0119] Figure 4 is a timing diagram illustrating an example of zooming in or out. In Figure 4, before time T0, camera processor 14 may be configured to receive frames from camera 12A at 30 fps, since zoom is not enabled.

[0116]

[0120] At time T0, a user may enable slow motion (e.g., via input through display 28). Starting at time T0, camera processor 14 may receive a first set of frames captured by first camera 12A at a first rate. The first rate may be relatively high, such as 120 fps or higher. Processing circuitry 42A may be configured to process the first set of frames to generate video content for display. Processing circuitry 42A may store the first set of frames in buffer 44A.

[0117]

[0121] At T1, CPU 16 and / or camera processor 14 may receive a selection to zoom in or out. For example, receiving a selection to zoom in or out may include one of receiving information indicating a pinch-zoom motion (e.g., via selection of one of icons 48A-48C) or receiving information indicating a selection to zoom.

[0118]

[0122] From time T1 to T2, the zoom factor for camera 12A may change. For example, it may take about 0.5 seconds to zoom from 0.5X to 1X. During this 0.5 seconds, camera processor 14 may receive frames captured by camera 12A. For example, during zooming in or out to a zoom threshold for first camera 12A, camera processor 14 may receive a second set of frames captured by first camera 12A at a second rate that is less than the first rate. Because the second set of frames is captured with camera parameters 34A selected for a zoom factor of 0.5X, the image quality of the second set of frames may be inferior to the image quality of the first set of frames. As an example, the second rate may be 30 fps.

[0119]

[0123] Processing circuitry 42A may process the second set of frames to generate video content for display and store the second set of frames (e.g., after processing) in buffer 44A. Camera processor 14 may cause buffer 44A to output the first set of frames and the second set of frames to system memory 30. Display interface 26 may receive the first set of frames and the second set of frames from system memory 30 and output the first set of frames and the second set of frames at a third rate less than the first rate (e.g., 30 fps) for display on display 28.

[0120]

[0124] At time T2, a zoom threshold for camera 12A is reached and camera processor 14 processes frames from second camera 12B. For example, switch 36 switches to route output from second camera 12B to processing circuit 42A (e.g., via digital zoom circuit 40A, if necessary). For example, after a zoom threshold for first camera 12A is reached, camera processor 14 may be configured to receive a third set of frames captured by second camera 12B of multiple cameras 12A-12C at a fourth rate (e.g., 120 fps or greater) that is greater than the second rate (e.g., the second rate is 30 fps). Processing circuit 42A may process the third set of frames to generate video content for display and store the third set of frames in buffer 44A. Buffer 44A may output the third set of frames to system memory 30. Display interface 26 may retrieve the third set of frames and output the third set of frames at a third rate (eg, 30 fps) for display.

[0121]

[0125] At time T3, the user may disable video capture with slow motion. In this case, camera processor 14 may continue to receive frames from second camera 12B. However, the rate at which camera processor 14 receives frames from second camera 12B may be relatively slow (e.g., 30 fps) because slow motion has been disabled. Again, in slow motion, frames are captured at a high rate and displayed at a low rate. When slow motion is disabled, the rate at which frames are captured may be approximately the same as the rate at which frames are displayed.

[0122]

[0126] 5 is a flow chart illustrating an example method of operation according to one or more examples described in this disclosure. The example of FIG. 5 is described with respect to one or more processors and a first camera, where the example one or more processors include one or more of camera processor 14, CPU 16, and display interface 26, and the example first camera includes any one of cameras 12A-12C.

[0123]

[0127] The one or more processors may receive a first set of frames captured by a first camera at a first rate (50). For example, slow-motion video capture may be enabled. In such a case, the first rate may be relatively high, such as 120 fps or more.

[0124]

[0128] One or more processors may process the first set of frames to generate video for display (52). For example, processing circuit 42A may be configured to process the first set of frames to generate video content and store the resulting first set of frames in buffer 44A.

[0125]

[0129] At some point, the one or more processors may receive a selection to zoom in or out (54). For example, receiving a selection to zoom in or out may include receiving a selection to zoom in or out with slow-motion video capture enabled.

[0126]

[0130] During the zoom in or out to the zoom threshold for the first camera, the one or more processors may receive a second set of frames captured by the first camera at a second rate that is less than the first rate (56). In some examples, receiving the second set of frames includes receiving the second set of frames with video capture in slow motion enabled. As described above, it may take an amount of time to increase or decrease the zoom factor for the first camera to reach the zoom threshold for the first camera. During the time that the zoom factor is increasing or decreasing to the zoom threshold, the frames captured (e.g., the second set of frames) tend to have poorer image quality because the camera parameters are not accurate for those zoom factors. In some examples, the image quality of the second set of frames is poorer than the image quality of the first set of frames.

[0127]

[0131] The one or more processors may process the second set of frames to generate video content for display (58). Processing the second set of frames may include performing a digital zoom in or out until a zoom threshold for the first camera is reached. For example, performing a digital zoom in or out until a zoom threshold for the first camera is reached may include performing cropping and resizing during an increase or decrease in zoom factor on the second set of frames.

[0128]

[0132] Buffer 44A may store the first set of frames and the second set of frames. In some examples, the one or more processors may generate video content for playback including the first set of frames and the second set of frames at a third rate that is less than the first rate (60). The one or more processors (e.g., display interface 26) may output the generated video content including the first set of frames and the second set of frames at the third rate that is less than the first rate for display on display 28.

[0129]

[0133] The first camera is one of a plurality of cameras (e.g., one of cameras 12A-12C) coupled to the one or more processors. After a zoom threshold for the first camera is reached, the one or more processors may be configured to receive a third set of frames captured by a second camera of the plurality of cameras at a fourth rate greater than the second rate, process the third set of frames to generate video content for display, and output the third set of frames at the third rate for display. For example, the one or more processors may generate video content for playback including the first set of frames, the second set of frames, and the third set of frames at the third rate, and then output the generated video content for playback for display.

[0130]

[0134] As described above, the multiple cameras may capture frames based on their respective camera parameters. For example, while or prior to the first camera capturing the first set of frames, the one or more processors may be configured to determine one or more of the AF parameters, AWB parameters, and AEC parameters (e.g., camera parameters 34A-34C) for each of the multiple cameras. In such an example, receiving the third set of frames captured by the second camera may include receiving the third set of frames based on one or more of the AF parameters, AWB parameters, and AEC parameters determined for the second camera while or prior to the first camera capturing the first set of frames.

[0131]

[0135] In some examples, the zoom threshold is a first zoom threshold. After the first zoom threshold for the first camera is reached, the one or more processors may be configured to receive a third set of frames captured by a second camera of the multiple cameras (e.g., switch 36 routes frames from the first camera to processing circuit 42A and frames from the second camera to processing circuit 42B) in parallel with processing the second set of frames, and process the third set of frames (e.g., with processing circuit 42B).

[0132]

[0136] The one or more processors may be configured to generate video content for playback at a third rate, the video content including the first set of frames, the second set of frames until a second zoom threshold for the first camera is reached, and the third set of frames after the second zoom threshold for the first camera is reached. For example, the one or more processors may output the second set of frames until a second zoom threshold for the first camera is reached for display, and output the third set of frames after the second zoom threshold for the first camera is reached for display. In this example, both the first camera and the second camera are configured to operate at a zoom factor greater than or equal to the first zoom threshold and less than or equal to the second zoom threshold.

[0133]

[0137] The following describes several exemplary techniques, which may be used separately or together in any practical combination.

[0134]

[0138] Clause 1. A device for generating video content, the device comprising: a first camera; and one or more processors coupled to the first camera, the one or more processors configured to: receive a first set of frames captured by the first camera at a first rate; process the first set of frames to generate video content for display; receive a selection to zoom in or out; receive a second set of frames captured by the first camera at a second rate less than the first rate during the zoom in or out to a zoom threshold for the first camera; process the second set of frames to generate video content for display; and generate video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

[0135]

[0139] Clause 2. The device of clause 1, wherein receiving a selection to zoom in or zoom out comprises receiving a selection to zoom in or zoom out with video capture in slow motion enabled, and receiving a second set of frames comprises receiving a second set of frames with video capture in slow motion enabled.

[0136]

[0140] Clause 3. The device of any of clauses 1 and 2, wherein the first camera is one of a plurality of cameras coupled to one or more processors, and wherein the device includes a second camera of the plurality of cameras, and wherein after a zoom threshold for the first camera is reached, the one or more processors are configured to receive a third set of frames captured by the second camera of the plurality of cameras at a fourth rate greater than the second rate and process the third set of frames to generate video content for display, and wherein generating video content for playback comprises generating video content for playback including the first set of frames, the second set of frames, and the third set of frames at the third rate.

[0137]

[0141] Clause 4. The device of clause 3, wherein one or more processors are configured to determine one or more of autofocus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters for each of the plurality of cameras while or before the first camera captures the first set of frames, and receiving a third set of frames captured by the second camera comprises receiving the third set of frames based on one or more of the AF parameters and AEC parameters determined for the second camera while or before the first camera captures the first set of frames.

[0138]

[0142] Clause 5. The device of any of clauses 1 to 4, wherein processing the second set of frames comprises performing digital zoom in or digital zoom out until a zoom threshold for the first camera is reached, and wherein performing digital zoom in or digital zoom out until a zoom threshold for the first camera is reached comprises performing cropping and resizing on the second set of frames while increasing or decreasing the zoom factor.

[0139]

[0143] Clause 6. The device of any of clauses 1 to 5, wherein the zoom threshold comprises a first zoom threshold, and wherein the first camera is one of a plurality of cameras coupled to the one or more processors, and wherein after the first zoom threshold for the first camera is reached, the one or more processors are configured to receive a third set of frames captured by a second camera of the plurality of cameras and process the third set of frames in parallel with processing the second set of frames, and wherein generating video content for playback comprises generating video content for playback at a third rate, the video content including the first set of frames, the second set of frames until the second zoom threshold for the first camera is reached, and the third set of frames after the second zoom threshold for the first camera is reached, and wherein both the first camera and the second camera are configured to operate at a zoom factor greater than or equal to the first zoom threshold and less than or equal to the second zoom threshold.

[0140]

[0144] Clause 7. A device according to any of clauses 1 to 6, wherein the first rate is greater than or equal to 120 frames per second.

[0141]

[0145] Clause 8. A device according to any of clauses 1 to 7, wherein the third rate is equal to the second rate.

[0142]

[0146] Clause 9. A device as described in any of clauses 1 to 8, wherein receiving a selection to zoom in or zoom out comprises one of receiving information indicating a pinch-zoom motion or receiving information indicating a selection to zoom.

[0143]

[0147] Clause 10. A device according to any of clauses 1 to 9, wherein the image quality of the second set of frames is inferior to the image quality of the first set of frames.

[0144]

[0148] Clause 11. A method for generating video content, the method comprising: receiving, with one or more processors, a first set of frames captured by a first camera at a first rate; processing, with the one or more processors, the first set of frames to generate video content for display; receiving, with the one or more processors, a selection to zoom in or out; receiving, with the one or more processors, during the zoom in or zoom out to a zoom threshold for the first camera, a second set of frames captured by the first camera at a second rate less than the first rate, with the one or more processors; processing, with the one or more processors, the second set of frames to generate video content for display; and generating, with the one or more processors, video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

[0145]

[0149] Clause 12. The method of clause 11, wherein receiving a selection to zoom in or zoom out comprises receiving a selection to zoom in or zoom out with video capture in slow motion enabled, and receiving a second set of frames comprises receiving a second set of frames with video capture in slow motion enabled.

[0146]

[0150] Clause 13. The method of any of clauses 11 and 12, wherein the first camera is one of a plurality of cameras coupled to one or more processors, and the method further comprises receiving a third set of frames captured by a second camera of the plurality of cameras at a fourth rate greater than the second rate after a zoom threshold for the first camera is reached, and processing the third set of frames to generate video content for display, wherein generating video content for playback comprises generating video content for playback including the first set of frames, the second set of frames, and the third set of frames at the third rate.

[0147]

[0151] Clause 14. The method of clause 13, further comprising determining one or more of autofocus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters for each of the plurality of cameras during or prior to the first camera capturing the first set of frames, wherein receiving a third set of frames captured by the second camera comprises receiving the third set of frames based on the one or more of the AF parameters and AEC parameters determined for the second camera during or prior to the first camera capturing the first set of frames.

[0148]

[0152] Clause 15. The method of any of clauses 11 to 14, wherein processing the second set of frames comprises performing digital zoom in or digital zoom out until a zoom threshold for the first camera is reached, and wherein performing digital zoom in or digital zoom out until a zoom threshold for the first camera is reached comprises performing cropping and resizing during an increase or decrease in the zoom factor on the second set of frames.

[0149]

[0153] Clause 16. The method of any of clauses 11 to 15, wherein the zoom threshold comprises a first zoom threshold, wherein the first camera is one of a plurality of cameras coupled to the one or more processors, and the method further comprises receiving a third set of frames captured by a second camera of the plurality of cameras in parallel with processing the second set of frames after the first zoom threshold for the first camera is reached and processing the third set of frames, and wherein generating video content for playback comprises generating video content for playback at a third rate, the video content including the first set of frames, the second set of frames until the second zoom threshold for the first camera is reached, and the third set of frames after the second zoom threshold for the first camera is reached, and wherein both the first camera and the second camera are configured to operate at a zoom factor greater than or equal to the first zoom threshold and less than or equal to the second zoom threshold.

[0150]

[0154] Clause 17. The method of any of clauses 11 to 16, wherein the first rate is greater than or equal to 120 frames per second.

[0151]

[0155] Clause 18. The method of any of clauses 11 to 17, wherein the third rate is equal to the second rate.

[0152]

[0156] Clause 19. The method of any of clauses 11 to 18, wherein receiving a selection to zoom in or out comprises one of receiving information indicating a pinch-zoom motion or receiving information indicating a selection to zoom.

[0153]

[0157] Clause 20. The method of any of clauses 11 to 19, wherein the image quality of the second set of frames is inferior to the image quality of the first set of frames.

[0154]

[0158] Clause 21. A computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to receive a first set of frames captured by a first camera at a first rate, process the first set of frames to generate video content for display, receive a selection to zoom in or out, receive a second set of frames captured by the first camera at a second rate less than the first rate during the zoom in or out to a zoom threshold for the first camera, process the second set of frames to generate video content for display, and generate video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

[0155]

[0159] Clause 22. A computer-readable storage medium as described in Clause 21, wherein the instructions for causing one or more processors to receive a selection to zoom in or zoom out comprise instructions for causing one or more processors to receive a selection to zoom in or zoom out with video capture in slow motion enabled, and the instructions for causing one or more processors to receive a second set of frames comprise instructions for causing one or more processors to receive the second set of frames with video capture in slow motion enabled.

[0156]

[0160] Clause 23. A computer-readable storage medium as described in any of clauses 21 and 22, wherein the first camera is one of a plurality of cameras, and the instructions further comprise instructions to cause the one or more processors to receive a third set of frames captured by a second camera of the plurality of cameras at a fourth rate greater than the second rate after a zoom threshold for the first camera is reached, and process the third set of frames to generate video content for display, wherein generating video content for playback comprises generating video content for playback including the first set of frames, the second set of frames, and the third set of frames at the third rate.

[0157]

[0161] Clause 24. The computer-readable storage medium of clause 23, further comprising instructions to cause the one or more processors to determine one or more of autofocus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters for each of the plurality of cameras during or prior to the first camera capturing the first set of frames, wherein receiving a third set of frames captured by the second camera comprises receiving the third set of frames based on one or more of the AF parameters and AEC parameters determined for the second camera during or prior to the first camera capturing the first set of frames.

[0158]

[0162] Clause 25. The computer-readable storage medium of any of clauses 21-24, wherein the zoom threshold comprises a first zoom threshold, wherein the first camera is one of a plurality of cameras, and the instructions further comprise instructions for causing the one or more processors to receive a third set of frames captured by a second camera of the plurality of cameras and process the third set of frames in parallel with processing the second set of frames after the first zoom threshold for the first camera is reached, and wherein generating video content for playback comprises generating video content for playback at a third rate, the video content including the first set of frames, the second set of frames until the second zoom threshold for the first camera is reached, and the third set of frames after the second zoom threshold for the first camera is reached, and wherein both the first camera and the second camera are configured to operate at a zoom factor greater than or equal to the first zoom threshold and less than or equal to the second zoom threshold.

[0159]

[0163] Clause 26. A device for generating video content, the device comprising: means for receiving a first set of frames captured by a first camera at a first rate; means for processing the first set of frames to generate video content for display; means for receiving a selection of to zoom in or zoom out; means for receiving a second set of frames captured by the first camera at a second rate less than the first rate during zooming in or out to a zoom threshold for the first camera; means for processing the second set of frames to generate video content for display; and means for generating video content for playback including the first set of frames and the second set of frames at a third rate less than the first rate.

[0160]

[0164] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium. In this manner, a computer-readable medium may generally correspond to a tangible computer-readable storage medium that is non-transitory. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0161]

[0165] By way of example and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be understood that computer-readable storage media and data storage media do not include carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. Disk and disc as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0162]

[0166] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated circuits or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the above structures, or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a composite codec. Also, the techniques may be fully implemented in one or more circuits or logic elements.

[0163]

[0167] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs) or sets of ICs (e.g., chipsets). In this disclosure, various components, modules, or units have been described to highlight functional aspects of devices configured to implement the disclosed techniques, but those components, modules, or units do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors described above, along with suitable software and / or firmware.

[0164]

[0168] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. 1. A device for generating video content, comprising: A first camera; one or more processors coupled to the first camera; wherein the one or more processors: receiving a first set of frames captured by the first camera at a first rate; processing the first set of frames to generate video content for display; receiving a selection to zoom in or zoom out; receiving a second set of frames captured by the first camera at a second rate less than the first rate during the zoom-in or zoom-out to a zoom threshold for the first camera; processing the second set of frames to generate video content for display; generating video content for playback comprising the first set of frames and the second set of frames at a third rate less than the first rate; A device configured to:

2. Receiving the selection to zoom in or out comprises receiving the selection to zoom in or zoom out with video capture in slow motion enabled; receiving the second set of frames comprises receiving the second set of frames with slow motion video capture enabled. The device of claim 1 .

3. the first camera is one of a plurality of cameras coupled to the one or more processors, the device comprising a second camera of the plurality of cameras; After the zoom threshold for the first camera is reached, the one or more processors: receiving a third set of frames captured by the second camera of the plurality of cameras at a fourth rate greater than the second rate; processing the third set of frames to generate video content for display; [0023] wherein generating video content for playback comprises generating video content for playback including the first set of frames, the second set of frames, and the third set of frames at the third rate; Optionally, during or prior to the first camera capturing the first set of frames, the one or more processors are configured to determine, for each of the plurality of cameras, one or more of autofocus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters; 2. The device of claim 1, wherein receiving the third set of frames captured by the second camera comprises receiving the third set of frames based on one or more of AF and AEC parameters determined for the second camera during or prior to the first camera capturing the first set of frames.

4. processing the second set of frames comprises performing a digital zoom in or out until the zoom threshold for the first camera is reached; Performing a digital zoom in or out until the zoom threshold for the first camera is reached includes: performing cropping and resizing on said second set of frames while increasing or decreasing the zoom factor; The device of claim 1 , comprising:

5. the zoom threshold comprises a first zoom threshold, the first camera being one of a plurality of cameras coupled to the one or more processors; After the first zoom threshold for the first camera is reached, the one or more processors: receiving a third set of frames captured by a second camera of the plurality of cameras in parallel with processing the second set of frames; processing the third set of frames; and [0023] generating video content for playback comprises generating, at the third rate, video content for playback including the first set of frames, the second set of frames until a second zoom threshold for the first camera is reached, and the third set of frames after the second zoom threshold for the first camera is reached; both the first camera and the second camera are configured to operate at a zoom factor greater than or equal to the first zoom threshold and less than or equal to the second zoom threshold; The device of claim 1 .

6. The device of claim 1 , wherein the first rate is greater than or equal to 120 frames per second and / or the third rate is equal to the second rate.

7. Receiving the choice to zoom in or out receiving information indicative of a pinch-zoom motion; or receiving information indicative of a selection to zoom; The device of claim 1 , comprising one of:

8. The device of claim 1 , wherein an image quality of the second set of frames is inferior to an image quality of the first set of frames.

9. 1. A method for generating video content, the method comprising: receiving, with one or more processors, a first set of frames captured by a first camera at a first rate; processing, with the one or more processors, the first set of frames to generate video content for display; receiving, with the one or more processors, a selection to zoom in or out; receiving, with the one or more processors, a second set of frames captured by the first camera at a second rate less than the first rate during the zoom-in or zoom-out to a zoom threshold for the first camera; processing, with the one or more processors, the second set of frames to generate video content for display; generating, with the one or more processors, video content for playback at a third rate less than the first rate, the video content including the first set of frames and the second set of frames; A method comprising:

10. Receiving the selection to zoom in or out comprises receiving the selection to zoom in or zoom out with video capture in slow motion enabled; receiving the second set of frames comprises receiving the second set of frames with slow motion video capture enabled.

10. The method of claim 9.

11. the first camera is one of a plurality of cameras coupled to the one or more processors; The method includes, after the zoom threshold for the first camera is reached, receiving a third set of frames captured by a second camera of the plurality of cameras at a fourth rate greater than the second rate; processing the third set of frames to generate video content for display; Further equipped with wherein generating video content for playback comprises generating video content for playback including the first set of frames, the second set of frames, and the third set of frames at the third rate; Optionally, determining one or more of autofocus (AF) parameters, auto white balance (AWB) parameters, and auto exposure control (AEC) parameters for each of the plurality of cameras during or prior to the first camera capturing the first set of frames; Further equipped with 10. The method of claim 9, wherein receiving the third set of frames captured by the second camera comprises receiving the third set of frames based on one or more of AF and AEC parameters determined for the second camera during or prior to the first camera capturing the first set of frames.

12. processing the second set of frames comprises performing a digital zoom in or out until the zoom threshold for the first camera is reached; Performing a digital zoom in or out until the zoom threshold for the first camera is reached includes: performing cropping and resizing on said second set of frames while increasing or decreasing the zoom factor; The method of claim 9 comprising:

13. the zoom threshold comprises a first zoom threshold, the first camera being one of a plurality of cameras coupled to the one or more processors; The method includes, after the first zoom threshold for the first camera is reached, receiving a third set of frames captured by a second camera of the plurality of cameras in parallel with processing the second set of frames; processing the third set of frames; and Further equipped with generating video content for playback comprises generating, at the third rate, video content for playback including the first set of frames, the second set of frames until a second zoom threshold for the first camera is reached, and the third set of frames after the second zoom threshold for the first camera is reached; both the first camera and the second camera are configured to operate at a zoom factor greater than or equal to the first zoom threshold and less than or equal to the second zoom threshold; 10. The method of claim 9.

14. 10. The method of claim 9, wherein the first rate is greater than or equal to 120 frames per second and / or the third rate is equal to the second rate.

15. A computer readable storage medium having instructions stored thereon that, when executed, cause one or more processors to perform the method of any one of claims 9 to 14.

Images