Panoramic centering user interface for a camera

Abstract

Aspects of the present disclosure relate to using an image capture device including a camera to capture panoramic images. An example method includes: receiving a user command to start generating a panoramic image; using the camera to capture a plurality of images for generating the panoramic image; receiving an indication specifying a center region of interest (ROI) while capturing the plurality of images; completing the capture of the plurality of images; and automatically generating a panoramic image based on the plurality of images, the panoramic image being centered on the specified center ROI.

Description

Technical Field

[0001] In general, the present disclosure relates to systems and methods for image capture, and more particularly to the generation of panoramic images. Background Art

[0002] Many cameras are capable of generating panoramic images. For example, many cameras in handheld devices such as mobile phones and tablet devices are capable of generating panoramic images. Generating such panoramic images typically requires the user to rotate the camera while capturing multiple images of a scene. When the image capture is complete, the captured images can be combined into a single panoramic image having a field of view (FoV) greater than that of the individual captured images. Such panoramic images can have different fields of view based on the images captured for combination. For example, some panoramic images may have a 360-degree FoV. A panoramic image with a 360-degree FoV includes images captured during a full rotation of the camera by a user (such as a user rotating in place). A panoramic image with a 360-degree FoV can be referred to as a 360-degree panoramic image. Other panoramic images may have a smaller field of view (such as 180 degrees or other angles). Summary of the Invention

[0003] This Summary of the Invention is provided to introduce some concepts that are further described below in the Detailed Description in a simplified form. This Summary of the Invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. The systems, methods, and devices of the present disclosure each have several innovative aspects, none of which alone is responsible for the desired attributes disclosed herein.

[0004] One innovative aspect of the subject matter described in the present disclosure can be implemented as a method for generating a panoramic image using an image capture device. The method can be performed by a suitable image capture device including a camera, and can include: receiving a user command to start generating the panoramic image; using the camera to capture multiple images for generating the panoramic image; receiving an indication specifying a central region of interest (ROI) while capturing the multiple images; completing the capture of the multiple images; and automatically generating the panoramic image based on the multiple images, wherein the panoramic image is centered on the specified central ROI.

[0005] In some aspects, the received indication can be a user input specifying the central ROI, such as a touch gesture on a touch-sensitive user interface of the image capture device, such as a touch gesture at the central ROI on the touch-sensitive user interface, or an indication of a selection of a menu option on the touch-sensitive user interface when the camera is pointed at the central ROI. In some aspects, receiving the indication can include automatically determining the central ROI based on one or more characteristics of the plurality of images, such as an estimated depth of one or more objects in the plurality of images. In some aspects, before receiving the user command, the method can include receiving a user input that configures the device to operate in a mode configured to generate a recentered panoramic image.

[0006] In some aspects, the panoramic image can be a 360-degree panoramic image, and automatically generating the panoramic image can include determining split points corresponding to the specified central ROI; determining a start capture point and an end capture point based on the split points; and generating the panoramic image based on the start capture point and the end capture point. In some aspects, the panoramic image can be a panoramic image less than 360 degrees, and automatically generating the panoramic image can include comparing the specified central ROI with the maximum field of view of the panoramic image; and cropping the panoramic image to be centered on the specified central ROI.

[0007] Another innovative aspect of the subject matter described in this disclosure can be implemented in an image capture device. The image capture device can include one or more cameras; one or more processors; and a memory storing instructions for execution by the one or more processors. Execution of the instructions can cause the image capture device to perform operations including receiving a user command to start generating a panoramic image; using the one or more cameras to capture a plurality of images for generating the panoramic image; receiving an indication specifying a central region of interest (ROI) while capturing the plurality of images; completing the capture of the plurality of images; and automatically generating the panoramic image based on the plurality of images, wherein the panoramic image is centered on the specified central ROI.

[0008] In some aspects, the received indication can be a user input specifying the central ROI, such as a touch gesture on a touch-sensitive user interface of the image capture device, such as a touch gesture at the central ROI on the touch-sensitive user interface, or an indication of a selection of a menu option on the touch-sensitive user interface when the camera is pointed at the central ROI. In some aspects, receiving the indication can include automatically determining the central ROI based on one or more characteristics of the plurality of images, such as an estimated depth of one or more objects in the plurality of images. In some aspects, before receiving the user command, the method can include receiving a user input that configures the device to operate in a mode configured to generate a recentered panoramic image.

[0009] In some aspects, the panoramic image can be a 360-degree panoramic image, and automatically generating the panoramic image can include determining split points corresponding to the specified central ROI; determining a start capture point and an end capture point based on the split points; and generating the panoramic image based on the start capture point and the end capture point. In some aspects, the panoramic image can be a panoramic image of less than 360 degrees, and automatically generating the panoramic image can include comparing the specified central ROI with the maximum field of view of the panoramic image; and cropping the panoramic image to be centered on the specified central ROI.

[0010] Another innovative aspect of the subject matter described in this disclosure can be implemented in a non-transitory computer-readable storage medium storing instructions for execution by one or more processors of an image capture device including a camera. Execution of the instructions causes the image capture device to perform operations including receiving a user command to start generating a panoramic image; using the camera to capture a plurality of images for generating the panoramic image; receiving an indication specifying a central region of interest (ROI) while capturing the plurality of images; completing the capture of the plurality of images; and automatically generating the panoramic image based on the plurality of images, wherein the panoramic image is centered on the specified central ROI.

[0011] In some aspects, the received indication can be a user input specifying the central ROI, such as a touch gesture on a touch-sensitive user interface of the image capture device, such as a touch gesture at the central ROI on the touch-sensitive user interface, or an indication of a selection of a menu option on the touch-sensitive user interface when the camera is pointed at the central ROI. In some aspects, receiving the indication can include automatically determining the central ROI based on one or more characteristics of the plurality of images, such as an estimated depth of one or more objects in the plurality of images. In some aspects, before receiving the user command, the method can include receiving a user input that configures the device to operate in a mode configured to generate a recentered panoramic image.

[0012] In some aspects, the panoramic image can be a 360-degree panoramic image, and automatically generating the panoramic image can include determining split points corresponding to the specified central ROI; determining a start capture point and an end capture point based on the split points; and generating the panoramic image based on the start capture point and the end capture point. In some aspects, the panoramic image can be a panoramic image less than 360 degrees, and automatically generating the panoramic image can include comparing the specified central ROI with the maximum field of view of the panoramic image; and cropping the panoramic image to be centered on the specified central ROI.

[0013] Another innovative aspect of the subject matter described in this disclosure can be implemented in an image capture device. The image capture device can include one or more cameras; a unit for receiving a user command to start generating a panoramic image; a unit for using the one or more cameras to capture a plurality of images for generating the panoramic image; a unit for receiving an indication specifying a central region of interest (ROI) when capturing the plurality of images; a unit for completing the capture of the plurality of images; and a unit for automatically generating the panoramic image based on the plurality of images, wherein the panoramic image is centered on the specified central ROI. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Aspects of the present disclosure are illustrated by way of example and not limitation in the figures of the accompanying drawings, and in the drawings, like reference numerals refer to like elements.

[0015] Figure 1 is a block diagram of an example device configured to generate a panoramic image according to some implementations.

[0016] Figure 2 shows a trajectory diagram for generating a conventional 360-degree panoramic image.

[0017] Figure 3 Shows a trajectory diagram for generating a 360-degree panoramic image according to some implementations.

[0018] Figure 4 Shows a trajectory diagram for generating a panoramic image of less than 360 degrees according to some implementations.

[0019] Figure 5 Shows an example user interface window for an image capture device according to some implementations.

[0020] Figure 6 Is an illustrative flowchart depicting an example operation for generating a panoramic image according to some implementations.

[0021] Figure 7 Shows a block diagram of an example device that supports generating a panoramic image according to some implementations. Detailed Description

[0022] Aspects of the present disclosure can be used to automatically generate a panoramic image centered on a region of interest (ROI) specified during panoramic image generation. As described above, many image capture devices (such as cellular phones and tablet devices) include cameras and are capable of generating panoramic images. Such panoramic images can be generated based on, for example, images captured when a user rotates the camera of the image capture device. The captured images can be processed and combined into a single panoramic image. However, although a user may expect the panoramic image to be centered on a particular point (such as when the camera is facing a particular person or landmark), in practice, it may be difficult to generate a panoramic image centered on a particular point. For example, generating a 180-degree panoramic image centered on a particular point requires the user to start capturing images when the camera is oriented 90 degrees from the desired center point, which may be difficult for the user to determine. Similarly, for a 360-degree panoramic image, the user may need to start capturing images when the camera is oriented 180 degrees from the desired center point. For panoramic images other than 360-degree or 180-degree panoramic images, determining the appropriate starting point may be even more difficult.

[0023] Methods and apparatuses of example implementations include receiving an indication specifying a center region of interest (ROI) for a panoramic image. A panoramic image can then be generated centered on the specified center ROI. Such an indication can include user input, such as a touch gesture on a touch-sensitive user interface of an image capture device, a selection of a menu option on the touch-sensitive user interface, or another suitable user input. Automatically generating a panoramic image can include: determining split points for the captured panoramic image; determining a start capture point and an end capture point of the panoramic image based on the split points; and generating the panoramic image based on the start capture point and the end capture point. The captured panoramic image can be a 360-degree panoramic image, or can have a field of view less than 360 degrees.

[0024] In the following description, numerous specific details are set forth, such as examples of specific components, circuits, and processes, to provide a thorough understanding of the present disclosure. As used herein, the term "coupled" means directly connected or connected through one or more intervening components or circuits. Additionally, in the following description and for purposes of explanation, specific terms are set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that practicing the teachings disclosed herein may not require these specific details. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring the teachings of the present disclosure. Some portions of the following detailed description are presented in terms of processes, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. In the present disclosure, a process, a logic block, a process, etc., are conceived to be a consistent sequence of steps or instructions leading to a desired result. Steps are those steps that require physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system.

[0025] However, it should be remembered that all such and similar terms will be associated with appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise expressly stated (as will be apparent from the discussion below), it is to be appreciated that throughout this application, discussions using terms such as "access", "receive", "send", "use", "select", "determine", "normalize", "multiply", "average", "monitor", "compare", "apply", "update", "measure", "derive", "solve", etc. refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities within the computer system registers and memory and transforms it into other data similarly represented as physical quantities within the computer system memory or registers or other such information storage, transmission, or display devices.

[0026] In the various figures, a single box may be described as performing one or more functions; however, in actual practice, one or more of the functions performed by that box may be performed in a single component or across multiple components, and / or may be performed using hardware, using software, or using a combination of hardware and software. To clearly illustrate this interchangeability of hardware and software, various illustrative components, boxes, modules, circuits, and steps are described generally below in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and the design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be construed as causing a departure from the scope of the present disclosure. Additionally, example devices may include components other than those shown, including well-known components such as processors, memories, and the like.

[0027] Aspects of the present disclosure apply to any suitable image capture device that includes one or more cameras and is configured to or capable of capturing images or video (such as a security system having one or more cameras, a smart phone, a tablet device, a laptop computer, a digital video and / or still camera, a web camera, etc.). Although the following describes an image capture device having or coupled to one camera, aspects of the present disclosure apply to devices having any number of cameras and are thus not limited to devices having only one camera. Aspects of the present disclosure apply to devices configured to capture still images as well as capture video and may be implemented in devices having or coupled to cameras of different capabilities.

[0028] The term "device" or "image capture device" is not limited to a single physical object or a specific number of physical objects (such as a smart phone, a camera controller, a processing system, etc.). As used herein, a device can be any image capture device having one or more components that can implement at least some portions of the present disclosure. Although the following description and examples use the term "device" to describe various aspects of the present disclosure, the term "device" is not limited to a specific configuration, type, or number of objects.

[0029] Figure 1 FIG. is a block diagram of an example device 100 configured to generate a panoramic image according to some implementations. Example device 100 may include or be coupled to a camera 102, a processor 104, a memory 106 storing instructions 108, and a controller 110. In some implementations, device 100 may include (or be coupled to) a display 114 and a plurality of input / output (I / O) components 116. Device 100 may also include a power supply 118, which may be coupled to or integrated into device 100. Device 100 may include additional features or components not shown. For example, a wireless interface for a wireless communication device may be included, which may include a plurality of transceivers and a baseband processor. In another example, device 100 may include or be coupled to additional cameras in addition to camera 102. Device 100 may also include or be coupled to one or more flashlights, depth sensors, or other suitable imaging components.

[0030] Camera 102 may be any suitable camera capable of capturing individual image frames (such as still images) and / or capturing video (such as a series of captured image frames). Camera 102 may include a single image sensor, or it may be a dual camera module or any other suitable module having one or more image sensors (such as one or more CMOS image sensors).

[0031] Memory 106 may be a non-transitory or non-temporary computer-readable medium storing computer-executable instructions 108 to perform all or part of one or more operations described in the present disclosure. Processor 104 may be one or more suitable processors capable of executing scripts or instructions of one or more software programs (such as instructions 108) stored within memory 106. In some aspects, processor 104 may be one or more general-purpose processors that execute instructions 108 to cause device 100 to perform any number of functions or operations. In additional or alternative aspects, processor 104 may include integrated circuits or other hardware to perform functions or operations without using software.

[0032] Although in Figure 1In the example, they are shown as being coupled to each other via the processor 104, but the processor 104, the memory 106, the controller 110, the display 114, and the I / O components 116 can be coupled to each other in various arrangements. For example, the processor 104, the memory 106, the controller 110, the display 114, and / or the I / O components 116 can be coupled to each other via one or more local buses (not shown for simplicity).

[0033] The display 114 can be any suitable display or screen that allows user interaction and / or presents items (such as captured images, videos, or preview images) for the user to view. In some aspects, the display 114 can be a touch-sensitive display. The I / O components 116 can be or include any suitable mechanism, interface, or device that receives input (such as commands) from the user and provides output to the user. For example, the I / O components 116 can include (but are not limited to) a graphical user interface, a keyboard, a mouse, a microphone, and speakers, etc. The display 114 and / or the I / O components 116 can provide a preview image to the user and / or receive user input for adjusting one or more settings of the camera 102.

[0034] The controller 110 can include one or more controllers. One or more controllers can be configured to control the camera 102. The controller 110 can include an image signal processor (ISP) 112, which can be one or more image signal processors for processing the captured image frames or videos provided by the camera 102. In some aspects, the controller 110 or the ISP 112 can execute instructions from the memory (such as instructions 108 from the memory 106 or instructions stored in a separate memory coupled to the ISP 112). In other aspects, the controller 110 or the ISP 112 can include specific hardware. The controller 110 or the ISP 112 can alternatively or additionally include a combination of specific hardware and the ability to execute software instructions. For example, the ISP 112 can include or otherwise be configured to implement one or more stages of an image processing pipeline.

[0035] Figure 2 A trajectory diagram 200 for generating a conventional 360-degree panoramic image is shown. The trajectory diagram 200 shows a top view of the user 210 initiating the generation of a 360-degree panoramic image while facing the optical axis of the camera at the starting point 220. The camera can be included in an image capture device. During the generation of the panoramic image, the user 210 rotates the camera to the right along a path 230 for a full 360-degree rotation (at Figure 2in a clockwise direction) until the optical axis faces the starting point 220 again. For example, the path 230 can be a horizontal path for a horizontal panoramic image, a vertical path for a vertical panoramic image, or any other suitable path. Additionally, it should be noted that although the path 230 is shown as advancing to the right of the user (in Figure 2 in a clockwise direction), other paths can advance to the left of the user, upward, or at any suitable angle, or in any traveling or rotational direction. As further discussed below, the user interface of the image capture device can display one or more graphical elements to assist the user 210 in keeping the camera level while rotating the camera along the path 230. For example, the user interface can include the display 114 and / or the I / O component 116 of the device 100. As the camera rotates along the path 230, the camera can capture multiple images from multiple perspectives. The image capture device can combine the multiple images captured by rotating the camera into a 360-degree panoramic image centered at point 240. Point 240 is 180 degrees from the starting point 220 along the path 230. To generate the panoramic image, the image capture device can align the edges of the captured images and combine them using appropriate techniques (such as image stitching, stretching, warping, morphing, etc.) to generate a 360-degree panoramic image.

[0036] As discussed above, based on the user selecting an appropriate starting point, the user may have difficulty centering the panoramic image as desired. For example, to generate a 360-degree panoramic image, the user must select a starting point that is 180 degrees from the desired point along a path that rotates the camera 360 degrees. This determination can be even more complex for panoramic images that are less than 360 degrees in the FoV. Thus, example implementations allow for receiving an indication specifying the desired center point of the panoramic image. Such a desired center point can also be referred to as a center region of interest (ROI). The images captured during the rotation of the camera can then be automatically processed to generate a panoramic image centered at the center ROI. Example implementations can thus allow for a simple and reliable selection of the desired center point of the panoramic image while allowing the user to start generating the panoramic image at any starting point.

[0037] Figure 3 A trajectory diagram 300 for generating a 360-degree panoramic image according to some implementations is shown. The trajectory diagram 300 shows a top view of the user 310 initiating the generation of a 360-degree panoramic image while facing the optical axis of the camera at the starting point 320. For example, the camera can be Figure 1 the camera 102 of the device 100. During the generation of the panoramic image, the user 310 rotates the camera to the right along the path 330 for a full 360-degree rotation (in Figure 3in a clockwise direction in the middle), until the optical axis faces the starting point 320 again. For example, the path 330 can be a horizontal path for a horizontal panoramic image, a vertical path for a vertical panoramic image, or any other suitable path. Additionally, it should be noted that although the path 330 is shown as advancing to the right of the user (in Figure 3 in a clockwise direction in the middle), other implementations can include paths that advance to the left of the user, upward, or at any suitable angle, or in any suitable traveling or rotational direction. As further discussed below, the user interface of the image capture device can display one or more graphical elements to assist the user 310 in keeping the camera level while rotating the camera along the path 330. As the camera rotates along the path 330, the camera can capture multiple images from multiple perspectives. For example, the user interface can include the display 114 and / or the I / O component 116 of the device 100. For traditional panoramic image generation techniques, the center point would be at point 340. However, during the capture of multiple images, an indication specifying the desired center point 350 can be received, such as a user selection, an automatic selection, or any other suitable indication. After the camera has rotated 360 degrees along the entire path 330, the image capture device can automatically process the multiple images into a 360-degree panoramic image centered at point 350. Similar to Figure 2 , the image capture device can align the edges of the captured images and combine them using suitable techniques such as image stitching to generate a 360-degree panoramic image. Additionally, the image capture device can determine a split point 360 that is 180 degrees from the center point 350. The image capture device can then stitch the starting capture point of the panoramic image to the ending capture point, in other words, stitch the image captured immediately after starting to generate the panoramic image at the starting point 320 to the image captured immediately before reaching the starting point 320 after rotating the camera through the entire path 330. The image capture device can use image stitching, stretching, warping, deforming, or other suitable image stitching techniques to stitch the starting capture point to the ending capture point. The image capture device can also split the panoramic image at the split point 360 to generate a panoramic image centered at the desired center point 350.

[0038] Other aspects of the example implementation can allow for the automatic recentering of panoramic images with a field of view less than 360 degrees. It is even more difficult for the user to determine the appropriate starting point for a panoramic image with a field of view less than 360 degrees using traditional techniques. The required starting point is not approximately 180 degrees from the desired center point, but rather is half of the total field of view away from the desired center point. Thus, for a 200-degree panoramic image, for traditional panoramic image capture techniques, the starting point must be 100 degrees from the desired center point. The example implementation can allow for the reception of an indication of the center ROI during the capture of multiple images for generating a panoramic image. As with the above regarding Figure 3As in the described example, a panoramic image centered on the central ROI can then be automatically generated. However, contrary to the example of Figure 3 , generating the re-centered panoramic image can include cropping a portion of the captured image such that the panoramic image is centered on the central ROI.

[0039] Figure 4 FIG. 400 shows a trajectory for generating a panoramic image of less than 360 degrees according to some implementations. FIG. 400 shows a top view of a user 410 initiating the generation of a panoramic image of less than 360 degrees while facing the optical axis of the camera at a starting point 420. For example, the camera can be the camera 102 of the device 100 of Figure 1 . During the generation of the panoramic image, the user 410 rotates the camera to the right (in a clockwise direction in Figure 4 ) along a path 430 for less than 360-degree rotation until the optical axis faces a stopping point 440. For example, the path 430 can be a horizontal path for a horizontal panoramic image, a vertical path for a vertical panoramic image, or any other suitable path. Additionally, it should be noted that while the path 430 is shown as progressing to the right of the user (in a clockwise direction in Figure 3 ), other implementations can include paths that progress to the left of the user, upward, or at any suitable angle, or in any suitable direction of travel or rotation. As discussed further below, the user interface of the image capture device can display one or more graphical elements to assist the user 410 in keeping the camera level while rotating the camera along the path 430. For example, the user interface can include the display 114 and / or the I / O component 116 of the device 100 of Figure 1 . As the camera rotates along the path 430, the camera can capture multiple images from multiple perspectives. For traditional panoramic image capture techniques, the center point would be at point 450. However, during the capture of the multiple images, an indication specifying the desired center point 460 can be received, such as a user selection, an automatic selection, or another suitable indication. After the camera has rotated along the entire path 430, the image capture device can automatically process the multiple images to generate a panoramic image of less than 360 degrees centered on point 460. Similar to Figure 1, the image capture device can align the edges of the captured images and combine them using appropriate techniques such as image stitching, stretching, warping, morphing, or other suitable image mosaicking techniques to generate a panoramic image. To generate a recentered panoramic image centered at point 460, a portion of the captured images can be cropped or trimmed. For example, the end point 470 of the panoramic image can be determined, and the portion of the image captured between the end point 470 and the stop point 440 can be cropped such that the panoramic image can include the images captured along the path 430 between the start point 420 and the end point 470. In this way, the panoramic image is centered at the desired center point 460 instead of point 450.

[0040] Note that while the trajectory diagram 400 depicts cropping a portion of the image captured between the end point 470 and the stop point 440 (in other words, trimming a portion of the captured image from the end of the path 430), in some other implementations, a portion of the captured image can alternatively be cropped from the start of the image capture path. For example, in some implementations, a determination can be made as to whether the desired center point is closer to the starting capture point or the stopping capture point. When the desired center point is closer to the starting capture point, a portion of the captured image can be cropped from the end of the image capture path, as Figure 4 shown. Alternatively, when the desired center point is closer to the stopping capture point, a portion of the image can alternatively be cropped from the start of the image capture path.

[0041] In some implementations, it may be required that the desired center point be at least a minimum angular distance from the starting capture point or the ending capture point. For example, if the user indicates a desired center point that is too close to the starting capture point, the resulting panoramic image may be undesirably small. Thus, in some implementations, when the desired center point is less than a threshold angular distance from the starting capture point or the stopping capture point, the user indication of the desired center point can be ignored. In other words, when the desired center point is too close to the starting capture point or the stopping capture point, the panoramic image can be generated in a conventional manner rather than being automatically processed to center the generated panoramic image at the desired center point.

[0042] As discussed above, when the desired center point is selected to be too close to the stopping capture point, the generated panoramic image may be undesirably small in the FoV. In some implementations, when selecting the desired center point, an indication to continue rotating the camera and continue capturing images can be provided to the user such that the resulting panoramic image has at least a threshold FoV. For example, if the threshold FoV is 100 degrees, the indication can remind the user not to stop capturing images until the camera has rotated 50 degrees from the desired center point. For example, such an indication can be provided on the user interface of the image capture device (such as Figure 1a visual indication on the display of the device 100). The indication can also take any other suitable form, such as an audio indication.

[0043] According to an example implementation, the received indication specifying the central ROI can take any one of a plurality of forms. In some implementations, the received indication can be a user input specifying the central ROI, such as a touch gesture on a touch-sensitive user interface (such as the display 114 of the device 100). In other implementations, the user input can indicate the selection of a menu option or a user interface button, or the pressing of a physical button or key on the image capture device. In some implementations, the user input can be a touch gesture at any location on the touch-sensitive user interface when the camera is facing the desired central ROI. In some implementations, the user input can be a haptic gesture, a verbal command, a gesture detected by a second camera of the device 100 (such as a gesture detected by the front camera of the device 100). In some other implementations, receiving the indication can include automatically determining the central ROI based on one or more characteristics of the plurality of captured images during the generation of a panoramic image. The one or more characteristics can include the depth of one or more objects in the plurality of captured images, for example, determined by one or more depth sensors of the device 100. In some implementations, the central ROI can be determined as the point corresponding to the nearest object of at least a threshold size in the plurality of images. In other implementations, the central ROI can be determined as the point corresponding to the largest object within the specified depth range. In some other implementations, the central ROI can be determined based on one or more face detection operations. For example, if one or more faces are detected when capturing images for generating a panoramic image, the central ROI can be determined at the location of one of the detected faces (such as at the nearest detected face, at the midpoint between two detected faces, etc.).

[0044] Figure 5 shows an example user interface window 500 for an image capture device according to some implementations. For example, the user interface window 500 can be displayed on Figure 1 the device 100. As described above with respect to Figures 2 - 4As mentioned, the user interface of the image capture device can display one or more graphical elements to assist the user in keeping the camera level when capturing multiple images for generating a panoramic image. For example, the thumbnail image 510 can be displayed on the user interface window 500 to help the user keep the image frame of the image capture device at a constant level during image capture. Additional user guidance can be depicted, such as the center horizontal guidance 520, which can indicate the current level of the image frame (e.g., using the arrow of the center horizontal guidance 520). The center horizontal guidance 520 can also include a recommended level for the user, i.e., the center line 521 of the center horizontal guidance 520. Thus, during the capture of multiple images, the user can adjust the level of the image frame to keep the arrow aligned with the center line 521. The center horizontal guidance 520 can also include an upper frame indicator 522 and a lower frame indicator 523, which represent the recommended levels for the user to maintain the top and bottom of the image frame during the capture of multiple images. The user interface window 500 can also include a button 530, which can be selected to initiate the generation of a panoramic image, to indicate the center ROI for the panoramic image, or to make other selections.

[0045] In some implementations, the user interface window 500 can include a mode identifier 540, which indicates that the image capture device is in a mode configured to generate a recentered panoramic image. In some implementations, before generating a panoramic image, the image capture device can be placed in a mode for generating a recentered panoramic image. For example, user input such as a button press or a selection of a menu option can place the image capture device in a mode for generating a recentered panoramic image. The mode identifier 540 can indicate the current mode in which the image capture device is configured. In some implementations, the mode identifier 540 can also display mode options 541. The mode options 541 can indicate one or more currently enabled configuration options for the current mode. For example, the mode options 541 can indicate the currently enabled option type for specifying the center ROI, such as via a user touch gesture, a user button press, a haptic gesture, a verbal command, or an automatic specification based on one or more characteristics of multiple captured images, as discussed above.

[0046] Figure 6 is an illustrative flowchart depicting an example operation 600 for generating a panoramic image according to some implementations. The operation 600 can be performed by any suitable image capture device that includes a camera and is configured to generate a panoramic image. In some implementations, the operation 600 can be performed by a device in Figure 1performed by the device 100 shown in FIG. At block 602, the device 100 receives a user command to begin generating a panoramic image. At block 604, the device 100 uses a camera to capture a plurality of images for generating the panoramic image. At block 606, the device 100 receives an indication specifying a center region of interest during the capture of the plurality of images. At block 608, the device 100 finishes capturing the plurality of images. At block 610, the device 100 automatically generates a panoramic image based on the plurality of images, wherein the panoramic image is centered on the specified center ROI.

[0047] In some aspects, the indication received at block 606 can be a user input specifying the center ROI, such as a touch gesture on a touch-sensitive user interface of the device 100, such as a touch gesture at the center ROI of a preview displayed on the touch-sensitive user interface, or a selection of a menu option on the touch-sensitive user interface indicating when the camera is pointed at the center ROI. In some aspects, receiving the indication at block 606 can include automatically determining the center ROI based on one or more characteristics of the plurality of images, such as the estimated depth of one or more objects in the plurality of images.

[0048] In some aspects, the panoramic image can be a 360-degree panoramic image, and automatically generating the panoramic image at block 610 can include: determining split points corresponding to the specified center ROI; determining a start capture point and an end capture point based on the split points; and generating the panoramic image based on the start capture point and the end capture point.

[0049] In some aspects, the panoramic image can be a panoramic image of less than 360 degrees, and automatically generating the panoramic image at block 610 can include: comparing the specified center ROI with the maximum field of view of the panoramic image; and cropping the captured panoramic image to be centered on the specified center ROI.

[0050] In some aspects, before receiving the user command at block 602, the device 100 can receive a user input indicating that the device 100 is to operate in a mode configured to generate a recentered panoramic image.

[0051] Figure 7 A block diagram of an example device 700 that supports capturing panoramic images is shown, according to some implementations. In some implementations, the device 700 is configured to perform operation 600 described above with reference to Figure 6 described. The device 700 can be the device 100 described above with reference to Figure 1Example implementation of the described device 100. For example, the wireless communication device 700 can be a chip, an SoC, a chipset, a package or a device including at least one processor (such as processor 104), at least one memory (such as memory 106), and including at least one camera (such as camera 102).

[0052] The device 700 includes: a module 702 for receiving a user command to start generating a panoramic image; a module 704 for using at least one camera to capture a plurality of images for generating a panoramic image; a module 706 for receiving an indication specifying a central ROI during the capture of the plurality of images; a module 708 for completing the capture of the plurality of images; and a module 710 for automatically generating a panoramic image based on the plurality of images, wherein the panoramic image is centered on the specified central ROI. One or more portions of modules 702, 704, 706, 708, and 710 can be implemented at least partially in hardware or firmware. For example, modules 702 and 706 can be implemented at least partially by a display and input / output components (such as display 114 and input / output component 116). Modules 704 and 708 can be implemented at least partially by a camera (such as camera 102). In some implementations, at least some of modules 702, 704, 706, 708, and 710 can be implemented at least partially as software stored in a memory (such as memory 106). For example, one or more portions of modules 702, 704, 706, 708, and 710 can be implemented as non-transitory instructions or "code" (such as instruction 108, which can be executed by a processor (such as processor 104) to perform the functions or operations of the corresponding module).

[0053] Module 702 is configured to receive a user command to start generating a panoramic image. Module 704 is configured to use at least one camera to capture a plurality of images for generating a panoramic image. Module 706 is configured to receive an indication specifying a central ROI during the capture of the plurality of images. Module 708 is configured to complete the capture of the plurality of images. Module 710 is configured to automatically generate a panoramic image based on the plurality of images, wherein the panoramic image is centered on the specified central ROI.

[0054] The techniques described herein can be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a particular manner. Any feature described as a module or component can also be implemented together in an integrated logic device, or separately as discrete but interoperable logic devices. If implemented in software, the techniques can be at least partially embodied by a non-transitory processor-readable storage medium (such as Figure 1implemented by the memory 106 in the example device 100, the storage medium includes instructions 108, and when the instructions 108 are executed by the processor 104 (or the controller 110 or the ISP 112), the device 100 is caused to execute one or more of the above methods. The non-transitory processor-readable data storage medium may form a part of a computer program product, and the computer program product may include packaging material.

[0055] The non-transitory processor-readable storage medium may include a random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read only memory (EEPROM), flash memory, other known storage media, etc. Additionally or alternatively, the techniques may be implemented at least in part via a processor-readable communication medium that carries or conveys code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer or other processor.

[0056] The various illustrative logical blocks, modules, circuits, and instructions described in connection with the embodiments disclosed herein may be executed by one or more processors such as Figure 1 the processor 104 or the ISP 112 in the example device 100. Such processors may include, but are not limited to, one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), application specific instruction set processors (ASIPs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. As used herein, the term "processor" may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functions described herein may be provided within dedicated software modules or hardware modules configured as described herein. Further, these techniques may be fully implemented in one or more circuits or logic elements. A general purpose processor may be a microprocessor and / or any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0057] Although the present disclosure shows illustrative aspects, it should be noted that various changes and modifications can be made herein without departing from the scope of the appended claims. For example, one or more steps in operation 600 can be omitted, or one or more additional processing steps can be added. Additionally, unless otherwise explicitly stated, the functions, steps, or acts of the method claims according to the aspects described herein need not be performed in any particular order. For example, if performed by device 100, controller 110, processor 104, and / or ISP 112, the steps of the described example operations can be performed in any order and at any frequency. Further, although elements may be described or claimed in the singular form, the plural form is contemplated unless explicitly stated to be limited to the singular form. Thus, the present disclosure is not limited to the shown examples, and any unit for performing the functions described herein is included in the aspects of the present disclosure.

Claims

1. A method for generating a panoramic image, the method being performed by an image capture device including a camera, the method comprising: Receiving a user command to start generating the panoramic image; Using the camera to capture a plurality of images for generating the panoramic image; Receiving, while capturing the plurality of images, a user input specifying a central region of interest (ROI) to be used as the center of the panoramic image; Completing the capture of the plurality of images; And Automatically generating, based on the plurality of images, the panoramic image centered on the ROI specified by the user input.

2. The method according to claim 1, wherein, The user input specifying the central region of interest (ROI) includes a touch gesture on a touch-sensitive user interface of the image capture device.

3. The method according to claim 2, wherein The touch gesture includes a touch gesture at the central region of interest (ROI) of a preview displayed on the touch-sensitive user interface.

4. The method according to claim 2, wherein The touch gesture indicates a selection of a menu option on the touch-sensitive user interface when the camera is pointed at the central region of interest (ROI).

5. The method according to claim 1, wherein The panoramic image is a 360-degree panoramic image, and wherein automatically generating the panoramic image includes: Determining a split point corresponding to the specified central region of interest (ROI); Determining a start capture point and an end capture point based on the split point; and Generating the panoramic image based on the start capture point and the end capture point.

6. The method according to claim 1, wherein The panoramic image is a panoramic image of less than 360 degrees, wherein automatically generating the panoramic image includes: Comparing the specified central region of interest (ROI) with the maximum field of view of the panoramic image; and Cropping the panoramic image to be centered on the specified central region of interest (ROI).

7. The method according to claim 1, further comprising: Before receiving the user command to start generating the panoramic image, receiving a user input configuring the camera to operate in a mode configured to generate a re-centered panoramic image.

8. An image capture device, comprising: One or more cameras; One or more processors; And A memory storing instructions that, when executed by the one or more processors, cause the image capture device to perform operations including the following: Receiving a user command to start generating a panoramic image using the one or more cameras; Using the one or more cameras to capture a plurality of images for generating the panoramic image; Receiving, while capturing the plurality of images, a user input specifying a central region of interest (ROI) to be used as the center of the panoramic image; Completing the capture of the plurality of images; And Automatically generating, based on the plurality of images, the panoramic image centered on the ROI specified by the user input.

9. The image capturing device according to claim 8, wherein, The user input specifying the central region of interest (ROI) includes a touch gesture on a touch-sensitive user interface of the image capture device.

10. The image capturing device according to claim 9, wherein, The touch gesture includes a touch gesture at the central region of interest (ROI) of a preview displayed on the touch-sensitive user interface.

11. The image capturing device according to claim 9, wherein, The touch gesture indicates selection of a menu option on the touch-sensitive user interface when the one or more cameras are pointed at the center region of interest (ROI).

12. The image capturing device according to claim 8, wherein, The panoramic image is a 360-degree panoramic image, and wherein execution of the instructions for automatically generating the panoramic image causes the image capture device to perform operations further comprising: Determine split points corresponding to the specified center region of interest (ROI); Determine a start capture point and an end capture point based on the split points; and Generate the panoramic image based on the start capture point and the end capture point.

13. The image capturing device according to claim 8, wherein, The panoramic image is a panoramic image of less than 360 degrees, wherein execution of the instructions for automatically generating the panoramic image causes the image capture device to perform operations further comprising: Compare the specified center region of interest (ROI) with the maximum field of view of the panoramic image; and Crop the panoramic image to be centered on the specified center region of interest (ROI).

14. The image capturing device according to claim 8, wherein, Execution of the instructions causes the image capture device to perform operations further comprising: before receiving the user command to start generating the panoramic image, receive user input that configures the one or more cameras to operate in a mode configured to generate a recentered panoramic image.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of an image capture device including a camera, cause the image capture device to perform operations comprising: Receive a user command to start generating a panoramic image; Use the camera to capture a plurality of images for generating the panoramic image; During capture of the plurality of images, receive user input specifying a center region of interest (ROI) to be used as the center of the panoramic image; Complete capture of the plurality of images; And Automatically generate the panoramic image centered on the ROI specified by the user input based on the plurality of images.

16. The non-transitory computer-readable storage medium according to claim 15, wherein, The user input specifying the center region of interest (ROI) includes a touch gesture on the touch-sensitive user interface of the image capture device.

17. The non-transitory computer-readable storage medium according to claim 16, wherein, The touch gesture includes a touch gesture at the center region of interest (ROI) of a preview displayed on the touch-sensitive user interface.

18. The non-transitory computer-readable storage medium according to claim 16, wherein, The touch gesture indicates selection of a menu option on the touch-sensitive user interface when the camera of the image capture device is pointed at the center region of interest (ROI).

19. The non-transitory computer-readable storage medium according to claim 15, wherein, The panoramic image is a 360-degree panoramic image, and wherein execution of the instructions for automatically generating the panoramic image causes the image capture device to perform operations further comprising: Determine split points corresponding to the specified center region of interest (ROI); Determine a start capture point and an end capture point based on the split points; and Generate the panoramic image based on the start capture point and the end capture point.

20. The non-transitory computer-readable storage medium according to claim 15, wherein, The panoramic image is a panoramic image of less than 360 degrees, and wherein the execution of the instructions for automatically generating the panoramic image causes the image capture device to perform operations further including the following: Compare the specified center region of interest (ROI) with the maximum field of view of the panoramic image; and Crop the panoramic image to center on the specified center region of interest (ROI).

21. The non-transitory computer-readable storage medium according to claim 15, wherein, The execution of the instructions causes the image capture device to perform operations further including the following: Before receiving the user command to start generating the panoramic image, receive user input that configures the image capture device to operate in a mode configured to generate a recentered panoramic image.

22. An image capture device, comprising: One or more cameras; A unit for receiving a user command to start generating a panoramic image; A unit for using the one or more cameras to capture a plurality of images for generating the panoramic image; A unit for receiving user input specifying a center region of interest (ROI) during the capture of the plurality of images to be used as the center of the panoramic image; A unit for completing the capture of the plurality of images; And A unit for automatically generating the panoramic image centered on the ROI specified by the user input based on the plurality of images.

23. The image capturing device according to claim 22, wherein, The user input specifying the center region of interest (ROI) includes a touch gesture on the touch-sensitive user interface of the image capture device.

24. The image capturing device according to claim 23, wherein, The touch gesture includes a touch gesture at the center region of interest (ROI) of a preview displayed on the touch-sensitive user interface.

25. The image capturing device according to claim 23, wherein, The touch gesture indicates the selection of a menu option on the touch-sensitive user interface when the camera is pointed at the center region of interest (ROI).

26. The image capturing device according to claim 22, wherein, The panoramic image is a 360-degree panoramic image, and wherein the unit for automatically generating the panoramic image includes: A unit for determining a segmentation point corresponding to the specified center region of interest (ROI); A unit for determining a starting capture point and an ending capture point based on the segmentation point; and A unit for generating the panoramic image based on the starting capture point and the ending capture point.

27. The image capturing device according to claim 22, wherein, The panoramic image is a panoramic image of less than 360 degrees, wherein the unit for automatically generating the panoramic image includes: A unit for comparing the specified center region of interest (ROI) with the maximum field of view of the panoramic image; and A unit for cropping the panoramic image to center on the specified center region of interest (ROI).

28. The image capturing device according to claim 22, further comprising: A unit for receiving user input that configures the camera to operate in a mode configured to generate a recentered panoramic image before receiving the user command to start generating the panoramic image.

Images