Image capture device, image capture device control method and program

Abstract

To easily generate an image group that contains multiple images with a sense of unity.SOLUTION: The imaging apparatus includes display means for displaying a live image and a captured image of a subject on a display unit, photographing means for photographing the subject, and recording means for recording a capturing image of the subject and a captured image of the subject taken by the photographing means in a recording unit in association with arrangement information including a display position of the capturing image of the subject and a display position of the captured image.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] In recent years, it has become possible to view multiple images posted on social network services (SNS) and the like in a list display. When multiple images displayed in a list have similar subjects, compositions, brightness, colors, etc., they can create a sense of unity and give a visually pleasing impression. As a technique for shooting multiple images, Patent Document 1 discloses a technique for displaying an image to be reshot and a live view image, which is a moving image being shot, side by side, allowing a user to create a pose or expression during shooting while checking the image to be reshot. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-116872 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, even if a live view image, which is a moving image being shot, is displayed alongside a previously shot image, it may be time-consuming to take a picture that has a consistent look with the previously shot image, such as changing the pose and position of the subject and taking the picture again.

[0005] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide an imaging device that can easily generate an image group including a plurality of images that have a sense of unity. [Means for solving the problem]

[0006] In order to achieve the above object, an imaging device according to the present invention comprises: a display means for displaying a live image of a subject and a plurality of captured images on a display unit; a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject; An imaging means for imaging the subject; a recording means for recording the photographed image of the subject photographed by the photographing means and the already-photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the already-photographed image; 、 an uploading means for uploading the photographed image of the subject and the photographed image together with the arrangement information recorded in the recording unit to an external server; having The display means displays a live image of the subject captured using the shooting conditions changed by the change means, together with the plurality of captured images. Effect of the Invention

[0007] According to the present invention, an image group including a plurality of images having a uniform look can be easily generated. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an imaging apparatus. [Diagram 2] FIG. 1 is a diagram illustrating an example of the appearance of an imaging apparatus. [Diagram 3] 4 is a flowchart illustrating an image capturing process according to the first embodiment. [Figure 4] 4A and 4B are diagrams for explaining the arrangement of live images and captured images. [Diagram 5] 5A to 5C are diagrams illustrating adjustment of a live image. [Figure 6] 10 is a flowchart illustrating an image capturing process according to the second embodiment. [Figure 7] 10A to 10C are views for explaining examples of screens in the photographing process according to the second embodiment. [Figure 8] FIG. 11 is a diagram illustrating metadata indicating features of an image according to the third embodiment. [Figure 9]11 is a flowchart illustrating an image capturing process according to the third embodiment. [Figure 10] 11 is a flowchart illustrating a captured image arrangement process according to the third embodiment. [Figure 11] FIG. 13 is a diagram illustrating a UI according to the third embodiment. [Figure 12] 13 is a flowchart illustrating a captured image arrangement process according to the fourth embodiment. [Figure 13] FIG. 13 is a diagram illustrating a UI according to the fourth embodiment. [Figure 14] 13 is a flowchart illustrating a captured image arrangement process according to the fifth embodiment. [Figure 15] 20 is a flowchart illustrating a captured image arrangement process according to a sixth embodiment. [Figure 16] FIG. 23 is a diagram illustrating a template of the display position of a captured image according to the seventh embodiment. [Figure 17] FIG. 13 is a diagram illustrating a live view display screen according to the seventh embodiment. [Figure 18] 20 is a flowchart illustrating an image capturing process according to the eighth embodiment. [Figure 19] FIG. 23 is a diagram illustrating a live view display screen according to the eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] <Embodiment 1> Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. In the first embodiment, a live view display screen that displays a live image, which is a moving image of a subject, displays a captured image that has been captured in the past together with the live image. The display position at which the live image is displayed on the screen of the imaging device and the display position at which the captured image is arranged can be determined based on a user's operation. The user can capture an image that has a sense of unity with the captured image by adjusting the composition, brightness, color, etc. of the live image to match the captured image. There are various elements that make the user feel a sense of unity. For example, commonality in composition, such as the position and size of the subject in the image being similar to each other, or the position of the horizon in a landscape image being the same height, can be cited. Another example is commonality in the categories of subjects, such as food, people, landscapes, and animals. Furthermore, commonality in the color and gradation of the subject, and commonality in filters applied to the image, such as saturation emphasis, can be cited.

[0010] The imaging device according to the first embodiment will be described as a smartphone with a camera function, but is not limited thereto. The imaging device may be other devices with an imaging function, such as a digital camera, a digital video camera, a mobile phone, or other portable electronic devices.

[0011] (Device configuration) 1 is a block diagram showing an example of the configuration of an image capturing apparatus 100 according to the first embodiment.

[0012] The control unit 101 is a processor such as a CPU or a DSP. The control unit 101 reads out a control program for each block included in the imaging device 100 from a ROM 102, and loads and executes the program in a RAM 103. This allows the control unit 101 to control the operation of each block included in the imaging device 100.

[0013] The ROM 102 is a non-volatile memory that is electrically erasable and recordable. The ROM 102 stores operation programs for each block of the image pickup device 100, as well as parameters used for the operation of each block.

[0014] The RAM 103 is a rewritable volatile memory that is used to load programs executed by the control unit 101 and the like, and to temporarily store data generated by the operation of each block of the imaging device 100.

[0015] The optical system 104 is made up of a group of lenses including a zoom lens and a focus lens, and forms a subject image on the imaging surface of the imaging unit 105. The imaging unit 105 is an imaging element such as a CCD or CMOS sensor. The optical image formed on the optical axis is photoelectrically converted, and the resulting analog image signal is output to the A / D converter 106 .

[0016] The A / D conversion unit 106 converts the input analog image signal into digital image data. The digital image data output from the A / D conversion unit 106 is temporarily stored in the RAM 103.

[0017] The image processing unit 107 applies various image processing to the image data stored in the RAM 103 in order to develop, display, or record digital image data. Examples of image processing applied to the image data include pixel defect correction processing caused by the optical system 104 or the image sensor, demosaicing processing, white balance correction processing, color interpolation processing, gamma processing, and the like.

[0018] The recording unit 108 records data including image data on a built-in recording medium. The display unit 109 includes a display device such as an LCD (Liquid Crystal Display), and displays images stored in the RAM 103 and images recorded in the recording unit 108 on the display device. The display unit 109 also displays an operation user interface for receiving instructions from a user.

[0019] The input unit 110 is an input interface including various physical operation members such as a touch panel, a shutter button, etc., and receives instructions input from a user. The communication unit 111 wirelessly connects to an external server and transmits and receives data including image data and position information described below. An example of the external server is a server that manages a service such as a social network service (hereinafter referred to as SNS) that receives images taken by a user and makes them available to viewers of the SNS.

[0020] Under the control of the control unit 101, the imaging device 100 performs live view display in which analog image signals sequentially output from the imaging unit 105 are sequentially displayed on a display device via the A / D conversion unit 106, RAM 103, image processing unit 107, and display unit 109. During the live view display, the user can adjust the composition in preparation for actual shooting, which assumes that the image data will be recorded on a recording medium. During the live view display, the user can also prepare for actual shooting by changing exposure conditions such as aperture and shutter speed, and image processing conditions for changing image quality such as brightness, gradation, and color tone.

[0021] Fig. 2 is a diagram illustrating an example of the appearance of an imaging device. With reference to Fig. 2, the appearance of a smartphone as the imaging device 100 will be described. Fig. 2(A) shows a front view of the imaging device 100 (smartphone), and Fig. 2(B) shows a rear view of the imaging device 100.

[0022] The imaging device 100 includes a display unit 109 and a camera unit 201. The camera unit 201 includes the optical system 104, the imaging unit 105, and the A / D conversion unit 106 described in FIG.

[0023] (Photographing process according to the first embodiment) Fig. 3 is a flowchart illustrating an example of an image capturing process according to embodiment 1. The image capturing process of the imaging device 100 in this embodiment will be described in detail with reference to Fig. 3. Each step of the image capturing process is executed by the control unit 101 or by each block of the imaging device 100 that receives an instruction from the control unit 101.

[0024] 4 and 5, a live view display screen and an image group display screen displayed on the display unit 109 in the shooting process shown in FIG. 3 will be described. The live view display screens in FIG. 4(A) to FIG. 4(C), FIG. 5(A), and FIG. 5(B) are display screens before shooting that display live images and allow shot images to be arranged and live images to be adjusted. FIG. 5(C) The image group display screen in is a post-shooting display screen for displaying an image group consisting of a captured image obtained by capturing an adjusted live image and an arranged captured image. Fig. 4 is a diagram for explaining the arrangement of a live image and a captured image. Fig. 5 is a diagram for explaining the adjustment of a live image.

[0025] 3 is started when the user switches to a shooting mode capable of taking pictures with the camera of the imaging device 100. For example, the user can switch to the shooting mode by starting a camera function application on a smartphone.

[0026] In step S301, the control unit 101 controls the optical system 104, the imaging unit 105, the A / D conversion unit 106, the RAM 103, the image processing unit 107, and the display unit 109 to start live view display.

[0027] During the live view display period, the imaging device 100 captures and sequentially acquires live images. The acquired live images are displayed on the display device of the display unit 109. While checking the sequentially displayed live images, the user can adjust the composition and change the exposure conditions and image processing conditions. Note that the processes of steps S302, 303, 304, 305, and 306 described below are processes executed during the live view display period.

[0028] In step S302, the control unit 101 displays a template of a display position where the image is to be arranged on the display unit 109. The template is a plurality of areas that are divided in advance. The control unit 101 displays a template that has been set in advance or a template that has been selected by the user.

[0029] The template for the display position of an image will be described with reference to Fig. 4(A). The template for the display position includes a plurality of frames (areas) for arranging images. For example, the user can select the display position of an image by dragging an image selected from unarranged images displayed on the display unit 109 into a specific area of ​​the template.

[0030] 4(A) shows a live view display screen on which a display position template 401 is displayed on the display unit 109. The template 401 includes a total of nine rectangular areas of three rows and three columns. Each rectangular area indicates the position at which an image is to be placed. One image is placed in one rectangular area. That is, nine images can be placed on the template 401.

[0031] The template of the display position is not limited to the example of FIG. 4(A). For example, the template of the display position may be divided into four areas in which four images can be arranged. The divided areas are not limited to rectangular, and may be of various shapes such as circular or polygonal. The template of the display position may be one in which the sizes of the areas are different. Furthermore, the display position of the image may be determined by the user freely arranging the image within a predetermined area without using a template.

[0032] In step S303, the control unit 101 determines a display position for displaying a live image of a subject to be photographed from the display position template 401 displayed on the live view display screen. The control unit 101 can determine an area selected by the user from the areas included in the template 401 as the display position for the live image.

[0033] With reference to Fig. 4(B), the determination of the display position of a live image will be described. With a display position template 401 displayed as shown in Fig. 4(A), the user selects a rectangular area in which the live image is to be displayed by a touch operation. The control unit 101 sets the rectangular area selected by the user as the display position of the live image, and displays the live image in that rectangular area. The live view display screen in Fig. 4(B) is a display screen in which the user selects the central rectangular area 402 out of nine rectangular areas. as the display position of the live image. The control unit 101 determines a rectangular area 402 as the display position of the live image, and displays a live image of a plate and a slice of bread with a fried egg on it in the rectangular area 402. Note that the subject is not limited to food, and may be a person, an animal, a building, a landscape, or the like.

[0034] The display position of the live image is not limited to being selected and determined by the user, but may be preset for each template, for example.

[0035] In step S304, the control unit 101 arranges the shot image in the template 401 at the display position based on a user operation. The user operation is, for example, an operation of dragging and dropping an image selected from the unarranged shot images displayed on the display unit 109 to a position (area) in the template 401 where the image is to be arranged. The control unit 101 arranges the shot image in the area to which the user has moved the image selected from the unarranged shot images by the drag-and-drop operation.

[0036] Selection and arrangement of captured images will be described with reference to Fig. 4(C). The live view display screen in Fig. 4(C) shows an example in which five captured images are arranged around a rectangular area 402, which is the display position of the live image, by the process in step S304.

[0037] When the display position of the live image is determined in step S303, the control unit 101 displays a list of unarranged captured images 403 in a position separate from the template 401. In the example of FIG. 4(C), the list of unarranged captured images 403 displays five captured images that have not yet been arranged. The images displayed in the list of unarranged captured images 403 are acquired from the recording unit 108. The user can arrange the captured images one by one by selecting a desired captured image while checking the unarranged captured images displayed on the live view display screen, and dragging and dropping it to the desired position on the template 401.

[0038] In order to create a sense of unity when images in an image group are displayed in a list, the user selects and arranges images that have similar subjects, composition, brightness, color, etc. to the live image and the arranged photographed image from the unarranged photographed image list 403. In the example of FIG. 4(C), the images of chickens and flowers in the unarranged photographed image list 403 have a different subject from the bread in the live image displayed in the rectangular area 402. For this reason, the user selects and arranges an image that has the same subject as the live image, bread, instead of the chicken and flower images, from the unarranged photographed image list 403. In this way, the control unit 101 arranges the photographed images in the template 401 based on the user's operation of arranging a desired image in a desired position.

[0039] The arrangement of the captured images is not limited to being determined based on a user operation. The control unit 101 may analyze each image in the list of captured images 403 that have not been arranged, and arrange images in the template 401 that have similar subjects, compositions, brightness, colors, and the like to the live image.

[0040] In step S305, the control unit 101 adjusts the shooting conditions of the live image based on an instruction from the user so that the composition, brightness, color, etc. of the live image are similar to those of the arranged captured image. The shooting conditions are, for example, conditions related to image processing that can be adjusted when capturing the live image, such as the composition, exposure conditions, color, and gradation.

[0041] A method of adjusting the composition will be described as an example of adjusting the shooting conditions with reference to Fig. 5(A) and Fig. 5(B). Fig. 5(A) and Fig. 5(B) show an example of a live view display screen on which eight captured images are arranged in step S304. A zoom bar 501 is a user interface (UI) for changing the zoom magnification of the live image. The shooting button 502 is a UI for performing actual shooting by pressing (touch operation).

[0042] 5(A), the bread that is the subject of the live image in rectangular area 402 is displayed at a smaller size than the bread in the captured images arranged around it. If a photograph is taken in this state, the subject size of the live image and the captured image will differ, and the display of the image group that includes the captured image of the subject and the captured image will lack unity and will be undesirable.

[0043] The live view display screen in FIG. 5(B) shows an example in which the user increases the zoom magnification of the live image to provide a sense of unity in the display of the image group. The user can increase the zoom magnification by moving the knob of the zoom bar 501 in the + direction. The size of the subject increases by increasing the zoom magnification. In FIG. 5(A), the subject of the live image is smaller than the subjects of the surrounding captured images, but in FIG. 5(B), by increasing the zoom magnification, the subject becomes approximately the same size as the surrounding captured images. The user can change the zoom magnification of the live image to make the subject sizes of the live image and the captured images uniform, thereby enabling the live image to be displayed with a sense of unity in the list of captured images and the list of captured images.

[0044] 5(A) and 5(B) show examples of adjusting the composition, but are not limited thereto. The user may be able to adjust shooting conditions such as the brightness and color of a live image on the live view display screen. That is, the live view display screen may include a UI for adjusting the brightness, color, and the like of a live image. The user can shoot an image that has a more unified look compared to previously shot images by adjusting any one of the composition, brightness, color, and the like, or a combination of these.

[0045] Furthermore, the shooting conditions are not limited to being changed by the user, but may be automatically changed or suggested to be changed by the imaging device 100. For example, the control unit 101 can automatically change the brightness, color, etc. of the live image so that a sense of unity is created with respect to the captured images. Regarding the composition, the control unit 101 can automatically change the zoom magnification of the live image so that the size of the subject is uniform between the live image and the captured image.

[0046] In the automatic adjustment of the shooting conditions, the imaging device 100 adjusts the shooting conditions so that the captured image and the live image have the same composition, brightness, and color. The shooting conditions are image processing conditions for changing the composition, exposure conditions, color, gradation, etc.

[0047] A method of adjusting the composition will be described as an example of adjusting the shooting conditions with reference to Fig. 5(A) and Fig. 5(B). Fig. 5(A) and Fig. 5(B) show an example of a live view display screen in which eight shot images are selected and arranged in step S304. A shooting button 502 is a UI for accepting a shooting operation.

[0048] 5(A), the bread in the live image displayed in rectangular area 402 is displayed at a smaller size than the bread in the surrounding captured images. If a photograph is taken in this state, the subject size will be different between the live image in rectangular area 402 and the captured images, and the list display of these images may not have a consistent look.

[0049] In order to give a sense of uniformity to the size of the subject, the control unit 101 calculates the size of the subject in the captured images, for example, the vertical and horizontal lengths of a rectangle surrounding the subject. For example, the control unit 101 can set the average size of each captured image as the size of the subject in the live image, and adjust the size of the subject in the live image. Note that if the size of the area displaying the captured images and the area displaying the live image differ, the control unit 101 adjusts the size of each area. The size of the object in the live image can be set according to the area ratio.

[0050] The control unit 101 adjusts the focal length of the optical system 104 so that the live image is displayed at the set subject size. In the example of Fig. 5(A), the subject of the live image displayed in the rectangular area 402 is smaller than the subjects of the surrounding captured images. Therefore, the control unit 101 changes the focal length of the lens to the telephoto side so that the subject size appears larger, as shown on the live view display screen in Fig. 5(B).

[0051] The imaging device 100 equipped with a plurality of optical systems 104 may switch the optical system 104 to be used depending on the size of the subject. The imaging device 100 equipped with a fixed focal length lens like a smartphone can adjust the subject size of the live image to that of the captured image by cropping and enlarging the image (electronic zoom) instead of adjusting the focal length. The imaging device 100 can also adjust the subject size of the live image to that of the captured image by using both the focal length change of the optical system 104 and electronic zoom.

[0052] By adjusting the subject size, the subject in the live image, which is smaller than the subjects in the surrounding captured images in Fig. 5(A), can be enlarged to the same size as the subjects in the surrounding captured images, as in Fig. 5(B).By automatically changing the zoom magnification, the subject sizes in multiple images become uniform, so the list of multiple images displayed on the live view display screen after shooting has a unified look.

[0053] The process of unifying the size of the subject in a plurality of images is not limited to adjusting the size of the subject in the live image based on the captured image. The control unit 101 may adjust the size of the subject in the captured image based on the live image. For example, if the subject in the captured image is smaller than the subject in the live image, the control unit 101 can increase the size of the subject in the captured image by cropping and enlarging the captured image.

[0054] The control unit 101 can provide a unified feeling of blur (depth of field) as well as the size of the subject by controlling the optical system, processing the image, or by combining these processes. For example, a list display of portrait images is more preferable when the unified feeling of blur is provided. The depth of field varies depending on the shooting conditions such as the focal length, the subject distance, and the aperture value. The control unit 101 can make the depth of field of the live image the same as that of the shot image by automatically determining the shooting conditions for the live image based on the shooting conditions of the shot image. The control unit 101 can unify the feeling of blur of the list display by matching the depth of field of the live image to that of the shot image.

[0055] In step S306, the control unit 101 determines whether the adjustment of the photographing conditions in step S305 is complete. For example, the control unit 101 can determine whether the adjustment of the photographing conditions is complete based on whether the user has completed the adjustment of the photographing conditions and pressed the shooting button 502. If the adjustment of the photographing conditions is complete (step S306: Yes), the process proceeds to step S307. If the adjustment of the photographing conditions is not complete (step S306: No), the process returns to step S302, and the processes of steps S302, S303, S304, and S305 are repeated until the adjustment of the photographing conditions is complete. In this way, the control unit 101 can capture an image that has a sense of unity with the captured images by adjusting the arrangement, composition, brightness, and color of the live image based on the user's instruction.

[0056] In step S307, when the adjustment of the live image is completed in step S306 and the photographing button 502 is pressed, the control unit 101 performs actual photographing. Since the image is captured in the state adjusted in step S305, the image will have substantially the same composition, brightness, and color as the captured image arranged on the live view display screen.

[0057] The control unit 101 associates the captured images of the live images captured by the main shooting with the captured images arranged on the live view display screen as one image group. The control unit 101 also records the captured images of the live images and the captured images in the recording unit 108 in association with arrangement information including the display positions of the live images and the respective display positions of the captured images.

[0058] In step S307, the user presses the shooting button 502 to perform the actual shooting, but the actual shooting may be performed automatically. When the control unit 101 determines in step S305 that the adjustment of the shooting conditions, such as the size of the subject of the live image, has been completed in accordance with the shot image, the actual shooting can also be performed automatically.

[0059] Furthermore, the control unit 101 may notify the user that the adjustment of the shooting conditions has been completed. For example, the control unit 101 displays a message notifying the user that the adjustment of the shooting conditions has been completed on the display unit 109. The user can check the message displayed on the display unit 109 and perform actual shooting.

[0060] In step S308, the control unit 101 acquires the image data and arrangement information of the image group recorded in the recording unit 108, and displays the image data of the image group on the display unit 109 based on the acquired arrangement information.

[0061] With reference to Fig. 5(C), an image group display screen that displays image data within an image group will be described. The image group display screen in Fig. 5(C) shows an example in which an image of a subject captured by actual photography is displayed in a rectangular area 503, with previously captured images associated with the captured image displayed around it. Based on the image group layout information, the control unit 101 lays out the image data within the image group on the display surface of the display unit 109. When each image is displayed on the image group display screen, the photographing process is completed.

[0062] In the above-described first embodiment, the user can adjust the composition, brightness, color, etc. of the live image while comparing it with the captured image arranged on the live view display screen. This allows the user to capture a live image that has a sense of unity with the captured image. Therefore, the user can easily generate a group of images that have a sense of unity.

[0063] In the first embodiment, the adjustment is described for adjusting the composition of the live image to match the captured image displayed on the live view display screen, but the adjustment of the live image is not limited to this. For example, the adjustment of the live image includes adjustment of the exposure conditions, or adjustment of the image processing conditions for changing the image quality such as brightness, gradation, and color tone.

[0064] Specifically, the adjustment of the exposure conditions may be a process of adjusting the brightness of the live image by adjusting the amount of exposure compensation, and the adjustment of the image processing conditions may be a process of adjusting the vividness of the live image by adjusting the amount of saturation emphasis.

[0065] Furthermore, the imaging device 100 may upload the image data and arrangement information of the generated image group to an external server that provides a service such as SNS, thereby enabling the user to post an image group with a uniform look.

[0066] <Embodiment 2> In the second embodiment, the captured images displayed on the live view display screen are displayed on a service such as a social networking site. In this embodiment, the image capturing apparatus according to the second embodiment acquires the image capturing information from an external server that provides the image capturing information. The configuration of the image capturing apparatus according to the second embodiment is similar to that of the image capturing apparatus 100 according to the first embodiment shown in FIG.

[0067] The external server receives the image groups of captured images and the corresponding arrangement information from the multiple imaging devices 100 and records them in a recording unit of the external server. When accessed from a web browser or the like, the external server can display a list of captured images included in the image groups based on the arrangement information.

[0068] (Photographing process according to the second embodiment) Fig. 6 is a flowchart illustrating an example of an image capturing process according to the second embodiment. The image capturing process of the imaging device 100 in this embodiment will be described in detail with reference to Fig. 6. Each step of the image capturing process is executed by the control unit 101, or by each block of the imaging device 100 that receives an instruction from the control unit 101. Note that the same steps as those in the image capturing process according to the first embodiment shown in Fig. 3 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0069] Also, with reference to Fig. 7, a live view display screen and an image group display screen displayed on the display unit 109 in the shooting process shown in Fig. 6 will be described. The live view display screens of Fig. 7(A) and Fig. 7(B) are screens on which captured images are arranged based on image data and arrangement information of the image group acquired from an external server. The image group display screen of Fig. 7(C) is a screen that displays images in an image group after shooting, and has a UI for sending a captured image of a subject and the corresponding arrangement information to an external server.

[0070] In step S301, the control unit 101 starts live view display. In step S601, the control unit 101 acquires, via the communication unit 111, shot images of an image group recorded in an external server and corresponding arrangement information.

[0071] In step S602, the control unit 101 arranges the captured images (image data in the image group) acquired in step S601 on the live view display screen based on the arrangement information.

[0072] The arrangement of captured images will be described with reference to Fig. 7(A). Fig. 7(A) shows an example in which seven captured images obtained from an external server are arranged on a live view display screen based on the corresponding arrangement information. The arrangement information includes information on the template of the display position used to arrange the captured images.

[0073] 7A shows an example in which seven captured images are displayed in seven of a template including nine rectangular areas. Of the nine rectangular areas, no captured images are arranged in the lower center and lower right rectangular areas. In this way, the imaging device 100 can acquire captured images of an image group recorded in an external server and corresponding arrangement information, and arrange the captured images on a live view display screen based on the acquired arrangement information. In other words, the imaging device 100 can reproduce image data in an image group on a live view display screen in the same arrangement as when the image data is viewed by accessing an external server.

[0074] The display position of a captured image on the live view display screen may be changed by the user. The user can change the position of a captured image by dragging and dropping the captured image to the desired position. If a captured image is already located at the desired position, the image to be changed may be swapped with the image at the desired position.

[0075] In step S603, the control unit 101 determines the display position of the live image. For example, the control unit 101 can determine an area selected by the user from among rectangular areas in the template in which no captured image is arranged, as the display position of the live image.

[0076] With reference to Fig. 7(B), the determination of the display position of a live image will be described. In the example of Fig. 7(A), the user selects, by touch operation, one of the nine rectangular areas, a rectangular area at the bottom center or bottom right, in which no captured image is located, as the display position of a live image. Fig. 7(B) shows an example in which the user selects the rectangular area 701 at the bottom center. The control unit 101 displays a live image in the rectangular area 701 selected by the user.

[0077] In steps S305 and S306, the control unit 101 changes the shooting conditions of the live image based on the user's instruction and determines whether the adjustment of the shooting conditions is completed, similarly to the first embodiment. If the adjustment of the shooting conditions is not completed (step S306: No), the process returns to step S601, and the processes of steps S601, S602, S603, and S305 are repeated until the adjustment of the shooting conditions is completed.

[0078] In steps S307 and S308, the control unit 101 performs actual shooting, similarly to the first embodiment, associates the shot images of the live images with the shot images arranged in the template as an image group, and displays them on the image group display screen.

[0079] In step S604, the control unit 101 determines, based on a user instruction, whether or not to transmit the image data in the image group and the corresponding layout information to an external server.

[0080] With reference to Fig. 7(C), the reception of an instruction to send image data and arrangement information of an image group will be described. Fig. 7(C) shows an example of a screen in which a UI for receiving an instruction as to whether or not to send the image data and arrangement information of an image group to an external server has been added to the image group display screen that displays the image group in step S308 in Fig. 6. The user can check the consistency of the composition, brightness, color, etc. within the image group and determine whether or not to send the image data and arrangement information of the image group to an external server.

[0081] When the user wishes to transmit the image data and arrangement information of the image group to the external server, the user presses (touches) the "Yes" button. When the "Yes" button is operated, the control unit 101 determines that the image group and the corresponding arrangement information are to be transmitted to the external server (step S604: Yes), and the process proceeds to step S605.

[0082] If the user does not want to transmit the image data and arrangement information of the image group to the external server, the user presses (touches) the "No" button. When the "No" button is pressed, the control unit 101 determines that the image group and the corresponding arrangement information will not be transmitted to the external server (step S604: No), and the photographing process shown in Fig. 6 ends.

[0083] In step S605, the control unit 101 transmits the image data of the image group and the corresponding arrangement information to the external server via the communication unit 111, and the photographing process shown in FIG. 6 ends.

[0084] In the above-described second embodiment, the external server can generate a screen that displays the image data in the image group in the arrangement at the time of shooting, based on the image data of the image group and the corresponding arrangement information received from the imaging device 100. As a result, when the image group is viewed by accessing the external server from a web browser or the like, it is possible to view the image data in the same arrangement as when it was shot by the imaging device 100.

[0085] <Embodiment 3> On the live view display screen, a captured image to be compared with a live image is selected by a user operation from the list of unarranged captured images 403. In the third embodiment, in order to reduce the user's effort in selecting and arranging images having the same characteristics from among the captured images, the list of unarranged captured images 403 displays related captured images acquired based on the characteristics of the representative image selected by the user as selection candidates. The representative image is an image set as a representative image among captured images with the same tag. The captured images are classified by adding tags based on the characteristics of the images.

[0086] By the user selecting a representative image, candidates for captured images are narrowed down based on the characteristics of the representative image. This reduces the user's effort in selecting a captured image. The configuration of the imaging device according to the third embodiment is the same as that of the imaging device 100 according to the first embodiment shown in FIG. The following describes processing that differs from that of the first embodiment.

[0087] (Tag addition processing) The control unit 101 adds tags to captured images in advance and determines a representative image for each tag. First, the tag addition process for adding tags to captured images will be described.

[0088] When a new captured image is added to the recording unit 108, the control unit 101 adds a tag to the added target image. The control unit 101 extracts image features from the captured image added to the recording unit 108. Image features include, for example, the composition, color, and subject of the image.

[0089] In extracting composition features, the control unit 101 detects the contour of a subject from a captured image to be extracted. Based on the detected contour, the control unit 101 determines which of the predetermined composition categories the image falls into. The predetermined composition categories include, for example, a Japanese flag composition, a rule of thirds composition, and a diagonal composition.

[0090] The control unit 101 can extract the composition features by determining the composition pattern based on the contour of the subject. Note that the control unit 101 may extract the composition features by any method as long as the method is capable of extracting the composition features. For example, the control unit 101 may extract the composition features based on the position, size, etc. of the subject. The control unit 101 may also determine the composition by cutting out the periphery of the subject.

[0091] In extracting color characteristics, the control unit 101 extracts a representative color from a captured image of the extraction target. The representative color is the most frequently occurring pixel value in the captured image of the target. The control unit 101 may extract a plurality of representative colors including the second most frequently occurring pixel value, the third most frequently occurring pixel value, and so on. This allows the control unit 101 to extract color characteristics.

[0092] Note that the control unit 101 may extract color characteristics by any method as long as the method can extract color characteristics. For example, the control unit 101 may extract color characteristics by weighting according to the appearance frequency of a plurality of representative colors.

[0093] In subject feature extraction, the control unit 101 uses an image recognition means to determine which of the predefined subject classifications the subject in the image belongs to. The predefined subject classifications are, for example, faces, animals, food, landscapes, etc.

[0094] The control unit 101 can extract the characteristics of the subject by using an image recognition technique. The control unit 101 may extract the characteristics of the subject by any method as long as the method can extract the characteristics of the subject. For example, the control unit 101 may extract the characteristics of the subject based on the shape, color, etc. of the subject. Features may be extracted.

[0095] The control unit 101 adds the image feature information extracted from the captured image as a tag to the captured image. The tag is added, for example, by adding the feature of the target image as metadata to the target image data.

[0096] Fig. 8 is a diagram illustrating tags (metadata) that indicate characteristic information of an image. Metadata is data that is added to captured image data. In the example of Fig. 8, the tag for the composition of the target image is "Japanese flag", the tag for the color is "#E3E9DF,#DA4011", and the tag for the subject is "food".

[0097] (Representative image setting process) The control unit 101 sets a representative image for the captured images to which tags have been added. The representative image setting process will be described below.

[0098] The control unit 101 sets a representative image for a group of images having the same tag value as a captured image newly added to the recording unit 108, for example. A representative image may be set for a group of images having one tag value that is the same, or may be set for a group of images having multiple tags that have the same values. Note that the tag values ​​are not limited to being the same; for example, an image group whose color characteristics are color codes of similar colors may be treated as an image group having the same tag value.

[0099] The representative image may be selected by any method as long as one image that is representative of a group of images with the same tag value can be selected. For example, the control unit 101 may select the representative image based on the shooting date and time of each image included in the group of images with the same tag value. Specifically, the control unit 101 may select the image with the most recent shooting date and time as the representative image from among the images included in the group of images with the same tag value.

[0100] The control unit 101 can also set a representative image for a combination of multiple tags. For example, for captured images having the tag information shown in Fig. 8, a representative image is set for an image group having a composition of "Japanese flag" and a subject of "food". By combining multiple tags, the control unit 101 can set a representative image for an image group with more specific conditions.

[0101] The information about the representative image is added to the image data as metadata, similar to the tag information. The information about the representative image includes the type of tag and the value of the tag that are common to the image group to which the representative image belongs.

[0102] The representative image may be set when a new captured image is added to the recording unit 108, or when a predetermined number of images are added. In addition, if a representative image has already been set for a group of images with the same tag value, the control unit 101 may not execute the process of setting the representative image.

[0103] (Photographing process according to the third embodiment) FIG. 9 is a flowchart illustrating the photographing process according to the third embodiment. FIG. 10 is a flowchart illustrating the photographed image arrangement process according to the third embodiment. The photographed image arrangement process is a subflow of the photographing process. The photographing process of the imaging device 100 in this embodiment will be described with reference to FIGS. 9 and 10. The photographing process shown in FIG. 9 is similar to the photographing process shown in FIG. 3 with respect to the processes other than step S901. The same processes as those in FIG. 3 are denoted by the same reference numerals and will not be described. The photographed image arrangement process shown in FIG. 10 illustrates the detailed process of step S901.

[0104] In step S1001, the control unit 101 accepts a selection operation of a representative image from the user. A UI for accepting the selection of a representative image from the user will be described with reference to Fig. 11. Fig. 11(A) shows a state in which a plurality of representative images are displayed in a display area 1102 of the display unit 109 as selection candidates. In the example of Fig. 11(A), the display area 1102 displays five representative images 1101.

[0105] The control unit 101 accepts the selection of a representative image by a user's operation on the display area 1102. For example, the user can select a desired representative image by a touch operation or a drag-and-drop operation. The control unit 101 places the selected representative image in any rectangular area of ​​the display position template 401. The placement position of the selected representative image may be a position designated by the user, for example, a rectangular area to which the user has moved the representative image by a drag-and-drop operation.

[0106] When a desired representative image is not displayed in the display area 1102, the user can switch candidates for the representative image displayed in the display area 1102. For example, the control unit 101 receives an instruction to display another representative image by a user's slide operation on the display area 1102. When the control unit 101 receives an instruction from the user, it switches the representative image displayed in the display area 1102 to the other representative image. In this way, even when the desired representative image is not displayed, the user can display the desired representative image by switching the display of the display area 1102.

[0107] Note that the control unit 101 may switch the display in the display area 1102 by accepting an operation by the user to specify a tag type (composition, color, subject) of an image that the user prioritizes. In this case, the control unit 101 displays a UI for accepting the tag type specification on the display unit 109. When the user specifies a tag type, the control unit 101 can display a representative image for the specified tag type in the display area 1102. This allows the control unit 101 to present to the user captured images that match the tags (image features) that the user prioritizes in step S1004.

[0108] In step S1002, the control unit 101 determines whether or not the user has selected a representative image. If a representative image has been selected (step S1002: Yes), the process proceeds to step S1003. If a representative image has not been selected (step S1002: No), the process proceeds to step S1005. If a representative image has not been selected, for example, this is the case when the user has not performed an operation to select a representative image for a predetermined period of time.

[0109] When a representative image is selected, the selected representative image is placed in, for example, a rectangular area 1105 as shown in Fig. 11(B). When the user performs a drag-and-drop operation to move the representative image to any rectangular area of ​​the template 401, the control unit 101 places the selected representative image in the rectangular area designated by the user.

[0110] In step S1003, the control unit 101 acquires a related photographed image from the recording unit 108 based on the representative image selected by the user. The related photographed image is an image related to the representative image selected by the user, and is an image presented to the user as a candidate for a photographed image to be arranged in the template 401. The related photographed image can be a photographed image having the same tag value as the representative image.

[0111] When acquiring a related captured image, the control unit 101 may accept an operation by the user to specify the type of tag (composition, color, subject) of the image to be prioritized. In this case, the control unit 101 displays a UI for accepting the tag type specification on the display unit 109. When the user selects multiple tags, the control unit 101 acquires related captured images having the same tag value selected by the user. This allows the control unit 101 to acquire, from the recording unit 108, related captured images according to the tag type prioritized by the user with priority.

[0112] In step S1004, the control unit 101 displays the related captured images acquired in step S1003 on the display unit 109 as candidates for captured images to be arranged on the template 401.

[0113] Fig. 11(B) shows an example of the display on display unit 109 that has changed from the state shown in Fig. 11(A) by the processing in steps S1002 to S1004. In the example of Fig. 11(B), the representative image selected in step S1002 has been placed in the center left area, which is one of the rectangles in template 401, by the user's drag-and-drop operation.

[0114] The control unit 101 displays the related photographed images 1103 acquired in step S1003 in the display area 1104 as candidates for photographed images to be arranged in the template 401. In the example of Fig. 11(B), the control unit 101 displays five related photographed images 1103 as candidates. Note that the control unit 101 excludes the related photographed images that have already been arranged in the rectangular area of ​​the template 401 from the candidates for the display area 1104, and displays another related photographed image that has not yet been arranged in the display area 1104.

[0115] When a desired related photographed image is not displayed in the display area 1104, the user can switch the related photographed image displayed in the display area 1104. For example, the control unit 101 receives an instruction to display another related photographed image by a slide operation of the user on the display area 1104. When the control unit 101 receives an instruction from the user, it switches the related photographed image displayed in the display area 1104 to another related photographed image. In this way, even when a desired related photographed image is not displayed, the user can display the desired related photographed image by switching the display of the display area 1104.

[0116] If the number of related photographed images acquired in step S1003 is less than the number that can be displayed in the display area 1104, the control unit 101 may additionally display other photographed images. The other photographed images to be added are, for example, photographed images having the same value in any of the tags of the representative image, or photographed images having a closer shooting date / time or shooting location. This allows the user to easily select other photographed images even if the number of related photographed images is less than the number that can be displayed in the display area 1104. After the processing of step S1004 is completed, the processing proceeds to step S1006.

[0117] In step S1005, since the user has not selected a representative image, the control unit 101 displays the captured images as candidates for the captured images to be arranged in the template 401. The candidates for the captured images in step S1005 may be displayed in any order of priority. For example, the control unit 101 may display the captured images in order of the shooting date and time. Note that the control unit 101 excludes the captured images that have already been arranged in the rectangular area of ​​the template 401 from the candidate display, and displays the unarranged captured images in the display area 1104.

[0118] In step S1006, the control unit 101 accepts a user's instruction to arrange the (related) captured image displayed in the display area 1104 in steps S1004 and S1005 in the template 401. The process of arranging the selected (related) captured image in the template 401 is similar to step S304 in the first embodiment.

[0119] Each time a (related) captured image is placed on the template 401, the control unit 101 The (related) photographed image arranged in the template 401 is excluded from the candidate display in the display area 1104. When one (related) photographed image is arranged in the template 401, the control unit 101 displays an additional (related) photographed image in the display area 1104.

[0120] In step S1007, the control unit 101 determines whether or not a command to end the arrangement of the captured images has been issued from the user. If an end command has been issued (step S1007: Yes), the captured image arrangement process shown in Fig. 10 ends and proceeds to step S305 in Fig. 9. If an end command has not been issued (step S1007: No), the process returns to step S1001.

[0121] If a representative image has already been selected and arranged in the template 401, the control unit 101 may return to step S1003 instead of step S1001 to continue the process.

[0122] In the third embodiment, the control unit 101 displays candidates for captured images to be arranged on the template 401 based on the representative image selected by the user. This enables the imaging device 100 to reduce the user's effort in selecting captured images to compare with a live image.

[0123] In step S1006, the control unit 101 may switch, in response to a user's instruction, whether to display captured images related to the representative image or captured images under other conditions as candidates to be displayed in the display area 1104. Captured images under other conditions are, for example, images captured during a certain period of time, images captured at a specific location, all captured images, etc. In this way, even if a captured image desired by the user is not displayed as a candidate, the control unit 101 can present captured images under other conditions to the user.

[0124] Furthermore, when the control unit 101 places the related photographed image selected by the user in step S1006 on the template 401, the control unit 101 may update candidates for related photographed images to be displayed in the display area 1104. For example, the control unit 101 finds tags whose values ​​are common between the tag of the representative image selected by the user in step S1002 and the tag of the related photographed image selected by the user in step S1006. The control unit 101 preferentially displays photographed images having the same values ​​as these common tags as new related photographed images in the display area 1104. By updating the candidates to be displayed in the display area 1104 based on the related photographed image selected by the user, the control unit 101 can easily display related photographed images desired by the user as candidates.

[0125] In addition, the related captured images are not limited to captured images with the same tag value, but may be captured images with tag values ​​in a similar range. The similar range of tag values ​​can be determined in advance. For example, in the composition tag, a three-division (vertical) composition and a two-division (vertical) composition can be defined as being in a similar range. In the color tag, color codes of similar colors can be defined as being in a similar range.

[0126] <Embodiment 4> In the third embodiment, in order to reduce the user's effort in selecting a captured image to compare with a live image, related captured images related to a representative image selected by the user are displayed as candidates for captured images on the display unit 109. In contrast, in the fourth embodiment, candidates for captured images are acquired from the recording unit 108 based on acquisition conditions for captured images selected (specified) by the user, and are displayed as candidates on the display unit 109.

[0127] In the fourth embodiment, the user specifies the acquisition conditions for the captured image candidates, and the captured image candidates are narrowed down based on the image features (tags) related to the acquisition conditions. Since the candidates according to the specified acquisition conditions are displayed, the user can reduce the effort of selecting the captured image. It is possible to reduce the amount of data transfer that can be performed by the image capturing apparatus according to the fourth embodiment. The configuration of the image capturing apparatus according to the fourth embodiment is similar to that of the image capturing apparatus 100 according to the first embodiment shown in Fig. 1. Furthermore, in the fourth embodiment, a tag addition process is executed in the same manner as in the third embodiment. A description of the process similar to that in the third embodiment will be omitted.

[0128] (Photographing process according to the fourth embodiment) The photographing process of the imaging device 100 in this embodiment will be described with reference to Fig. 9 and Fig. 12. The photographing process of the imaging device 100 according to the fourth embodiment is similar to the photographing process according to the third embodiment except for step S901 in Fig. 9.

[0129] Fig. 12 is a flowchart illustrating a captured image arrangement process according to the fourth embodiment. The captured image arrangement process is a subflow of the capture process. The captured image arrangement process illustrated in Fig. 12 illustrates detailed processing of step S901. Among the steps in Fig. 12, the same processes as those in Fig. 10 are denoted by the same reference numerals and description thereof will be omitted.

[0130] In step S1201, the control unit 101 accepts an operation from a user to specify acquisition conditions for a captured image. The acquisition conditions for a captured image include, for example, image composition conditions, image color conditions, and image subject conditions.

[0131] With reference to Fig. 13, a UI displayed on the display unit 109 for accepting designation of acquisition conditions for a captured image from the user will be described. Fig. 13(A) shows a UI for allowing the user to designate each item of the acquisition conditions for a captured image. A selection box 1301 is a UI for designating composition conditions. In response to a tap operation on the selection box 1301 by the user, the control unit 101 displays selection items for composition conditions shown in Fig. 13(B). The control unit 101 accepts the selection of the composition conditions by the user's tap operation.

[0132] A select box 1302 is a UI for specifying color conditions. In response to a tap operation by the user, the control unit 101 displays selection items for color conditions shown in Fig. 13(C). The control unit 101 accepts the selection of the color conditions by the user's tap operation.

[0133] A selection box 1303 is a UI for specifying the conditions of the subject. In response to a tap operation by the user, the control unit 101 displays selection items of the conditions of the subject shown in Fig. 13(D). The control unit 101 accepts the selection of the conditions of the subject by the tap operation by the user.

[0134] In this way, the control unit 101 can receive an instruction from a user to specify the acquisition conditions of a captured image. Note that the acquisition conditions of a captured image, the UI for specifying the acquisition conditions, and the operation method by the user are not limited to the above examples. An instruction for the acquisition conditions of a captured image may be received from the user by any method.

[0135] For example, the composition condition in Fig. 13(B) may be selected from an illustration representing the composition, rather than the name of the composition. Also, the color condition in Fig. 13(C) may be selected from a color sample, rather than specifying RGB. Also, the subject condition in Fig. 13(D) may be selected from an illustration representing the subject, rather than the name of the subject.

[0136] In step S1202, the control unit 101 determines whether or not the user has specified the acquisition conditions for the captured images. If the acquisition conditions for the captured images have been specified (step S1202: Yes), the process proceeds to step S1203. If the acquisition conditions for the captured images have not been specified (step S1202: No), the process proceeds to step S1005. If the acquisition conditions for the captured images have not been specified, the user may, for example, select the acquisition conditions for the captured images for a predetermined period of time. This occurs when no operation is performed to specify acquisition conditions.

[0137] In step S1203, the control unit 101 acquires related photographed images from the recording unit 108 based on the acquisition conditions of the photographed images specified by the user. The control unit 101 acquires, as related photographed images, photographed images to which a tag with the same value as the acquisition conditions of the photographed images specified by the user is added. The processes from step S1004 to step S1007 are the same as those in FIG. 10.

[0138] In the fourth embodiment, the control unit 101 displays candidates for captured images to be arranged on the template 401 based on the captured image acquisition conditions specified by the user. This enables the imaging device 100 to reduce the user's effort in selecting a captured image to compare with a live image.

[0139] <Embodiment 5> In the third and fourth embodiments, the representative image selected by the user and the related captured images related to the acquisition conditions of the captured images specified by the user are displayed as candidates for captured images on the display unit 109. In contrast, in the fifth embodiment, in order to reduce the user's effort in selecting captured images to compare with a live image, candidates for captured images are acquired from the recording unit 108 based on the characteristics of the live image and displayed as candidates on the display unit 109.

[0140] Since candidates for captured images are narrowed down based on the characteristics of the live image, the user can reduce the effort required to select a captured image similar to the live image. The configuration of the imaging device according to the fifth embodiment is similar to that of the imaging device 100 according to the first embodiment shown in FIG. In the fifth embodiment, a tag addition process is executed similarly to the third embodiment. A description of the process similar to that of the third embodiment will be omitted.

[0141] (Photographing process according to the fifth embodiment) The photographing process of the imaging device 100 in this embodiment will be described with reference to Fig. 9 and Fig. 14. The photographing process of the imaging device 100 according to the fifth embodiment is similar to the photographing process according to the third embodiment except for step S901 in Fig. 9. Fig. 14 is a flow chart illustrating the photographed image arrangement process according to the fifth embodiment. The photographed image arrangement process is a subflow of the photographing process. The photographed image arrangement process shown in Fig. 14 illustrates the detailed process of step S901.

[0142] In step S1401, the control unit 101 receives an instruction from the user to update the candidate display of the captured image. The display unit 109 displays a UI such as an update button for receiving an instruction from the user to update the candidate display of the captured image. The control unit 101 receives the instruction to update the candidate display of the captured image by the user's operation of tapping the update button.

[0143] In step S1402, the control unit 101 determines whether or not a user has instructed to update the candidate display of captured images. If a user has instructed to update (step S1402: Yes), the process proceeds to step S1403. If a user has not instructed to update (step S1402: No), the process proceeds to step S1005.

[0144] In step S1403, the control unit 101 extracts features of the live image. The method of extracting features may be the same as the method of extracting features in the tag addition process described in embodiment 3. The control unit 101 can obtain information on features related to the composition, color, subject, etc. of the live image by the same method of extracting features in the tag addition process.

[0145] Although the live image changes over time, the control unit 101 may analyze the image characteristics using the live image at the timing when the user issues an instruction to update the candidate display of captured images in step S1402.

[0146] In step S1404, the control unit 101 acquires, based on the features of the live image, related captured images from the recording unit 108. The control unit 101 acquires, as related captured images, captured images to which a tag having the same value as the features of the live image extracted in step S1403 is added.

[0147] In the fifth embodiment, the control unit 101 displays candidates for captured images to be arranged on the template 401 based on a live image. This enables the imaging device 100 to reduce the user's effort in selecting a captured image to compare with the live image.

[0148] In the description of FIG. 14, the control unit 101 uses the live image at the timing when the user instructs to update the candidate display of captured images for feature extraction, but this is not limited to the above. The control unit 101 may use the live image at the timing when a predetermined condition is satisfied for feature extraction. The timing when the predetermined condition is satisfied is, for example, the timing when the amount of change in the live image over time is equal to or less than a threshold. This allows the imaging device 100 to reduce the effort required by the user to instruct to update the candidate display of captured images. In addition, features of the live image are more easily extracted at the timing when the user decides on the composition to be captured.

[0149] <Embodiment 6> In the third embodiment, the related photographed images related to the representative image selected by the user are displayed as candidates for the photographed images on the display unit 109. In contrast, in the sixth embodiment, in order to further reduce the user's effort in arranging the photographed images, the related photographed images related to the representative image selected by the user are automatically arranged in the area within the template 401.

[0150] In the sixth embodiment, related captured images are automatically arranged in an area within the template 401, thereby reducing the user's effort in selection and arrangement. The configuration of the imaging device according to the sixth embodiment is similar to that of the imaging device 100 according to the first embodiment shown in FIG. In the sixth embodiment, tag addition processing and representative image setting processing are executed in the same manner as in the third embodiment. A description of the processing similar to that in the third embodiment will be omitted.

[0151] (Photographing process according to the sixth embodiment) The shooting process of the imaging device 100 in this embodiment will be described with reference to Figs. 9 and 15. The shooting process of the imaging device 100 according to the sixth embodiment is similar to the shooting process according to the third embodiment, except for step S901 in Fig. 9. Fig. 15 is a flowchart illustrating the shot image arrangement process according to the sixth embodiment. The shot image arrangement process is a subflow of the shooting process. The shot image arrangement process shown in Fig. 15 illustrates the detailed process of step S901. Among the steps in Fig. 15, the same processes as those in Fig. 10 are denoted by the same reference numerals, and the description thereof will be omitted.

[0152] In step S1501, the control unit 101 arranges the related photographed images acquired in step S1003 in each area in the template 401. Specifically, the control unit 101 acquires, from the related photographed images, a number of related photographed images sufficient to fill the empty areas in the template 401, and arranges them in the empty areas of the template 401, respectively.

[0153] For example, in the example screen of FIG. 4C, when there are three areas in which no captured images are placed, the control unit 101 obtains three related captured images from the recording unit 108 and places them in the empty areas.

[0154] The control unit 101 performs control so as not to automatically replace a live image already arranged by the user with a related captured image. The control unit 101 arranges the related captured image in an area of ​​the template 401 where there is no live image arranged by the user.

[0155] Furthermore, the control unit 101 may perform control so as not to automatically replace a photographed image that has been arranged by a user with a related photographed image. The control unit 101 manages a flag for determining whether an arranged photographed image has been arranged by a user or has been automatically arranged by the control unit 101 through the processing of step S1501. For example, the control unit 101 adds a flag of 0 to an arranged photographed image if it has been arranged by a user's instruction in step S1006, and adds a flag of 1 to an arranged photographed image if it has been arranged by the control unit 101 in step S1501. The control unit 101 can perform control so as not to replace a photographed image that has been arranged by a user (flag=0) with a related photographed image by referring to the value of the flag.

[0156] Furthermore, the control unit 101 may change the display form of the arranged captured image depending on whether the arranged captured image was arranged by the user or arranged by the control unit 101 in step S1501. By changing the display form, the user can easily distinguish whether the arranged captured image was arranged by the user or arranged automatically by the imaging device 100. The control unit 101 can change the display form by, for example, the color of the frame of the rectangular area of ​​the template 401.

[0157] Furthermore, when the number of related captured images is smaller than the number of images that can fill each area in the template 401, the control unit 101 may arrange only the related captured images acquired in step S1003. In this case, the control unit 101 may arrange the related captured images by giving priority to areas that are closer to the live image. By arranging the related captured images from areas that are closer to the live image, the user can easily compare the live image and the captured images.

[0158] In the sixth embodiment, the control unit 101 automatically arranges related captured images acquired based on a representative image selected by the user in the template 401. This enables the imaging device 100 to reduce the user's effort in selecting and arranging captured images to be compared with a live image.

[0159] The related photographed image arranged in the area of ​​the template 401 by the control unit 101 in step S1501 may be replaced with another photographed image by a user operation in step S1006. This allows the user to replace the related photographed image arranged by the control unit 101 with a photographed image desired by the user when the related photographed image arranged by the control unit 101 is not the desired image.

[0160] <Embodiment 7> When displaying a list of images, arranging images having the same characteristics can give the list of images a sense of unity. In addition, the list of images may also have a sense of unity by regularly arranging images having a first characteristic and images having a second characteristic.

[0161] The sixth embodiment is an embodiment in which related captured images having the same characteristics as a representative image selected by the user are automatically arranged in an area within the template 401. In contrast, the seventh embodiment is an embodiment in which a plurality of groups of captured images having different characteristics are automatically and regularly arranged.

[0162] In the seventh embodiment, multiple groups of captured images having different characteristics can be automatically arranged in a regular manner. This allows the user to realize a list display with a variety of variations with a simple operation. The configuration of the imaging device according to the seventh embodiment is the same as that of the imaging device 100 according to the first embodiment shown in FIG. 1. In addition, the same processes as those in the sixth embodiment are performed in the same manner. The same reference numerals are used and detailed explanations are omitted.

[0163] (Photographing process according to the seventh embodiment) 9 and 15, the photographing process of the image capturing device 100 in this embodiment will be described. Differences from the photographing process in the sixth embodiment will be specifically described.

[0164] 9, the control unit 101 accepts a user's instruction to select a template capable of specifying the display positions of captured images of a group having a first characteristic and the display positions of captured images of a group having a second characteristic. Hereinafter, the group having the first characteristic will be referred to as the first group, and the group having the second characteristic will be referred to as the second group.

[0165] An example of a template for the display position of a captured image will be described with reference to Fig. 16. Fig. 16(A) and 16(B) are examples of templates for the display position selectable by the user. A number of areas included in the template are divided into a number of groups and arranged in a regular pattern.

[0166] 16(A) and 16(B), rectangular area 1601 (similarly including a filled rectangular area) is a rectangular area in which a first group of images are arranged among the rectangular areas indicating the display position of template 401. Moreover, rectangular area 1602 (similarly including a shaded rectangular area) is a rectangular area in which a second group of images are arranged among the rectangular areas indicating the display position of template 401.

[0167] In Fig. 16(A), rectangular regions 1601 and 1602 are arranged in a regular checkerboard pattern, while in Fig. 16(B), rectangular regions 1601 and 1602 are arranged in a regular vertical stripe pattern.

[0168] The control unit 101 displays a template of the display position as shown in Fig. 16 on the display unit 109 and presents it to the user. The control unit 101 accepts the selection of the template from the user. The control unit 101 may make the display form of the rectangular area 1601, which is the display position of the first group of images, different from the display form of the rectangular area 1602, which is the display position of the second group of images. This allows the user to easily determine which group each rectangular area belongs to.

[0169] In step S1001 of Fig. 15, the control unit 101 accepts the selection of a representative image by the user. The user can move the desired representative image to any of the rectangular areas of the template 401, for example, by a drag-and-drop operation. The control unit 101 detects the user's operation and places the representative image selected by the user in the area specified by the drag-and-drop operation. This allows the control unit 101 to accept the selection and placement of the representative image by the user.

[0170] 15, the control unit 101 determines whether or not the user has selected a representative image. If a representative image has been selected (step S1002: Yes), the control unit 101 records in the recording unit 108 which feature group the rectangular area in which the representative image is located belongs to, and the process proceeds to step S1003. If a representative image has not been selected (step S1002: No), the process proceeds to step S1005.

[0171] In step S1501 in FIG. 15, the control unit 101 automatically arranges the related captured image acquired in step S1003 in a rectangular area of ​​the same feature group as the feature group of the rectangular area in which the representative image is arranged.

[0172] An example of a live view display screen on which related captured images are arranged will be described with reference to FIG. 17A shows an example in which a template having the display position shown in FIG. 16A is selected in step S302. In step S303, the live image is arranged in the rectangular area 402 in the center of the template 401. In step S1001, the representative image 1701 is arranged in the rectangle in the upper left corner of the template 401.

[0173] In step S1501, the control unit 101 reads out from the recording unit 108 which feature group the rectangular area in which the representative image 1701 is arranged belongs to. In the example of Fig. 17(A), the rectangular area in which the representative image 1701 is arranged belongs to group 1. The control unit 101 arranges the related photographed image acquired in step S1003 in a rectangular area at another display position that belongs to group 1, which is the same feature group as the representative image.

[0174] Fig. 17(B) illustrates a state in which the control unit 101 arranges a related captured image in a rectangular area at another display position belonging to the first group, from the state of Fig. 17(A). Note that the control unit 101 controls so that the user does not replace the arranged representative image and live image with the related captured image.

[0175] 17(C) shows an example in which the processes from step S1001 to step S1501 are repeated again from the state of FIG 17(B), and the representative image and related captured images are also arranged in the rectangular areas of the second group. In this way, the control unit 101 can configure a live view display screen in which the image features are unified for each feature group.

[0176] In the seventh embodiment, the control unit 101 can easily arrange a plurality of groups of captured images having different characteristics in the template 401. This enables the imaging device 100 to reduce the user's effort in regularly arranging captured images of a plurality of characteristic groups to be compared with a live image.

[0177] The control unit 101 may determine the condition of the characteristics of the image to be arranged in the rectangular area of ​​the second group based on the characteristics of the image of the first group. The condition of the image to be arranged in the rectangular area of ​​the second group may be, for example, a condition that the subject is smaller than the composition of the first group, or a condition that the color is compatible with the color of the first group.

[0178] Furthermore, the control unit 101 can select captured images belonging to each feature group based on tag information related to the features of the image. This allows the imaging device 100 to reduce the user's effort in regularly arranging captured images of multiple feature groups for comparison with a live image.

[0179] <Embodiment 8> In the eighth embodiment, the size of the display area (each area in the template) of the captured image and the live image is adjusted to make it easier to check the details of the live image, such as the focus or angle of view. The user can check the details of the live image, and adjust the composition, brightness, and color of the live image to match the captured image, and then capture it.

[0180] 1, the configuration of the imaging device according to the eighth embodiment includes an acceleration sensor that measures the speed of the imaging device 100. Descriptions of operations and processes similar to those of the first embodiment will be omitted, and characteristic processes of the eighth embodiment will be described with reference to Figs.

[0181] (Photographing process according to the eighth embodiment) The photographing process of the imaging device 100 in the eighth embodiment will be described with reference to FIG. 18. The photographing process according to the eighth embodiment is performed in accordance with the flowchart shown in FIG. 3 except for step S1801. Since the process is the same as that of the first embodiment, the same processes are denoted by the same reference numerals and the description thereof is omitted.

[0182] In step S1801, the control unit 101 accepts an operation from the user to adjust the shooting conditions so that the arranged captured image and the live image have the same composition, brightness, and color tone. The control unit 101 also accepts an operation for the user to check the details of the live image, such as the focus and angle of view.

[0183] The shooting conditions are image processing conditions for changing the composition, exposure conditions, color, gradation, etc. As an example of adjusting the shooting conditions, a method of adjusting the composition will be described with reference to Figures 5(A) and 5(B).

[0184] 5(A) and 5(B) show examples of a live view display screen in which eight captured images are selected and arranged in step S304. In addition, in Fig. 5(A) and Fig. 5(B), a zoom bar 501 is a UI for changing the zoom magnification of a live image, and a capture button 502 is a UI for capturing images by pressing or touching the button.

[0185] 5A, the bread in the live image arranged in the rectangular area 402 is displayed smaller than the bread in the surrounding captured images. If a photograph is taken in this state, the subject size of the live image and the captured images will differ, resulting in an uncoordinated and undesirable display of the list of these images.

[0186] In order to match the subject size of the live image to the subject size of the captured image, the user increases the zoom magnification of the live image. In the example of the live view display screen in Fig. 5(B), in order to increase the subject size of the live image, the user can increase the zoom magnification by moving the knob of the zoom bar 501 in the + direction. Through the zoom operation, the subject size of the live image, which was smaller than the surrounding captured images in Fig. 5(A), is enlarged to the same size as the surrounding captured images, as in Fig. 5(B).

[0187] By changing the zoom magnification, the subject size in the multiple images becomes uniform, so that the list display of the multiple images displayed on the live view display screen after shooting becomes a list display with a uniform look.

[0188] A method of checking whether an image is in focus will be described as an example of checking the details of a live image with reference to Fig. 19(A) to Fig. 19(C). Fig. 19(A) shows an example of a live view display screen in which a live image is placed in the center and eight captured images are placed around it. In Fig. 19(A), the display areas for the live image and the captured images are the same size. Having the display areas for the live image and the captured images the same size is suitable for adjustments that create a sense of unity.

[0189] However, with the size of the divided areas, it is difficult to check whether the live image is in focus, and the user may end up taking an image that is out of focus. In order to make it easy to check whether the image is in focus, the control unit 101, for example, accepts a pinch-out operation from the user and enlarges the rectangular area 402 that displays the live image as shown on the live view display screens in Fig. 19(B) and Fig. 19(C). This makes it easy to check whether the live image is in focus.

[0190] After checking the focus, the user pinches in. The control unit 101 accepts the pinch in from the user and returns the size of the rectangular area 402 displaying the live image to its original size, as shown on the live view display screen in FIG. 19(A). By changing the display size of the live image in response to the user's operation, the user can easily check the focus. do.

[0191] 19(B) and 19(C), a method for checking the angle of view will be described. For example, when the user moves the image capture device 100 at a speed equal to or greater than a predetermined threshold, the control unit 101 determines that the user is checking the rough angle of view of the live image. The speed of the image capture device 100 can be measured by an acceleration sensor included in the image capture device 100.

[0192] When the control unit 101 determines that the user is checking the angle of view of the live image, the control unit 101 automatically enlarges the rectangular area 402 displaying the live image, as shown in Fig. 19(B) and (C). The user can easily check the angle of view by changing the display size of the live image.

[0193] When the speed of the imaging device 100 falls below a predetermined threshold, the control unit 101 determines that the user has finished checking the angle of view. When the control unit 101 determines that the user has finished checking the angle of view, the control unit 101 automatically restores the size of the rectangular area 402 displaying the live image to its original size, as shown on the live view display screen in Fig. 19(A) . By changing the size of the rectangular area 402 displaying the live image according to the speed of the imaging device 100, the user can easily check the rough angle of view.

[0194] In the eighth embodiment, in step S1801, in order to allow the user to check the details of the live image, the control unit 101 adjusts the size of the rectangular area 402 that displays the live image. This allows the user to easily check the details of the live image, such as the focus and the rough angle of view.

[0195] In addition, when the size of the rectangular area 402 displaying the live image is increased, the number and arrangement of the captured images are the same as those in FIG. 19(A) in the live view display screens of FIG. 19(B) and FIG. 19(C), but are not limited to the same. For example, in the example of FIG. 19(D), the live image is enlarged and some of the captured images are displayed in a reduced size and arranged below the live image. In the example of FIG. 19(E), the live image is enlarged and displayed in the center and the captured images are displayed in a reduced size and arranged above and below the live image. In the example of FIG. 19(F), the live image is enlarged and displayed in the center and the captured images are not displayed. As exemplified in FIG. 19(D) to FIG. 19(F), the number and arrangement of the captured images may be freely changed according to the size of the live image.

[0196] <Other embodiments> The first to eighth embodiments (including modifications) are merely examples, and the present invention includes configurations obtained by appropriately modifying or changing the configurations of the respective embodiments within the scope of the gist of the present invention. In addition, the present invention also includes configurations obtained by appropriately combining the configurations of the respective embodiments.

[0197] For example, the first to eighth embodiments can be combined with the second embodiment. In this case, the imaging device 100 displays a live image and a captured image acquired from an external server such as an SNS on the live view display screen during shooting.

[0198] Specifically, when the eighth embodiment is combined with the second embodiment, the imaging device 100 displays a captured image acquired from an external server on the display unit 109 together with a live image. The imaging device 100 adjusts the size of the area displaying the live image, allowing the user to check the details of the live image. The user can adjust the composition, brightness, and color of the live image to match the captured image from the external server, and then capture the image.

[0199] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0200] 100: imaging device, 101: control unit, 105: imaging unit, 108: recording unit, 109: display unit

Claims

1. a display means for displaying a live image of a subject and a plurality of captured images on a display unit; a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject; An imaging means for imaging the subject; a recording means for recording, in a recording section, a photographed image of the subject photographed by the photographing means and the photographed image in association with layout information including a display position of the photographed image of the subject and a display position of the photographed image; an uploading means for uploading the photographed image of the subject and the photographed image together with the arrangement information recorded in the recording unit to an external server; having The display means displays a live image of the subject captured using the shooting conditions changed by the change means together with the plurality of captured images.

1. An imaging device comprising:

2. The display device further includes a determination unit for determining a display position of each of the live image of the subject and the captured image.

2. The imaging device according to claim 1 .

3. The display unit adjusts a live image of the subject in accordance with the changed photographing conditions and displays the image on the display unit.

2. The imaging device according to claim 1 .

4. The photographing conditions include at least one of a focal length condition, a subject distance condition, a subject depth condition, and an exposure condition.

4. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

5. The display means displays the photographed image of the subject and the already-photographed image on the display unit based on the arrangement information.

5. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

6. The display position of the photographed image of the subject and the display position of the photographed image are areas selected from a plurality of areas that are divided in advance.

6. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

7. The captured image is an image selected from the images acquired from the recording unit.

7. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

8. a display means for displaying a live image of a subject and a plurality of captured images on a display unit; a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject; An imaging means for imaging the subject; a recording means for recording the photographed image of the subject photographed by the photographing means and the already-photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the already-photographed image; having The display means displays a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images, The captured image is an image selected from images acquired from an external server.

1. An imaging device comprising:

9. The display device further includes a transmitting means for transmitting, to the external server, layout information including a display position of the photographed image of the subject and a display position of the photographed image.

9. The imaging device according to claim 8.

10. The display unit displays the photographed image of the subject and the already-photographed image on the display unit based on the arrangement information acquired from the external server.

10. The imaging device according to claim 9.

11. a display means for displaying a live image of a subject and a plurality of captured images on a display unit; a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject; An imaging means for imaging the subject; a recording means for recording the photographed image of the subject photographed by the photographing means and the already-photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the already-photographed image; having The display means includes: displaying a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images; The display unit displays related captured images acquired based on the characteristics of the representative image selected by the user as candidates for the captured images.

1. An imaging device comprising:

12. a display means for displaying a live image of a subject and a plurality of captured images on a display unit; a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject; An imaging means for imaging the subject; a recording means for recording the photographed image of the subject photographed by the photographing means and the already-photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the already-photographed image; having The display means includes: displaying a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images; The display unit displays related captured images acquired based on acquisition conditions designated by the user as candidates for the captured images.

1. An imaging device comprising:

13. a display means for displaying a live image of a subject and a plurality of captured images on a display unit; a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject; An imaging means for imaging the subject; a recording means for recording the photographed image of the subject photographed by the photographing means and the already-photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the already-photographed image; having The display means includes: displaying a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images; A related captured image obtained based on the characteristics of the live image of the subject is displayed on the display unit as a candidate for the captured image.

1. An imaging device comprising:

14. The display means displays on the display unit a template divided into a plurality of areas for displaying the live image of the subject and the photographed image, and automatically arranges the photographed image selected from the related photographed images in an area among the plurality of areas in which the live image of the subject and the photographed image are not arranged.

14. The imaging device according to claim 11,

15. The regions of the template are divided into groups; The display means automatically arranges the captured image selected from the related captured images acquired for each group in an area of ​​the corresponding group.

15. The imaging device according to claim 14.

16. An imaging device, comprising: a display means for displaying a live image of a subject and a plurality of captured images on a display unit; a change unit for changing a photographing condition of the subject based on the plurality of photographed images displayed on the display unit together with a live image of the subject; An imaging means for imaging the subject; a recording means for recording the photographed image of the subject photographed by the photographing means and the already-photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the already-photographed image; having The display means includes: displaying a live image of the subject captured using the photographing conditions changed by the change means together with the plurality of photographed images; Changing the display size of the live image of the object based on a user operation or the speed of the imaging device 1. An imaging device comprising:

17. displaying a live image of the subject and a plurality of captured images on a display unit; changing a photographing condition of the object based on the plurality of photographed images displayed on the display unit together with a live image of the object; displaying a live image of the object captured using the changed shooting conditions together with the plurality of captured images; taking an image of the subject; a step of recording the photographed image of the subject and the already-photographed image in a recording unit in association with layout information including a display position of the photographed image of the subject and a display position of the already-photographed image; uploading the photographed image of the subject and the photographed image together with the arrangement information recorded in the recording unit to an external server; 13. A method for controlling an imaging apparatus comprising:

18. A program for causing a computer to function as each of the means of the imaging apparatus according to any one of claims 1 to 16.

Images