Imaging device, control method, and program

Abstract

To assist photographers in taking appropriate photos. [Solution] The imaging device 100 according to this disclosure detects feature points in the image obtained by imaging. The imaging device 100 also determines the type of feature point to be aligned with the guide based on the positional relationship between the feature point detected in the first image and the guide, and outputs guidance information for guiding the position of the feature point corresponding to the type to be aligned with the guide based on the positional relationship between the feature point corresponding to the type detected in the second image and the guide.

Description

Technical Field

[0001] The present disclosure relates to a technique for guide display and shooting guidance in a photographing apparatus.

Background Art

[0002] As an aid when taking a photograph with a photographing apparatus, there is known a technique for displaying a guide individually set by a user of the photographing apparatus (hereinafter referred to as a "photographer") on a display device such as a liquid crystal display of the photographing apparatus. For example, Patent Document 1 discloses a technique for customizing the display mode of a guide by changing the width of a grid displayed on a display device of a photographing apparatus in conjunction with an operation by a photographer.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technique disclosed in Patent Document 1, although the photographing apparatus displays a guide individually set by the photographer, it is impossible to specify for which position of the photographing object (hereinafter referred to as a "subject") the displayed guide is intended. Therefore, the conventional technique has a problem in that the photographing apparatus cannot output information for guiding the alignment of the guide and the position relationship of the subject. As a result, appropriate photographing could not be performed.

Means for Solving the Problems

[0005] The imaging device according to this disclosure includes: a feature point detection means for detecting feature points in an image obtained by imaging; an object determination means for determining the type of feature point to be aligned with the guide based on the positional relationship between the feature point detected in the first image and the guide; and a guidance output means for outputting guidance information for guiding the position of the feature point corresponding to the type to be aligned with the position of the guide based on the positional relationship between the feature point corresponding to the type detected in the second image and the guide. [Effects of the Invention]

[0006] According to this disclosure, it is possible to provide support to photographers in order to take pictures appropriately. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram illustrating a use case related to Embodiment 1. [Figure 2] This figure shows an example of a workflow in a use case according to Embodiment 1. [Figure 3] Figure 3 shows an example of a display screen shown on the display device of the imaging device according to Embodiment 1. [Figure 4] This figure shows an example of a method for outputting guidance information according to Embodiment 1. [Figure 5] This is a block diagram showing an example of the hardware configuration of the imaging device according to Embodiment 1. [Figure 6] This is a block diagram showing an example of the functional configuration of the imaging device according to Embodiment 1. [Figure 7] This figure shows an example of feature point information related to Embodiment 1. [Figure 8] This figure shows an example of guide information related to Embodiment 1. [Figure 9] This flowchart shows an example of the processing flow in the imaging device according to Embodiment 1. [Figure 10] This figure shows an example of a shooting mode selection screen according to Embodiment 1. [Figure 11] FIG. 1 is a diagram showing an example of a guide selection screen for selecting a basic shape of a guide according to Embodiment 1 and an adjustment screen for adjusting a display position of the guide. [Figure 12] FIG. 4 is a flowchart showing an example of a detailed processing flow of target determination processing in the imaging apparatus according to Embodiment 1. [Figure 13] FIG. 7 is a diagram for explaining an example of target determination processing in the target determination unit according to Embodiment 1. [Figure 14] FIG. 10 is a diagram showing an example of a selection screen for target candidates according to Embodiment 1. [Figure 15] FIG. 13 is a flowchart showing an example of a detailed processing flow of guidance output processing in the imaging apparatus according to Embodiment 1. [Figure 16] FIG. 16 is a diagram showing an example of a workflow in a use case according to Embodiment 2. [Figure 17] FIG. 19 is a diagram showing an example of an additional proposal screen according to Embodiment 2. [Figure 18] FIG. 22 is a block diagram showing an example of a functional configuration of the imaging apparatus according to Embodiment 2. [Figure 19] FIG. 25 is a flowchart showing an example of a processing flow in the imaging apparatus according to Embodiment 2. [Figure 20] FIG. 28 is a flowchart showing an example of a flow of additional proposal processing in the imaging apparatus according to Embodiment 2. [Figure 21] FIG. 31 is a diagram showing an example of feature point information that is a determination target of feature points in the recommendation generation unit according to Embodiment 2. [Figure 22] FIG. 34 is a diagram showing an example when feature points according to Embodiment 2 are plotted on the same plane. [Figure 23] FIG. 37 is a diagram showing an example of a guide generated by the recommendation generation unit according to Embodiment 2.

MODE FOR CARRYING OUT THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples, and the technical scope of the present disclosure is not limited only to those configurations.

[0009] [Embodiment 1] At the shooting site of school photos or the like, each of many people may be photographed with a similar composition. In such a case, it is required to shorten the shooting time per person. In Embodiment 1, a use case in such a case will be described as an example.

[0010] FIG. 1 is a diagram for explaining a use case according to Embodiment 1. The photographer 101 operates the photographing device 100 to sequentially photograph each of the persons 102 to 104 to be photographed. Specifically, the photographer 101 causes each of the persons 102 to 104 to take the same pose and then takes the photograph. The photographing device 100 is a device having at least a photographing function and a live view function, such as a digital still camera, a digital video camera, or a smartphone.

[0011] FIG. 2 is a diagram showing an example of a workflow in the use case according to Embodiment 1. Each step in the following workflow will be described using a symbol starting with "S". First, in S201, the photographer 101 operates the photographing device 100 to create a guide. The guide is a frame line or the like displayed on the display device provided in the photographing device 100 to assist the photographer 101 when the photographer 101 determines the shooting composition. In the present embodiment, it is described that the photographer 101 creates the guide by operating the photographing device 100, but the guide may be created in an information device such as a PC (personal computer). In this case, for example, the photographer 101 causes the photographing device 100 to read information about the guide created using the information device.

[0012] Next, in S202, the imaging device 100 stores the information regarding the guide created by the photographer 101 in S201 in a memory device or the like provided in the imaging device 100. Then, in S203, the imaging device 100 reads the information regarding the guide stored in S202 and displays the guide indicated by that information on a display device provided in the imaging device 100.

[0013] Figure 3 shows an example of a display screen shown on the display device of the imaging device 100 according to Embodiment 1. Specifically, Figure 3(a) shows an example of the guide display mode. The dashed lines 301 and 302 each represent a guide, and the guide is represented by lines, frames, or a combination thereof. By displaying the guide, the imaging device 100 assists the photographer 101 in photographing each of the people 102 to 104 with a similar composition. Figure 3(b) will be described later.

[0014] After S203, in S204, the photographer 101 takes a still image or other image of the first person, for example, person 102, with the guide displayed on the display device of the camera 100. Next, in S205, the camera 100 saves the data of the image obtained by taking the image in S204 (hereinafter referred to as the "captured image") to a storage device or the like provided by the camera 100. Next, in S206, the camera 100 performs the following detection process based on the position of the guide and the position of the feature points in the image 300 of the first person (person 102), which are obtained by analyzing the captured image obtained by taking the image of the first person. Specifically, in the detection process based on the position of the guide and the position of the feature points in the image 300 of person 102, the camera 100 identifies which feature points the guides indicated by dashed lines 301 and 302 are intended to align with.

[0015] Figure 3(b) shows an example of the location of feature points extracted from the captured image obtained by shooting in S204. In Figure 3(b), each of the gray circles 303 to 307 indicates the location of a feature point in the image 300 of the first person (person 102). Specifically, circle 303 represents the location of the feature point extracted as the position corresponding to the right elbow of the first person in the captured image. Similarly, circles 304 to 307 represent the locations of feature points extracted as the positions corresponding to the right eye, left eye, nose, and mouth of the first person in the captured image, respectively.

[0016] In Figure 3(b), the linear guide indicated by the dashed line 301 intersects with the position of the feature point corresponding to the right elbow of the first person in the captured image. In this case, for example, the imaging device 100 identifies the guide indicated by the dashed line 301 as a guide for aligning the image of the right elbow during imaging. Also in Figure 3(b), the frame-shaped guide indicated by the dashed line 302 contains feature points corresponding to the right eye, left eye, nose, and mouth of the first person in the captured image. In this case, for example, the imaging device 100 identifies the guide indicated by the dashed line 302 as a guide for aligning the images of the right eye, left eye, nose, and mouth during imaging.

[0017] After S206, in S207, the camera 100 stores information indicating a specific result from S206 in a memory device or the like. Next, in S208, the photographer 101 adjusts the composition of the next person while checking the live view with the guide displayed on the display device of the camera 100 in order to take the next shot.

[0018] Next, in S209, the imaging device 100 determines the alignment status between the position of the displayed guide and the position of the person's image in the live view. Specifically, for example, in S207, the imaging device 100 uses information indicating a specific result from S206 to determine whether the positional relationship between the guide and the person's image is appropriate. In the example shown in Figure 3(b), the imaging device 100 determines whether the image of the person's right elbow overlaps with the guide indicated by the dashed line 301. The imaging device 100 also determines whether the images of the right eye, left eye, nose, and mouth are within the guide indicated by the dashed line 302. If the determination in S209 determines that the position of the person's image is inappropriate relative to the guide, i.e., misaligned, then in S210, the imaging device 100 outputs information regarding guidance for aligning the position of the person's image (hereinafter referred to as "guidance information").

[0019] Figure 4 shows an example of a method for outputting guidance information according to Embodiment 1. For example, the imaging device 100 outputs guidance information by superimposing a GUI (Graphical User Interface) corresponding to the guidance information onto the live view display when the guide is displayed. In Figure 4, circles 401 and 402 indicate the positions of feature points in the image 400 of person 103 that are not in the correct position relative to the guide indicated by the dashed line 301 or 302. Arrows 403 and 404 indicate, respectively, the direction in which the position of the feature points in the image 400 of person 103, indicated by circles 401 and 402, should be moved, and are an example of a method for outputting guidance information. For example, arrows 403 and 404 are represented by line segments that start at the current position of the feature point to be moved and end at the position to which the feature point should be moved to align with the guide. Furthermore, if, for example, a feature point indicated by circle 402 should be included within the guide of the frame shape indicated by dashed line 302, the endpoint of arrow 404 may be set to the center or approximately the center of the said frame shape.

[0020] The photographer 101 checks the content of the guidance information output in S210 and repeatedly performs the process in S208 to change at least one of the position and posture of the person and the position and posture of the shooting device 100 to achieve an appropriate shooting composition. After the adjustment of the shooting composition is complete, in S211 the photographer 101 takes a still image or other image. Next, in S212 the shooting device 100 saves the image data obtained from the shooting in S211 to a storage device or the like provided by the shooting device 100. Next, in S213 the photographer 101 checks whether the shooting of all the people to be photographed has been completed. If the shooting of at least some of the people to be photographed has not been completed, the processes from S208 to S213 are repeatedly executed until the shooting of all the people to be photographed has been completed. When the shooting of all the people to be photographed has been completed, the workflow shown in Figure 2 is completed.

[0021] Figure 5 is a block diagram showing an example of the hardware configuration of the imaging device 100 according to Embodiment 1. The imaging device 100 has a hardware configuration of CPU 501, ROM 502, RAM 503, storage device 504, imaging unit 505, power supply unit 506, display device 507, and input unit 508. The above hardware configuration is connected to each other via a bus so as to be able to communicate with each other. The CPU 501 is a control unit consisting of at least one processor or circuit, and controls the entire imaging device 100. The ROM 502 is an electrically erasable and recordable memory, and stores information related to the guide created in S201, as well as data and programs used in calculations by the CPU 501. The program referred to here is a computer program for executing various flowcharts, which will be described later. The RAM 503 operates as a work area for the CPU 501, and stores constants and variables used in calculations by the CPU 501, as well as programs read by the CPU 501 from the ROM 502.

[0022] The memory device 504 is a recording medium composed of semiconductor memory such as a memory card, and records the data of the captured image obtained by shooting. Information regarding the guide created in S201 may also be stored in the memory device 504. The imaging unit 505 is an image sensor composed of elements such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) that converts optical images into electrical signals. Furthermore, the imaging unit 505 has multiple microlenses on its sensor surface, enabling distance measurement of the subject using phase difference.

[0023] The power supply unit 506 supplies power to the imaging device 100. The power supply unit 506 is composed of, for example, a built-in battery such as a lithium-ion battery, or a removable portable battery. The configuration of the power supply unit 506 is not limited to this, and may also be composed of, for example, an AC / DC converter, in which case the power supplied from the AC power source may be converted to DC power and supplied to the imaging device 100. The display device 507 is composed of an LCD monitor or an electronic viewfinder, and displays a guide, live view, or various GUIs. The input unit 508 is composed of a touch panel or button switches, and receives operation input from the photographer 101.

[0024] Figure 6 is a block diagram showing an example of the functional configuration of the imaging device 100 according to Embodiment 1. The imaging device 100 has the following functional configurations: an image acquisition unit 601, a feature point detection unit 602, an operation acquisition unit 605, an imaging mode determination unit 606, a guide change unit 607, a guide acquisition unit 608, and a guide output unit 609. In addition to the above functional configurations, the imaging device 100 also has a target determination unit 610, a target output unit 611, a target determination unit 612, a guidance output unit 613, and a display control unit 614. The processing of each functional configuration shown in Figure 6 is realized by the CPU 501 loading programs stored in ROM 502, etc., into RAM 503 and executing them.

[0025] The image acquisition unit 601 acquires the captured image obtained by the imaging unit 505. The feature point detection unit 602 detects feature points contained in the captured image acquired by the image acquisition unit 601. For example, the feature point detection unit 602 acquires the estimated skeletal position result obtained by inputting the captured image into a trained skeletal estimation model obtained as a result of learning by deep learning or the like as the detection result of feature points contained in the captured image. The feature point detection process of the feature point detection unit 602 is not limited to the method described above, and the position of the image of an object in the captured image detected by object detection processing or the like may be used as a feature point. Feature point information indicating the feature point detection result of the feature point detection unit 602 is stored in, for example, the storage device 504.

[0026] Figure 7 shows an example of feature point information according to Embodiment 1. The feature point information shown in Figure 7 includes the following item columns: shooting ID 701, feature point type 702, location 703, and shooting date and time 704. The shooting ID 701 stores an ID (identifier) ​​as an item value to uniquely identify the shooting. The ID is information that can uniquely identify which shooting image was used to detect the feature points. A unique ID is assigned to each shooting. As an example, in the case of live view shooting, "LV" + "a 3-digit arbitrary number" is assigned to each shooting, and in the case of still image shooting, "IMG" + "a 3-digit arbitrary number" is assigned to each shooting.

[0027] The item value for Feature Point Type 702 stores information indicating the type of feature point detected in the captured image. As an example, it is explained below that the name of a predetermined body part obtained as an estimation result using a skeletal estimation model is stored as information indicating the type of feature point. The item value for Position 703 stores information indicating the position of each feature point in the captured image, obtained as an estimation result using a skeletal estimation model. As an example, it is explained below that the position of a feature point in a captured image is represented using a coordinate system with two axes, vertical (Y-axis) and horizontal (X-axis), with the upper left corner of the captured image as the origin. The item value for Capture Date and Time 704 stores information indicating the capture date and time of the captured image in which the feature points were detected.

[0028] Figure 8 is a diagram showing an example of guide information according to Embodiment 1. The guide information created by the photographer 101 in S201 will be explained with reference to Figure 8. The guide information shown in Figure 8 includes the item columns for shooting mode 801, type 802, start point 803, end point 804, and alignment target 805. Shooting mode 801 stores information about the shooting mode in which the guide is used as an item value. The shooting mode is set by the photographer 101, who sets the shooting mode by switching shooting modes according to the composition or scene to be photographed. When the shooting mode is changed by the photographer 101, the guide displayed on the display device 507 is changed. Type 802 stores information about the type of guide shape as an item value. Examples of guide shape types include "line" and "frame".

[0029] The start point 803 and end point 804 store information as item values ​​regarding the display position of the guide on the display screen when the opening degree is displayed on the display device 507. For example, if the item value of type 802 is "line", a line segment connecting the start point and the end point is displayed as a guide on the display device 507. Also, if the item value of type 802 is "frame", the start point represents the upper left position of the rectangle displayed as a guide on the display screen of the display device 507, and the end point represents the lower right position of the rectangle.

[0030] The alignment target 805 stores information as an item value indicating the name of the part corresponding to the feature point of the object to be aligned with each guide. The item values ​​stored in the item columns for shooting mode 801, type 802, start point 803, and end point 804 are entered when the photographer 101 creates them in S201. On the other hand, the item values ​​stored in the item column for alignment target 805 are entered during the saving process in S207. In addition, the alignment target 805 stores information indicating the type of feature point, such as the name of a predetermined part obtained as an estimation result using the skeletal estimation model, similar to the item value stored in the feature point type 702 shown in Figure 7.

[0031] The operation acquisition unit 605 acquires operation information corresponding to the operation from the photographer 101 received by the input unit 508. The shooting mode determination unit 606 determines the shooting mode by selecting the shooting mode set by the photographer 101 based on the operation information acquired by the operation acquisition unit 605. The guide acquisition unit 608 acquires information about the guide corresponding to the shooting mode determined by the shooting mode determination unit 606 from among the guide information corresponding to each shooting mode. The guide output unit 609 generates the guide indicated by the guide information acquired by the guide acquisition unit 608 as, for example, a GUI component, and outputs the generated GUI component to the display control unit 614.

[0032] The target determination unit 610 executes the process S206 in the workflow shown in Figure 2. Specifically, the target determination unit 610 determines the type of feature point of the target to be aligned with each guide based on the positional relationship between the feature point indicated by the feature point information detected by the feature point detection unit 602 and the guide acquired by the guide acquisition unit 608. The guide modification unit 607 modifies the guide information corresponding to the shooting mode determined by the shooting mode determination unit 606 based on the type of feature point of the target to be aligned determined by the target determination unit 610. The target output unit 611 generates a GUI component that shows the position of the target feature point determined by the target determination unit 610 and outputs the generated GUI component to the display control unit 614.

[0033] The target determination unit 612 executes the process at S209 in the workflow shown in Figure 2. Specifically, the target determination unit 612 determines whether the positional relationship between the guide indicated by the guide information acquired by the guide acquisition unit 608 and the feature points indicated by the feature point information, which is the detection result by the feature point detection unit 602, is appropriate. Details of the determination process in the target determination unit 612 will be described later. The guidance output unit 613 executes the process at S208 in the workflow shown in Figure 2. Specifically, if the target determination unit 612 determines that the positional relationship between the guide and the feature points is not appropriate, the guidance output unit 613 generates guidance information and outputs the generated guidance information to the display control unit 614. The display control unit 614 controls the output for displaying the GUI components related to the guide, GUI components indicating the position of the target feature points, and guidance information generated by the guide output unit 609, target determination unit 610, or guidance output unit 613 on the display device 507.

[0034] Figure 9 is a flowchart showing an example of the processing flow in the imaging device 100 according to Embodiment 1. Each processing step in the flowchart shown below will be explained using a symbol beginning with "S". Each processing step in the flowchart shown below is realized by the CPU 501 loading a program stored in the ROM 502, etc., into the RAM 503 and executing it. First, at S901, the shooting mode determination unit 606 determines the shooting mode by selecting the shooting mode set by the photographer 101 based on the operation information acquired by the operation acquisition unit 605.

[0035] Figure 10 shows an example of a shooting mode selection screen 1000 according to Embodiment 1. The shooting mode selection screen 1000 is displayed, for example, on a display device 507 and is displayed by control of the display control unit 614 based on an operation by the photographer 101. The photographer 101 selects one of several items displayed on the shooting mode selection screen 1000 and then presses the OK button 1001. When the OK button 1001 is pressed, the operation acquisition unit 605 acquires operation information corresponding to the item selected by the photographer 101. Subsequently, the shooting mode determination unit 606 determines the shooting mode based on the operation information acquired by the operation acquisition unit 605. Hereinafter, it will be explained that the items that the photographer 101 can select as a shooting mode on the shooting mode selection screen 1000 are "Portrait," "Group Photo," "User Setting (Existing)," and "User Setting (New)."

[0036] "Portrait" and "Group Photo" are shooting modes in which the guides pre-set on the camera device 100 at the factory default are displayed in the live view display during shooting. "User-defined (New) mode" is a shooting mode selected when the photographer 101 wants to create their own guide. When the OK button 1001 is pressed while "User-defined (New)" is selected, the photographer 101 creates the guide described later, and information about the newly created guide by the photographer 101 is added to the guide information. "User-defined (Existing) mode" is a shooting mode in which the guides indicated by information about guides already created in "User-defined (New) mode" and already saved in the guide information are displayed in the live view display during shooting. If there is no information about a user-created guide in the guide information, the "User-defined (Existing)" item may be disabled or hidden on the shooting mode selection screen 1000.

[0037] Following S901, in S902, the guide acquisition unit 608 determines whether the shooting mode determined in S901 is "User Setting (New)". If it is determined in S902 that it is not "User Setting (New)", the shooting device 100 executes the process in S906, which will be described later. If it is determined in S902 that it is "User Setting (New)", in S903, the guide acquisition unit 608 acquires information about the basic shape guide to be used to create a user-specific guide. Specifically, for example, the photographer 101 selects a desired basic shape guide from among several basic shape guides, and the guide acquisition unit 608 acquires information about the basic shape guide selected by the photographer 101.

[0038] After S903, in S904, the guide modification unit 607 changes the display position of the basic shape guide selected by the photographer 101 in S903. Specifically, for example, the basic shape guide selected in S903 is displayed on the display device 507, and the photographer 101 adjusts the display position of the guide while checking the displayed guide. The guide modification unit 607 changes the guide information based on the adjustment of the guide display position by the photographer 101. After S904, in S905, the guide modification unit 607 adds the information about the modified guide to the guide information and saves it.

[0039] Figure 11 shows an example of a guide selection screen 1100 for selecting the basic shape of a guide according to Embodiment 1, and adjustment screens 1110 and 1120 for adjusting the display position of the guide. Specifically, Figure 11(a) shows an example of the guide selection screen 1100. Figures 11(b) and (c) will be described later. The guide selection screen 1100, as an example, has a guide display area 1101, a mode display area 1102, selection buttons 1103 and 1104, and a start button 1105.

[0040] The guide display area 1101 displays the basic shape of the guide that has been pre-held by the imaging device 100. The guide display area 1101 has an aspect ratio that corresponds to, for example, the display screen of the display device 507, the image sensor of the imaging unit 505, or the aspect ratio of the captured image. The mode display area 1102 displays the names of the shooting modes that can use the basic shape displayed in the guide display area 1101.

[0041] The selection buttons 1103 and 1104 are buttons that change the basic shape of the guide displayed in the guide display area 1101. When the photographer 101 presses either the selection button 1103 or 1104, the basic shape of the guide displayed in the guide display area 1101 switches. In this case, the name of the shooting mode displayed in the mode display area 1102 also switches to match the basic shape of the guide displayed in the guide display area 1101. For example, when the selection button 1104 is pressed, the basic shape of the guide displayed in the guide display area 1101 switches to the one for "group photos," and accordingly, the name of the shooting mode displayed in the mode display area 1102 switches from "portrait" to "group photo." The start button 1105 is pressed after the selection of the basic shape of the guide is complete. When the start button 1105 is pressed, the display screen shown on the display device 507 transitions from the guide selection screen 1100 to the adjustment screen 1110.

[0042] Figures 11(b) and (c) show examples of adjustment screens 1110 and 1120. Specifically, the adjustment screen 1110 shown in Figure 11(b) displays the guide 1111 of the basic shape before adjustment of the display position, which was acquired in S903. The adjustment screen 1120 shown in Figure 11(c) displays the guide 1121 after adjustment of the display position in S904. A confirmation button 1112 is also displayed on the adjustment screens 1110 and 1120. The photographer 101 selects a guide displayed on the adjustment screen 1110, etc., by touching the touch panel or pressing a button switch, etc., and adjusts its position.

[0043] The confirmation button 1112 is used to save information about the guide after adjusting the display position. When the confirmation button 1112 is pressed, the shape of the guide according to the user settings is finalized, and the information about the finalized guide is saved in the guide information. If information about a guide created by the photographer 101 already exists in the guide information, the imaging device 100 may, for example, delete that information from the guide information and then add the information about the newly created guide to the guide information. At the time when the information about the newly created guide is saved, the type of feature point of the target to be aligned with the guide has not yet been finalized. Therefore, the item value for the alignment target 805 in the guide information is not entered at the time the new guide is created and saved.

[0044] Following S905, in S906, the guide acquisition unit 608 acquires information about the guide corresponding to the shooting mode determined in S901 by reading it from the guide information. If the shooting mode was determined to be "User Setting (New)" in S902, the guide acquisition unit 608 acquires information about the guide stored in S905 by reading it from the guide information. Next, in S907, the guide is displayed on the display device 507 based on the guide information acquired in S906.

[0045] Specifically, first, the guide output unit 609 generates a GUI component of the guide based on the guide information acquired in S906, and outputs the generated GUI component to the display control unit 614. Subsequently, the display control unit 614 displays the guide on the display device 507 by controlling the output of the GUI component related to the guide generated by the guide output unit 609. The display control unit 614 also controls the output of the live view to the display device 507, and displays the guide superimposed on the live view on the display device 507.

[0046] After S907, in S908, the target determination unit 612 determines whether the item value of the alignment target 805 is included in the guide information acquired in S906. If it is determined in S908 that the item value of the alignment target 805 is not included, the imaging device 100 executes the process in S909. If it is determined in S908 that the item value is included, the imaging device 100 executes the process in S910. In this embodiment, it is assumed that the item value of the alignment target 805 corresponding to an imaging mode other than "User setting (new)" has been entered, but it is not limited to this. For example, the item value of the alignment target 805 corresponding to all imaging modes may not have been entered, and the type of feature point of the target to be aligned with the guide may be determined in the process in S907 described later.

[0047] In S909, the imaging device 100 performs a determination process (hereinafter referred to as the "target determination process") to determine the type of feature point of the target to be aligned with the guide displayed in S907. Details of the target determination process in S909 will be described later. After S909, the imaging device 100 performs the process in S910. In S910, the imaging device 100 performs a guidance information output process (hereinafter referred to as the "guidance output process") as shown in Figure 4, according to the positional relationship between the guide displayed in S907 and the image of the person who is the subject. Details of the guidance output process in S910 will be described later.

[0048] Figure 12 is a flowchart showing a detailed example of the processing flow of the target determination process in the imaging device 100 according to Embodiment 1, that is, the process at S909 in the flowchart shown in Figure 9. First, at S1201, the image acquisition unit 601 acquires the data of the captured image obtained by the photographer 101 taking a photograph. Specifically, first, the photographer 101 changes at least one of the position and posture of the person and the position and posture of the imaging device 100 to obtain an appropriate shooting composition, referring to the guide displayed at S907. Next, when an appropriate shooting composition is achieved, the photographer 101 takes a photograph of the person by pressing the shutter button, etc. Then, at S1202, the image acquisition unit 601 saves the data of the captured image acquired at S1201 to the storage device 504, etc.

[0049] Next, in S1203, the feature point detection unit 602 detects feature points included in the captured image acquired in S1201 and saves the detection results as feature point information. In S1204, the target determination unit 610 selects an arbitrary guide from the one or more guides displayed in S907. Next, in S1205, the target determination unit 610 determines whether the type of guide selected in S1204 is a "line". If it is determined in S1205 that the type is a "line", the imaging device 100 executes the process in S1206. If it is determined that the type is not a "line" (in this embodiment, it is determined to be a "frame"), the imaging device 100 executes the process in S1207. In S1206, the target determination unit 610 determines the feature points that exist on the linear guide (including those nearly on the guide) selected in S1204 from among the feature points detected in S1203 as candidates for the feature points of the target to be aligned (hereinafter referred to as "target feature points"). Hereinafter, the candidate for the target feature point will be referred to as the "target candidate." In S1207, the target determination unit 610 determines the feature points detected in S1203 that are located within the frame of the frame shape guide selected in S1204 as target candidates.

[0050] Figure 13 is a diagram illustrating an example of the target determination process in the target determination unit 610 according to Embodiment 1. Specifically, Figure 13(a) is a diagram illustrating an example of the target candidate determination process when the guide type is "line". Figure 13(b) will be described later. Image 1301 is a captured image acquired in S1201 with the guide selected in S1204 and the feature points detected in S1203 superimposed. The dashed line 1302 is the guide selected in S1204. The circle 1303 is a feature point corresponding to the top of the head. In the example of Figure 13(a), the feature point of the top of the head, indicated by the circle 1303, is located on the guide indicated by the dashed line 1302. Therefore, the target determination unit 610 determines that the type of target candidate to be aligned with the guide indicated by the dashed line 1302 is the top of the head.

[0051] Figure 13(b) is a diagram illustrating an example of the process for determining target feature points when the guide type is "frame". Image 1311 is a captured image acquired in S1201 with the guide selected in S1204 and the feature points detected in S1203 superimposed. The dashed line 1312 is the guide selected in S1204. Circle 1313 is the feature point corresponding to the top of the head. Circle 1314 is the feature point corresponding to the right eye. Circle 1315 is the feature point corresponding to the left eye. Circle 1316 is the feature point corresponding to the nose. Circle 1317 is the feature point corresponding to the mouth. Circle 1318 is the feature point corresponding to the right shoulder. Circle 1319 is the feature point corresponding to the left shoulder. In the example of Figure 13(b), the feature points corresponding to circles 1313 to 1319 are included within the guide frame indicated by the dashed line 1312. Therefore, the target determination unit 610 determines that the types of target candidates to be aligned with the guide indicated by the dashed line 1312 are the top of the head, the right eye, the left eye, the nose, the mouth, the right shoulder, and the left shoulder.

[0052] After S1206 or S1207, in S1208, the target determination unit 610 determines a target feature point from among the one or more target candidates determined in S1206 or S1207, based on the operation from the photographer 101. Specifically, the imaging device 100 displays a selection screen for selecting a target candidate on the display device 507 via the target output unit 611 and the display control unit 64. The photographer 101 uses the input unit 508 to select one or more target candidates desired from among the one or more target candidates displayed on the selection screen on the display device 507. The target determination unit 610 determines a target feature point based on the operation information regarding the selection of a target candidate by the photographer 101, which is acquired via the operation acquisition unit 605.

[0053] Figure 14 shows an example of a target candidate selection screen according to Embodiment 1. The selection screen 1400 includes a guide display area 1401, a candidate selection area 1403, and a confirmation button 1402. The guide display area 1401 is an area that displays the guide selected in S1204. The candidate selection area 1403 displays a list of target candidates for the guide displayed in the guide display area 1401 as a list of checkboxes. The photographer 101 selects the target feature points from among the one or more target candidates displayed in the guide display area 1401 by checking the ones they want to be target feature points and unchecking the ones they do not want to be target feature points.

[0054] After selecting a target candidate to be used as a target feature point, photographer 101 presses the confirm button 1402. When photographer 101 presses the confirm button 1402, the target determination unit 610 saves information about the type of target feature point, using the target candidate checked on the selection screen 1400 as the target feature point. Specifically, the target determination unit 610 saves information about the type of target candidate checked on the selection screen 1400 in the column for alignment target 805 in the information corresponding to the guide selected in S1204.

[0055] After S1208, in S1209, the target determination unit 610 determines whether all guides have been selected in S1204. If it is determined in S1209 that at least some guides have not been selected, the target determination unit 610 returns to S1204 and repeatedly executes the process from S1204 to S1209 until it is determined in S1209 that all guides have been selected. In this repeated process, in S1204, the target determination unit 610 selects any guide that has not yet been selected. If it is determined in S1209 that all guides have been selected, the imaging device 100 terminates the process shown in the flowchart in Figure 12, that is, the process shown in S909 in Figure 9.

[0056] Figure 15 is a flowchart showing a detailed example of the processing flow of the induction output processing in the imaging device 100 according to Embodiment 1, that is, the processing of S910 in the flowchart shown in Figure 9. First, in S1502, the imaging device 100 performs live view shooting, and the image acquisition unit 601 acquires the captured image obtained by live view shooting (hereinafter referred to as "live view image"). Next, in S1503, the feature point detection unit 602 detects feature points included in the live view image acquired in S1502 and saves the detection result as feature point information. Next, in S1504, the target determination unit 612 uses the value of the alignment target 805 column in the guide information acquired in S906 to identify a target feature point from among the one or more feature points detected in S1503.

[0057] Next, in S1505, the target determination unit 612 determines whether the positional relationship between the guide and the target feature point identified in S1504 is appropriate, based on the guide information acquired in S906. An appropriate positional relationship means that, if the guide type is a line, the target feature point is located on the guide (including partially on the guide). If the guide type is a frame, an appropriate positional relationship means that the target feature point is located within the frame of the guide. If it is determined in S1505 that the relationship is not appropriate, in S1506, the guidance output unit 613 generates and outputs guidance information indicating that the position of the target feature point is misaligned with the guide. The guidance information output from the guidance output unit 613 is notified to the photographer 101 by being displayed on the display device 507 via the display control unit 614. An example of the notification method is shown in Figure 4, so an explanation is omitted.

[0058] Subsequently, the photographer 101, while confirming the notification, changes the position or orientation of the shooting device 100, or the position or orientation of the subject, so that the position of the target feature point relative to the guide is appropriate. The shooting device 100 also repeatedly executes the processes from S1502 to S1506 until it is determined in S1505 that the state is appropriate. If it is determined in S1505 that the state is appropriate, the shooting device 100 terminates the process of the flowchart shown in Figure 15, i.e., the process of S910. In this embodiment, guidance information is displayed on the display device 507 only when the position of the target feature point is misaligned with the guide, i.e., when the position of the target feature point relative to the guide is inappropriate, but this is not limited to this. For example, the shooting device 100 may notify that the position of the target feature point relative to the guide is appropriate by changing the color of the guide, etc., when the position of the target feature point relative to the guide is appropriate.

[0059] Following S910, in S911, the image acquisition unit 601 acquires the data of the captured image obtained by the photographer 101. Specifically, at the end of S910, the positional relationship between the guide and the target feature point is in an appropriate state. Therefore, the photographer 101 can take the picture while the positional relationship between the guide and the target feature point is in an appropriate state. Next, in S912, the image acquisition unit 601 saves the data of the captured image acquired in S911 to the storage device 504 or the like.

[0060] Next, in S913, the operation acquisition unit 605 determines whether or not it has acquired operation information indicating the end of shooting. The photographer 101 performs the operation to end shooting, for example, when they have finished photographing all the people being photographed. If it is determined in S913 that operation information indicating the end of shooting has not been acquired, the shooting device 100 returns to the process in S910 and repeatedly executes the processes from S910 to S913 until it is determined in S913 that operation information indicating the end of shooting has been acquired. If it is determined in S913 that operation information indicating the end of shooting has been acquired, the shooting device 100 terminates the process shown in the flowchart in Figure 9.

[0061] With the shooting device 100 configured as described above, if there is a discrepancy between the positional relationship between the guide individually set by the photographer 101 and the subject, guidance information can be output to correct the discrepancy. Therefore, by shooting while displaying the guide, the photographer 101 can photograph multiple subjects with similar compositions. Furthermore, with the shooting device 100 configured as described above, the photographer 101 can individually set the type of target feature point to be aligned with the guide. Therefore, the photographer 101 can edit the guide to achieve the composition desired by the photographer 101. As a result, the shooting device 100 configured as described above can assist the photographer in taking appropriate photographs.

[0062] In this embodiment, the photographer 101 is described as creating guide information in the camera 100, but the embodiment is not limited to this. For example, the photographer 101 may create guide information using an information device such as a PC and have the camera 100 read the created guide information. In this embodiment, the guidance information is described as being output using arrows or the like that are visible to the photographer 101, but the guidance information may also be output using strings of characters or sentences. In this case, the guidance information may also be output in a manner that is audible to the photographer 101, such as by voice.

[0063] [Embodiment 2] Embodiment 1 described an embodiment in which the photographer 101 determines the positional relationship between all guides and target feature points by taking a photograph while displaying the guides, and creates information related to the guides. In the embodiment according to Embodiment 1, the photographer 101 needs to actively perform the work related to creating the guides. In this embodiment, an embodiment will be described in which the photographing device 100 suggests to the photographer 101 the addition of guides based on the captured image.

[0064] Figure 16 is a diagram showing an example of a workflow in a use case according to Embodiment 2. In the following description, parts that overlap with the description of the workflow shown in Figure 2 will be denoted by the same reference numerals and their description will be omitted. After S212, at S1601, the imaging device 100 displays an additional suggestion screen on the display device 507 that suggests adding information about the guide.

[0065] Figure 17 shows an example of an additional suggestion screen 1700 according to Embodiment 2. The additional suggestion screen 1700 includes, for example, a guide display area 1701 and buttons 1705 and 1706. The additional suggestion screen 1700 is displayed, for example, in S212 when multiple shots are taken with the same composition. In the guide display area 1701, the currently set guide and the new guide that is being suggested for addition are superimposed on the still image taken and saved in S212 in response to the shooting instruction from the photographer 101 in S211. As an example, the guide display area 1701 shown in Figure 17 shows the case where multiple shots are taken with the composition shown in Figure 3(a). In the guide display area 1701, the dashed lines 1702 and 1703 indicate the currently set guide. The solid line 1704 indicates the new guide that is being suggested for addition by the shooting device 100.

[0066] New guides are generated and displayed, for example, at positions where feature points of the same type are similar in position in images obtained from multiple past captures. The additional suggestion screen 1700 shown as an example in Figure 17 is for cases where feature points of the type "left hand" are present at similar positions in multiple captured images. In such cases, the shooting device 100 determines that the photographer is trying to take a picture so that the position of the left hand is at a predetermined position in the captured image, and generates and suggests a new guide to align the position of the left hand with that predetermined position. Details of the method for generating new guides will be described later.

[0067] Button 1705 is pressed when the photographer 101 approves adding a new guide. When button 1705 is pressed, the information about the new guide displayed in the guide display area 1701 is added as guide information. Button 1706 is pressed when the photographer 101 denies adding a new guide. When button 1706 is pressed, the camera 100 terminates the display of the additional suggestion screen 1700, for example, without adding the information about the new guide as guide information.

[0068] Figure 18 is a block diagram showing an example of the functional configuration of the imaging device 100 according to Embodiment 2. In the following description, components that overlap with the description of the functional configuration shown in Figure 6 are denoted by the same reference numerals, and their descriptions are omitted. Note that the processing of each functional configuration shown in Figure 18 is realized by the CPU 501 loading programs stored in ROM 502, etc., into RAM 503 and executing them.

[0069] The recommendation generation unit 1815 generates a new guide based on the feature point information indicating the detection results of feature points by the feature point detection unit 602 and the guide information acquired by the guide acquisition unit 608. Specifically, the recommendation generation unit 1815 first determines whether there are any feature points of the same type that exist at similar positions in captured images in multiple past shooting sessions, among the one or more feature points that are not targeted for alignment with the guide. If it is determined that there are feature points that exist at similar positions, the recommendation generation unit 1815 generates a new guide corresponding to the position of the feature point as an additional suggestion. The recommendation output unit 1816 generates a GUI component corresponding to the new guide generated by the recommendation generation unit 1815 based on the information about the new guide and outputs the GUI component to the display control unit 1814. The display control unit 1814 generates an additional suggestion screen 1700 by superimposing the GUI component corresponding to the new guide output from the recommendation output unit 1816 with the GUI component corresponding to the already set guide and the live view image, etc., and displays it on the display device 507.

[0070] Figure 19 is a flowchart showing an example of the processing flow in the imaging device 100 according to Embodiment 2. In the following description, parts that overlap with the description of the flowchart shown in Figure 9 will be denoted by the same reference numerals and their explanation will be omitted. After S912, the imaging device 100 performs the process of proposing the addition of a new guide at S1901. After S1901, the imaging device 100 performs the process at S913.

[0071] Figure 20 is a flowchart showing an example of the additional suggestion process in the imaging device 100 according to Embodiment 2, that is, the processing flow of S1901 in the flowchart shown in Figure 19. In S2001, the recommendation generation unit 1815 determines whether or not feature point information exists for each of the multiple images obtained in the past, for example, from multiple recent imaging sessions. If feature point information exists for each of the multiple images in S2001, the imaging device 100 executes the process in S2004. In S2004, the recommendation generation unit 1815 determines whether or not there are any feature points among the feature points corresponding to each of the multiple images that are not subject to alignment and that are of the same type as each other and are located in close proximity to each other.

[0072] Referring to Figures 21 and 22, a method for determining whether feature points are located in close proximity to each other will be described. As an example, a method for determining whether feature points are located in close proximity to each other will be described for feature points included in three still images obtained from the most recent three captures. Figure 21 is a diagram showing an example of feature point information that is subject to determination in the recommendation generation unit 1815 according to Embodiment 2. The feature point information shown in Figure 21 includes, as an example, information on feature points corresponding to the left hand included in each of the three still images obtained from the most recent captures.

[0073] Figure 22 shows an example of plotting feature points corresponding to the left hand, as indicated by the feature point information shown in Figure 21, on the same plane. Circle 2201 plots the positions of feature points corresponding to the left hand included in IMG001. Similarly, circles 2202 and 2203 plot the positions of feature points corresponding to the left hand included in IMG002 and IMG003, respectively. X_Diff is the maximum horizontal distance between the positions of feature points of the same type, i.e., feature points corresponding to the left hand in Figure 22. Similarly, Y_Diff is the maximum vertical distance between the positions of feature points corresponding to the left hand that are of the same type. For example, the recommendation generation unit 1815 determines that feature points are located in close proximity to each other if X_Diff is less than a predetermined horizontal threshold X_TH, and Y_Diff is less than a predetermined vertical threshold Y_TH, in at least one of these cases.

[0074] If, in S2004, it is determined that there are feature points that are not subject to alignment but share a common type and are located in close proximity to each other, the imaging device 100 executes the process in S2005. In S2005, the recommendation generation unit 1815 generates a new guide that is recommended to be added. Information regarding the guide generated by the recommendation generation unit 1815 is transmitted to the recommendation output unit 1816.

[0075] Referring to Figure 23, the method for generating guides in the recommendation generation unit 1815 will be described. Figure 23 is a diagram showing an example of the method for generating guides in the recommendation generation unit 1815 according to Embodiment 2. Specifically, Figure 23 is a diagram showing an example of a guide generated based on the feature point information shown in Figure 21. Circle 2301 is a plot of the average value of the position of the feature point corresponding to the left hand in IMG001, IMG002, and IMG003. Line segment 2302 is a guide generated based on the feature point information shown in Figure 23.

[0076] For example, the recommendation generation unit 1815 generates a line segment passing through the position of circle 2301 as a guide. Specifically, for example, if X_Diff is not less than X_TH and Y_Diff is less than Y_TH, the recommendation generation unit 1815 determines that the photographer 101 wants to align the vertical position of the corresponding feature point and generates a linear guide extending horizontally. Similarly, for example, if X_Diff is less than X_TH and Y_Diff is not less than Y_TH, the recommendation generation unit 1815 generates a linear guide extending vertically. Also, for example, if X_Diff is less than X_TH and Y_Diff is less than Y_TH, the recommendation generation unit 1815 generates two linear guides extending vertically and horizontally.

[0077] In the example shown in Figure 23, the case where Y_Diff is less than Y_TH, rather than X_Diff being less than X_TH, is represented by a horizontally extending line segment 2302, and a guide is generated. The length of the guide can be arbitrary. In the example shown in Figure 23, the length of the guide is set to the maximum distance between feature points in the horizontal direction. Specifically, for example, when the recommendation generation unit 1815 generates a guide with a horizontally extending linear shape, the length of the guide is set to X_Diff, and when generating a guide with a vertically extending linear shape, the length of the guide is set to Y_Diff.

[0078] After S2005, in S2006, the recommendation output unit 1816 displays an additional suggestion screen, including the guide generated in S2005, on the display device 507 via the display control unit 1814. An example of the additional suggestion screen is shown in Figure 17, so its explanation is omitted. After S2006, if the photographer 101 presses button 1705, the guide information generated by the recommendation generation unit 1815 is added to the guide information, and then the shooting device 100 terminates the processing of the flowchart shown in Figure 20. However, if the photographer 101 presses button 1706 after S2006, the shooting device 100 terminates the processing of the flowchart shown in Figure 20 without adding the guide information generated by the recommendation generation unit 1815 to the guide information.

[0079] With the imaging device 100 configured as described above, a new guide recommended for addition can be generated based on the captured image obtained through imaging, and this can be proposed to the photographer 101. By adding the proposed guide, the photographer 101 can more easily take pictures with the same composition in subsequent imaging sessions. As a result, the imaging device 100 configured as described above can assist the photographer in taking pictures appropriately. In the above description, it was explained that a guide recommended for addition is generated using multiple captured images obtained from multiple imaging sessions, but the guide recommended for addition may also be generated using a single captured image obtained from, for example, the most recent imaging session.

[0080] [Other embodiments] The technology of this disclosure can also be implemented by supplying a program that implements one or more of the functions of the embodiments described above to a system or device via a network or storage medium, and by a process in which one or more processors in the computer of that system or device read and execute the program. Furthermore, the technology of this disclosure can also be implemented by a circuit such as an ASIC that implements one or more functions.

[0081] Furthermore, within the scope of this disclosure, the technologies described herein allow for free combination of each embodiment, modification of any component of each embodiment, or omission of any component in each embodiment.

[0082] [Structure of this disclosure] This disclosure includes the following configurations, methods, and programs.

[0083] <Configuration 1> A feature point detection means for detecting feature points in a captured image obtained through photography, A target determination means that determines the type of target corresponding to the feature point of the target to be aligned with the guide, based on the positional relationship between the feature point detected in the first captured image and the guide, A guidance output means outputs guidance information for guiding the position of a feature point corresponding to a certain type to the position of the guide, based on the positional relationship between the feature point corresponding to the type detected in the second captured image and the guide, A photographic device characterized by having the following features.

[0084] <Configuration 2> The first captured image is obtained by taking a photograph in a state where the characteristic points corresponding to the type of object to be aligned with the guide have not yet been determined. The imaging device described in configuration 1, characterized by the above.

[0085] <Structure 3> The second captured image is obtained by taking a photograph in a state where the characteristic points corresponding to the type of object to be aligned with the guide have been determined. A photographic device according to configuration 1 or 2, characterized by the above.

[0086] <Structure 4> The shape of the guide is a linear shape, a frame-like shape, or a shape formed by a combination of lines and a frame. A photographic device according to any one of configurations 1 to 3 characterized by the above.

[0087] <Composition 5> The target determination means, when the shape of the guide is linear, determines the type corresponding to the feature point present on the guide as the type of the feature point of the target to be aligned with the guide. The imaging device described in configuration 4, characterized by the above.

[0088] <Composition 6> The target determination means, when the shape of the guide is frame-shaped, determines the type corresponding to the feature point located inside the frame corresponding to the guide as the type of the feature point of the target to be aligned with the guide. The imaging device according to configuration 4 or 5, characterized by the above.

[0089] <Composition 7> An operation acquisition means for acquiring operation information corresponding to user input, Guide changing means for changing at least one of the position of the guide and the type of the feature point of the target that is aligned with the guide, based on the operation information, Furthermore, to have, A photographic device according to any one of configurations 1 to 6 characterized by the above.

[0090] <Structure 8> The feature point detection means acquires skeletal information obtained as a result of skeletal estimation for the image of the object being captured in the captured image obtained by shooting, as feature point information in the captured image. A photographic device according to any one of configurations 1 to 7, characterized by the above.

[0091] <Composition 9> The feature point detection means acquires object information obtained as a result of object detection on the captured image obtained by shooting, as feature point information in the captured image. A photographic device according to any one of configurations 1 to 8 characterized by the above.

[0092] <Composition 10> Guide generation means that generates a new guide that the user can add based on the position of the feature points detected in the second captured image, Furthermore, to have, A photographic device according to any one of configurations 1 to 9 characterized by the above.

[0093] <Composition 11> The guide generation means generates a new guide when the same type of feature point is detected in multiple captured images obtained through multiple shooting sessions. The imaging device described in configuration 10, characterized by the above.

[0094] <Composition 12> The guide generation means generates a new guide in the shape of a horizontal line when the range of feature points of the same type among the feature points that are not to be aligned with the already set guide in each of the plurality of captured images is within the range of a vertical threshold, The imaging device described in configuration 11, characterized by the above.

[0095] <Composition 13> The guide generation means generates a new guide in the shape of a vertical line when the range of feature points of the same type among the feature points that are not to be aligned with the already set guide in each of the plurality of captured images is within the range of a horizontal threshold, The imaging device according to configuration 11 or 12, characterized by the above.

[0096] <Method> A feature point detection step for detecting feature points in the captured image obtained by shooting, A target determination step, which determines the type of target corresponding to the feature point of the target to be aligned with the guide, based on the positional relationship between the feature point detected in the first captured image and the guide, A guidance output step that outputs guidance information for aligning the position of the feature point corresponding to the type with the position of the guide, based on the positional relationship between the feature point corresponding to the type detected in the second captured image and the guide, A method for controlling a photographic device, characterized by including the following:

[0097] <Program> A program for causing a computer to function as an imaging device as described in any one of configurations 1 to 13. [Explanation of symbols]

[0098] 100 Imaging device 602 Feature Point Detection Unit 610 Target Determination Section 613 Induction Output Section

Claims

1. A feature point detection means for detecting feature points in a captured image obtained through photography, A target determination means that determines the type of target corresponding to the feature point of the target to be aligned with the guide, based on the positional relationship between the feature point detected in the first captured image and the guide, A guidance output means outputs guidance information for aligning the position of a feature point corresponding to a certain type with the position of the guide, based on the positional relationship between the feature point corresponding to the type detected in the second captured image and the guide. A photographic device characterized by having the following features.

2. The first captured image is obtained by taking a photograph in a state where the characteristic points corresponding to the type of object to be aligned with the guide have not yet been determined. The imaging apparatus according to claim 1, characterized by the following:

3. The second captured image is obtained by taking a photograph in a state in which the characteristic points corresponding to the type of object to be aligned with the guide have been determined. The imaging apparatus according to claim 1, characterized by the following:

4. The shape of the guide is a linear shape, a frame-like shape, or a shape formed by a combination of lines and a frame. The imaging apparatus according to claim 1, characterized by the following:

5. The target determination means, when the shape of the guide is linear, determines the type corresponding to the feature point present on the guide as the type of the feature point of the target to be aligned with the guide. The imaging apparatus according to claim 4, characterized by the following:

6. The target determination means, when the shape of the guide is frame-shaped, determines the type corresponding to the feature point located inside the frame corresponding to the guide as the type of the feature point of the target to be aligned with the guide. The imaging apparatus according to claim 4, characterized by the following:

7. An operation acquisition means for acquiring operation information corresponding to user input, Guide changing means for changing at least one of the position of the guide and the type of the feature point of the target that is aligned with the guide, based on the operation information, Furthermore, to have, The imaging apparatus according to claim 1, characterized by the following:

8. The feature point detection means acquires skeletal information obtained as a result of skeletal estimation for the image of the object being captured in the captured image obtained by shooting, as feature point information in the captured image. The imaging apparatus according to claim 1, characterized by the following:

9. The feature point detection means acquires object information obtained as a result of object detection on the captured image obtained by shooting, as feature point information in the captured image. The imaging apparatus according to claim 1, characterized by the following:

10. Guide generation means that generates a new guide that the user can add based on the position of the feature points detected in the second captured image, Furthermore, to have, The imaging apparatus according to claim 1, characterized by the following:

11. The guide generation means generates a new guide when the same type of feature point is detected in multiple captured images obtained through multiple shooting sessions. The imaging apparatus according to claim 10, characterized by the following:

12. The guide generation means generates a new guide in the shape of a horizontal line when the range of feature points of the same type among the feature points that are not to be aligned with the already set guide in each of the plurality of captured images is within the range of a vertical threshold, The imaging apparatus according to claim 11, characterized by the following:

13. The guide generation means generates a new guide in the shape of a vertical line when the range of feature points of the same type among the feature points that are not to be aligned with the already set guide in each of the plurality of captured images is within the range of a horizontal threshold, The imaging apparatus according to claim 11, characterized by the following:

14. A feature point detection step for detecting feature points in the captured image obtained by shooting, A target determination step, which determines the type of target corresponding to the feature point of the target to be aligned with the guide, based on the positional relationship between the feature point detected in the first captured image and the guide, A guidance output step that outputs guidance information for aligning the position of the feature point corresponding to the type with the position of the guide, based on the positional relationship between the feature point corresponding to the type detected in the second captured image and the guide, A method for controlling a photographic device, characterized by including the following:

15. A program for causing a computer to function as a photographic device according to any one of claims 1 to 13.

Images