Camera device, image processing method, and recording medium

Abstract

The present invention provides an imaging device, an image processing method, and an image processing program capable of improving operability during manual focusing. The method extracts multiple subjects from dynamic image data obtained by imaging, outputs first dynamic image data containing a magnified area including a specific subject among the multiple subjects to a display destination, outputs second dynamic image data containing a magnified area including each of the multiple subjects to the display destination in place of the first dynamic image data if any of the multiple subjects in the dynamic image data changes, and outputs third dynamic image data containing a magnified area including the selected subject to the display destination in place of the second dynamic image data if selection of the subject is indicated.

Description

Technical Field

[0001] The present invention relates to an imaging device, an image processing method, and an image processing program, and more particularly to an imaging device, an image processing method, and an image processing program for outputting dynamic image data obtained by imaging to a display destination. Background Art

[0002] Patent Document 1 describes a technology in which live view display is automatically switched based on focus information of an imaging device during focusing.

[0003] Previous technical literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2020-92315 Summary of the Invention

[0006] One embodiment of the technology disclosed herein provides an imaging device, an image processing method, and an image processing program capable of improving operability during manual focusing.

[0007] Means for solving technical problems

[0008] (1) An imaging device including a processor, wherein:

[0009] The processor performs the following processing in a focusing mode: extracts multiple subjects from dynamic image data obtained by camera imaging, outputs first dynamic image data in which an area including a specific subject among the multiple subjects is magnified to a display destination, and when any one of the multiple subjects in the dynamic image data changes, outputs second dynamic image data in which an area including the multiple subjects is magnified to the display destination instead of the first dynamic image data; and when selection of a subject is indicated, outputs third dynamic image data in which an area including the selected subject is magnified to the display destination instead of the second dynamic image data.

[0010] (2) The imaging device according to (1), wherein

[0011] After the processor outputs the second moving image data to the display destination instead of the first moving image data, if no instruction for subject selection is given within a first time, the processor outputs the first moving image data to the display destination instead of the second moving image data.

[0012] (3) The imaging device according to (1) or (2), wherein:

[0013] The plurality of subjects enlarged in the second moving image data include at least a subject that has changed.

[0014] (4) The imaging device according to any one of (1) to (3), wherein

[0015] The processor further performs a process of accepting the setting of a condition for determining whether the subject has changed.

[0016] (5) The imaging device according to any one of (1) to (3), wherein

[0017] The processor further performs processing for correcting a condition for determining whether the subject has changed based on a history of instructions for selecting the subject.

[0018] (6) The imaging device according to (4) or (5), wherein:

[0019] The condition includes at least one of an option for determination and a threshold value for determination.

[0020] (7) The imaging device according to any one of (1) to (6), wherein

[0021] The processor determines whether the subject has changed according to a change in at least one of the position, size, and distance to the subject.

[0022] (8) The imaging device according to any one of (1) to (7), wherein

[0023] The processor outputs the image in which the enlarged images of the plurality of subjects are sequentially switched to a display destination as second moving image data.

[0024] (9) The imaging device according to any one of (1) to (8), wherein

[0025] The processor outputs an image including the entire image of the moving image data in addition to the enlarged images of the plurality of subjects to a display destination as second moving image data.

[0026] (10) The imaging device according to (9), wherein

[0027] The processor performs processing such that, in the second moving image data, the enlarged images are surrounded by frames of different colors, and corresponding subjects in the entire image are surrounded by frames of the same color.

[0028] (11) The imaging device according to any one of (1) to (10), wherein

[0029] The processor further performs processing for selecting an object to be enlarged in the second moving image data from a plurality of subjects extracted from the moving image data.

[0030] (12) The imaging device according to (11), wherein

[0031] The processor further performs a process of accepting setting of a condition for selecting an object to be enlarged in the second moving image data.

[0032] (13) The imaging device according to (11), wherein

[0033] The processor further performs processing for correcting a condition for selecting an object to be enlarged in the second moving image data based on a history of instructions for selecting a subject.

[0034] (14) The imaging device according to any one of (11) to (13), wherein

[0035] The processor selects an object to be enlarged in the second moving image data based on a change in at least one of a position, a size, and a subject distance.

[0036] (15) The imaging device according to any one of (1) to (13), wherein

[0037] After the processor outputs the second dynamic image data to the display destination, if no selection of the subject is indicated within the first time, the subject determined to have changed when the first dynamic image data is replaced by the second dynamic image data is excluded from the objects for detecting changes during the second time after the first time.

[0038] (16) The imaging device according to any one of (1) to (15), wherein

[0039] In the case of determining whether a subject has changed for multiple options, if the processor does not indicate the selection of the subject within the first time after outputting the second dynamic image data to the display destination, the option determined to have changed when the first dynamic image data is replaced by the second dynamic image data is excluded from the conditions for detecting the change during the second time after the first time.

[0040] (17) The imaging device according to any one of (1) to (16), wherein

[0041] When the depth of field during imaging is set to be narrower than a specific value, the processor increases the magnification ratio of the subject during enlarged display.

[0042] (18) The imaging device according to any one of (1) to (17), wherein

[0043] The processor reduces the magnification ratio of the subject during the magnified display when the value of the high-frequency component in the magnified area is lower than a specific value.

[0044] (19) The imaging device according to any one of (1) to (18), wherein

[0045] The display unit provided in the device body and / or an external display device connected via a connection unit provided in the device body is used as a display destination.

[0046] (20) An image processing method, wherein:

[0047] A plurality of subjects are extracted from dynamic image data obtained by video recording, and first dynamic image data in which an area including a specific subject among the plurality of subjects is magnified is output to a display destination. When any one of the plurality of subjects in the dynamic image data changes, second dynamic image data in which an area including the plurality of subjects is magnified is output to the display destination instead of the first dynamic image data. When selection of a subject is indicated, third dynamic image data in which an area including the selected subject is magnified is output to the display destination instead of the second dynamic image data.

[0048] (21) An image processing program that causes a computer to perform the following processing: extracting multiple subjects from dynamic image data obtained by video recording, outputting first dynamic image data that magnifies an area including a specific subject among the multiple subjects to a display destination, and when any one of the multiple subjects in the dynamic image data changes, outputting second dynamic image data that magnifies the areas of the multiple subjects instead of the first dynamic image data to the display destination; and when selection of a subject is indicated, outputting third dynamic image data that magnifies the area including the selected subject instead of the second dynamic image data to the display destination. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a block diagram showing an example of the hardware configuration of an imaging device.

[0050] Figure 2 This is a conceptual diagram of the function that tracks and zooms in on a specific subject.

[0051] Figure 3 This is a diagram showing an example of changes in the subject.

[0052] Figure 4 This is a diagram showing an example of transition of screen display.

[0053] Figure 5 This is a block diagram of the functions implemented by the CPU regarding the display control of the live view.

[0054] Figure 6 This figure shows an example of the subject extraction result.

[0055] Figure 7 This is a block diagram of the functions of the variation measurement unit.

[0056] Figure 8 This is a conceptual diagram showing an example of the change in measurement position.

[0057] Figure 9 This is a conceptual diagram showing an example of measuring the change in size.

[0058] Figure 10 This is a conceptual diagram showing an example of measuring the change in phase difference.

[0059] Figure 11 This is a diagram showing the relationship between the priority of a subject and the display position on the subject selection screen.

[0060] Figure 12 This is a flowchart showing an example of the procedure for determining whether or not a subject has changed.

[0061] Figure 13 This is a flowchart showing the procedure for switching the live view display.

[0062] Figure 14 This is a flowchart showing the procedure for switching the live view display.

[0063] Figure 15 A diagram showing an example of transition of a live view display

[0064] Figure 16 A diagram showing an example of transition of a live view display

[0065] Figure 17 This is a diagram showing an example of a setting screen for triggering display switching.

[0066] Figure 18 This is a block diagram showing an example of the functions implemented by the CPU when automatically optimizing the conditions for determining whether the subject has changed.

[0067] Figure 19 This is a flowchart showing the procedure for correcting the conditions for determining whether the subject has changed.

[0068] Figure 20 This is a table showing an example of the number of switching times and switching rates for each option for determination change.

[0069] Figure 21 This is a diagram showing an example of a screen for setting an adjustment threshold.

[0070] Figure 22 FIG. 1 is a diagram showing an example of a subject selection screen displayed in the imaging device of this embodiment.

[0071] Figure 23 FIG. 1 is a diagram showing a modified example of the display of the subject selection screen.

[0072] Figure 24 FIG. 1 is a diagram showing another modified example of the display of the subject selection screen.

[0073] Figure 25 This is a diagram showing an example of a screen for adjusting coefficients of a calculation expression for calculation priority.

[0074] Figure 26 This is a block diagram showing an example of the function implemented by the CPU when display switching caused by a change in a specific option is prohibited.

[0075] Figure 27 FIG. 1 is a diagram showing another example of screen division in the subject selection screen.

[0076] Figure 28 FIG. 1 is a diagram showing another example of screen division in the subject selection screen.

[0077] Figure 29 FIG. 1 is a diagram showing another example of screen division in the subject selection screen.

[0078] Figure 30 This figure shows an example of the transition of the screen when switching and displaying the enlarged candidate subject. DETAILED DESCRIPTION

[0079] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0080] [First embodiment]

[0081] As one of the live view functions included in a camera device, a magnified display function is known. The live view magnified display function is a function that cuts out a portion of the entire image displayed as a normal live view image and magnifies and displays it in the entire live view display area. Typically, the live view magnified display function is used when manually focusing. Furthermore, typically, when using the magnified display function, the user manually selects the magnified position. However, when manually focusing, the user manually selecting the magnified position has the disadvantage of making the operation cumbersome. The following describes a camera device that can use the live view magnified display function to improve operability during manual focusing.

[0082] [Hardware Structure of Camera Device]

[0083] Figure 1This is a block diagram showing an example of the hardware configuration of the imaging device.

[0084] The camera device 10 of this embodiment is a camera device capable of capturing dynamic images, and includes a camera optical system 12, a camera unit 14, a display unit 16, a storage unit 18, a connection unit 20, an operation unit 22, a CPU (Central Processing Unit) 24, a ROM (Read Only Memory) 26, and a RAM (Random Access Memory) 28 in a device body not shown.

[0085] The imaging optical system 12 includes a plurality of lens groups for forming a subject image on an imaging element, an aperture for adjusting light intensity, and the like. As a mechanism for adjusting light intensity, a neutral density filter or a neutral density filter (ND filter), which is a type of neutral density filter, may be included in addition to or in place of the aperture.

[0086] The imaging optical system 12 performs focusing by moving a portion of the lens group or the entire lens group back and forth along the optical axis. The focusing lens group is an example of a focus lens. Actuators such as DC motors, linear motors, stepping motors, and ultrasonic motors are used to drive the focusing lens group.

[0087] The imaging optical system 12 can be manually focused. The focusing actuator is driven by operating an operating ring (focus ring) provided on the lens barrel. Manual focusing can also be performed using an operating lever or the like in addition to the operating ring.

[0088] In the case of manual focus, the focus mode (focus mode) is set to manual focus mode (ManualFocus Mode). In addition to the manual focus mode, the focus mode also includes an automatic focus mode (Auto Focus Mode). The focus mode is switched, for example, by a switch (focus mode switching switch). In addition, there may also be a semi-automatic focus mode. The semi-automatic focus mode includes the following modes: although the focus is adjusted manually, the focus status is also monitored by the automatic focus adjustment device, and when the manually focused object can be identified, the focus is continuously adjusted by automatic focus; and the focus is first adjusted by automatic focus, and then fine-tuned by manual adjustment.

[0089] The imaging unit 14 includes an imaging element that converts an optical image into an electrical signal. Examples of the imaging element include a CMOS image sensor (Complementary Metal Oxide Semiconductor) having a predetermined color filter array (e.g., a Bayer array) or a CCD image sensor (Charged Coupled Device).

[0090] In the imaging device 10 of this embodiment, a CMOS image sensor equipped with a driver, an ADC (Analog to Digital Converter), and a signal processing unit is used as an imaging element. In this case, the imaging element is driven by the built-in driver to operate. Furthermore, the signal from each pixel is converted into a digital signal by the built-in ADC. Furthermore, the built-in signal processing unit performs signal processing such as correlated double sampling, gain processing, and correction processing on the signal from each pixel. Furthermore, the signal processing can be performed on either the analog signal from each pixel or the digital signal from each pixel.

[0091] Furthermore, the imaging device 10 of this embodiment uses an imaging element embedded with phase difference pixels as an imaging element. By using an imaging element embedded with phase difference pixels, it is possible to obtain phase difference information of a subject within the image. Furthermore, based on this acquired phase difference information, it is possible to detect the direction of focus shift and the amount of focus shift (defocus) of the subject within the image. Imaging elements embedded with phase difference pixels and methods for detecting phase difference using such imaging elements are well known. Therefore, a detailed description thereof will be omitted.

[0092] The video is captured at a predetermined frame rate. The frame rate may be fixed or set to any value by the user.

[0093] The display unit 16 is composed of a display such as an LCD (Liquid Crystal Display), an organic electroluminescent (EL) display (Organic Light Emitting Diode display, OLED display). In the imaging device 10 of this embodiment, a touch panel is provided on the display surface of the display constituting the display unit 16. That is, the display constituting the display unit 16 is composed of a touch panel display. In addition, the touch panel may be a so-called embedded structure built into the display unit 16. There is a method in which an EVF (Electronic View Finder) is also included in the display unit 16. However, in the case where the display unit 16 is composed of an EVF, the display unit 16 does not have a touch panel, but can have a line of sight detection function, for example.

[0094] The display unit 16 displays a live view during shooting and also serves as a user interface for making various settings.

[0095] The storage unit 18 is used to store various data in addition to captured moving image data (recorded moving image data). The storage unit 18 is comprised of, for example, non-volatile semiconductor memory such as EEPROM (Electrically Erasable Programmable Read-only Memory) including flash memory, or an SSD (Solid State Drive) incorporating such semiconductor memory. The storage unit 18 can be integrally mounted on the device (so-called built-in memory) or removable (so-called memory card).

[0096] The connection unit 20 includes a terminal for connecting to an external device (eg, an external display device, an external recording device, etc.) As a connection standard, for example, HDMI (High-Definition Multimedia Interface) can be adopted (HDMI is a registered trademark).

[0097] The operating unit 22 includes various operating components for operating the camera device 10. These components include a touch panel in addition to various operating buttons such as a power button, a record button, a mode dial, and a command dial. In particular, in the camera device 10 of this embodiment, the operating unit 22 includes a focus mode switch for switching focus modes, a live view button for turning the live view display on and off, and a tracking zoom display button for turning the tracking zoom display function on and off during live view. The focus mode switch is, for example, a slide switch, and by switching the switch position, the focus mode (focusing mode) is selectively switched to autofocus mode, manual focus mode, or the like. The live view button is, for example, a push button, and each press of the button switches the live view display on and off. The tracking zoom display function will be described later.

[0098] CPU 24 is an example of a processor. By executing prescribed programs, CPU 24 functions as the control unit and image processing unit of the camera device 10. The controls performed by CPU 24 include, in addition to camera control such as automatic exposure and autofocus, display control of the display unit 16 and recording control of the storage unit 18. Furthermore, the image processing performed by CPU 24 includes the generation of dynamic image data for recording and display (live view) and compression of the recorded dynamic image data. The recorded and displayed dynamic image data is generated by applying prescribed image processing (so-called development processing) to RAW data (unprocessed dynamic image data output from the camera unit 14). The image processing herein includes offset processing, gamma correction processing, demosaicing processing, RGB / YCrCb conversion processing, and white balance processing. These are well-known processes, and therefore detailed descriptions thereof are omitted. Compression processing is the process of generating a dynamic image file in a prescribed compressed format. A well-known codec can be used as the codec used for compression. For example, codecs (MPEG-1, MPEG-2, MPEG-4, etc.) standardized by MPEG (Moving Picture Experts Group) can be employed.

[0099] Furthermore, the above-described respective processes performed by the CPU 24 may be partially or entirely performed by hardware provided inside the CPU 24 .

[0100] The ROM 26 stores various data necessary for programs executed by the CPU 24 and control, etc. The memory constituting the ROM 26 includes an EEPROM including a flash memory.

[0101] The RAM 28 can be used as a work area when the CPU 24 performs various processes.

[0102] [Display control for live view]

[0103] Next, the live view display control will be described.

[0104] [Overview of Live View Display Control]

[0105] The imaging device 10 of this embodiment includes a function for tracking and magnifying a specific subject (tracking zoom display function) as a live view function. The operating unit 22 includes a tracking zoom display button for turning this tracking zoom display function on and off. The tracking zoom display button is, for example, a push-button. Pressing the tracking zoom display button while live view is on turns the tracking zoom display function on. Furthermore, pressing the tracking zoom display button while the tracking zoom display function is on turns the tracking zoom display function off. In the imaging device 10 of this embodiment, the tracking zoom display function can be turned on only when the manual focus mode is set. That is, in other focus modes, the tracking zoom display function is always off.

[0106] Figure 2 This is a conceptual diagram of the function that tracks and zooms in on a specific subject.

[0107] Figure 2 (A) shows a normal live view display. In a normal live view display, the entire image is displayed in the entire display area of ​​the live view. The entire image refers to an image that displays the entire image captured by the camera unit 14. In addition, the image when the entire image is displayed at a specified field of view is also included in the entire image here. Hereinafter, as needed, the method of displaying the entire image in the entire display area of ​​the live view is referred to as the entire display. The display area of ​​the live view refers to the area set as the area for displaying the live view within the entire display area of ​​the display unit 16. Therefore, when the entire display area of ​​the display unit 16 is set as the display area for the live view, the live view image is displayed in the entire display area of ​​the display unit 16.

[0108] Here, the user turns on the tracking zoom display function and selects subject A as the object to be zoomed in. After the selection is completed, the display of the screen is switched, as shown in FIG. Figure 2 As shown in (B), an image of subject A as a single unit, magnified at a predetermined magnification ratio, is displayed as the live view image. Hereinafter, this display method will be referred to as single magnification display, as needed. Specifically, the method of displaying a single unit magnified subject is referred to as single magnification display. Subsequently, the single magnification display is performed to follow the movement of subject A. In other words, the magnified image of subject A is always displayed, with subject A positioned at (approximately) the center of the live view display area.

[0109] This function supports manual focus. Therefore, the zoomed-in subject is selected. By tracking and zooming in on the subject to be focused, the user can focus on the focusing operation, improving operability.

[0110] When changing the magnified object, it is generally considered that the magnification is temporarily canceled, the display returns to normal, and the selection operation is performed again.

[0111] However, if the object to be magnified is changed in such a sequence, the operation becomes cumbersome and the focusing operation may be neglected during the operation.

[0112] Therefore, in the imaging device 10 of the present embodiment, when a change of a predetermined amount or more is observed in the subject being imaged, the zoom is automatically released and the screen switches to the subject selection screen.

[0113] Figure 3 This is a diagram showing an example of changes in the subject.

[0114] In the imaging device 10 of the present embodiment, zoom-in is released when at least one of the position, size, and subject distance changes.

[0115] Figure 3 (A) shows a captured image at a certain point in time. Figure 3 (B) shows the Figure 3 An image captured at time (A) a predetermined time later. In this example, the positions of subjects A and B have changed. Furthermore, the subject distance to subject B has also changed. In this captured image, if at least one of the position, size, or subject distance of the subject being captured changes by a predetermined amount or more, zoom is released.

[0116] Figure 4 This is a diagram showing an example of transition of screen display.

[0117] Figure 4 (A) shows an example of screen display during single magnification display. Figure 4 (A) shows an example in which subject A is selected as the target for enlargement.

[0118] Figure 4(B) shows an example of a subject selection screen. This example shows an example of dividing a screen into four equal parts, and displaying a candidate subject for enlargement in each area. The candidate subject is enlarged and displayed in each area. The display of each area follows the movement of the subject. That is, in each area, an enlarged image of each subject is displayed with the subject located at the center (approximately the center) of each area. The image of the subject enlarged and displayed in each area is an example of an enlarged image. The user selects one of the subjects (enlarged images) displayed in each area to select the object to be enlarged.

[0119] Figure 4 (C) shows that Figure 4 (B) shows an example of a single magnified display when subject B is selected in the subject selection screen. Figure 4 As shown in (C), when a magnified subject (subject B in this example) is selected, the display switches to single magnification, displaying the selected subject (subject B in this example) in magnified form. The single magnification display then follows the movement of the selected subject (subject B in this example).

[0120] The selection is made within the specified time. If no selection is made within the specified time, the display will be restored to the original state. Figure 4 In the example shown, Figure 4 As shown in (A), the screen switches to a screen in which the subject A is magnified and displayed alone.

[0121] As described above, by controlling the live view display, good operability can be achieved during manual focus adjustment.

[0122] [Functions related to live view display control]

[0123] Figure 5 This is a block diagram of functions implemented by the CPU regarding live view display control.

[0124] like Figure 5 As shown, regarding the live view display, the CPU 24 functions as a subject extraction unit 110, a change amount measurement unit 112, an enlargement candidate selection unit 114, a display switching determination unit 116, and a display-use dynamic image data generation unit 118. Each function is realized by the CPU 24 executing a predetermined program (image processing program).

[0125] The subject extraction unit 110 processes the moving image data obtained through video capture and extracts the subject from each frame. In the imaging device 10 of this embodiment, a person's face is extracted as the subject. Known techniques can be used to extract a person's face from an image. For example, techniques using image recognition models generated through machine learning, deep learning, or the like can be used to extract the face region.

[0126] Figure 6 FIG. 1 is a diagram showing an example of subject extraction results.

[0127] like Figure 6 As shown, the extracted subject is surrounded by a bounding box BB, and its position and size are determined. The position of the subject is determined, for example, by the coordinate position (x, y) of the center or center of gravity of the bounding box BB. Furthermore, the size of the subject is determined, for example, by the area of ​​the bounding box BB.

[0128] The extraction of the subject is performed periodically at a predetermined extraction cycle, and the subjects are numbered in the order of extraction (k=1, 2, ...).

[0129] The variation measurement unit 112 measures the variation in position, size, and phase difference for each subject extracted by the subject extraction unit 110. This measurement is performed periodically at a predetermined measurement cycle. Therefore, the variation is measured for each cycle. The measurement cycle is set based on the subject extraction cycle.

[0130] Figure 7 This is a block diagram of the functions of the variation measurement unit.

[0131] like Figure 7 As shown, the variation measuring section 112 has the functions of a position variation measuring section 112A, a size variation measuring section 112B, and a phase difference variation measuring section 112C.

[0132] The position change amount measurement unit 112A measures the change amount of the position of each subject based on the extraction result of the subject extracted by the subject extraction unit 110 .

[0133] Figure 8 This is a conceptual diagram showing an example of the amount of change in the measurement position. Figure 8 (B) shows a camera image at a certain measurement time point, Figure 8 (A) shows a captured image at a measurement time point one cycle before.

[0134] The change in the position of the object is measured as the displacement of the coordinate position. For example, the coordinate position of the object at a certain measurement time point is set to (x2, y2). If the coordinate position of the object at the measurement time point one cycle ago is set to (x1, y1), the change in position d is calculated by ((x2-x1) 2 +(y2-y1) 2 ) 1 / 2 To calculate.

[0135] The size change amount measurement unit 112B measures the amount of change in the size of each subject based on the extraction result of the subject extracted by the subject extraction unit 110 .

[0136] Figure 9 This is a conceptual diagram showing an example of the amount of change in measurement size. Figure 9 (B) shows a camera image at a certain measurement time point, Figure 9 (A) shows a captured image at a measurement time point one cycle before.

[0137] The change in the size of the object is measured, for example, as the change in the area of ​​a bounding box BB surrounding the object being measured. For example, let S2 be the area of ​​the bounding box BB surrounding the object being measured at a certain measurement time point. If S1 is the area of ​​the bounding box BB of the object at the measurement time point one cycle prior, the size change is calculated as S2 / S1.

[0138] The phase difference variation measurement unit 112C measures the phase difference variation for each subject extracted by the subject extraction unit 110 at a predetermined measurement cycle. As described above, the imaging device 10 of this embodiment uses an imaging element embedded with phase difference pixels. Therefore, by processing the moving image data obtained through imaging, information about the phase difference of each subject can be obtained.

[0139] Figure 10 This is a conceptual diagram showing an example of measuring the amount of change in phase difference. Figure 10 (B) shows a camera image at a certain measurement time point, Figure 10 (A) shows a captured image at a measurement time point one cycle before.

[0140] exist Figure 10 The example shown shows an example where the subject being measured moves closer to the imaging device 10. In this case, the subject distance changes. This change in subject distance also changes the phase difference. The change in phase difference is calculated as the difference from the phase difference at the measurement point one cycle prior.

[0141] As described above, the phase difference changes according to the subject distance, so detecting the phase difference is synonymous with detecting the subject distance.

[0142] The enlargement candidate selection unit 114 selects a subject to be displayed on the subject selection screen from among the multiple subjects extracted by the subject extraction unit 110. The enlargement candidate selection unit 114 selects a subject (enlargement candidate subject) to be displayed on the subject selection screen based on the information on the amount of change in position, size, and phase difference of each subject measured by the amount of change measurement unit 112. In the imaging device of this embodiment, four subjects as enlargement candidates are displayed on the subject selection screen (see FIG. 1 ). Figure 4 (B)), select 4 subjects.

[0143] The selection of the subject is performed by calculating the priority of each subject extracted from the image and extracting the top four subjects with the highest priority. The priority is calculated, for example, by the following formula.

[0144] Priority = w1×(change in position) + w2×(change in size) + w3×(change in phase difference)×w3

[0145] Here, w1, w2, and w3 are preset coefficients. That is, the priority of each subject is calculated by multiplying the change in position, size, and phase difference by the predetermined coefficients w1, w2, and w3, respectively, and calculating the sum of the multiplication results.

[0146] By calculating the priorities in this manner and selecting subjects with high priorities as candidates for enlargement, it is possible to select subjects as candidates for enlargement including at least the subject that has changed.

[0147] Figure 11 This is a diagram showing the relationship between the priority of a subject and the display position on the subject selection screen.

[0148] As described above, in this embodiment, the subject selection screen is divided into four equal parts, and subjects that are candidates for enlargement are displayed in each area in an enlarged manner. In the four-part screen, the upper left area is set as the first display area A1, the upper right area is set as the second display area A2, the lower left area is set as the third display area A3, and the lower right area is set as the fourth display area A4. The subject with the highest priority is displayed in the first display area A1, the subject with the second priority is displayed in the second display area A2, the subject with the third priority is displayed in the third display area A3, and the subject with the fourth priority is displayed in the fourth display area A4.

[0149] In addition, when the priorities are equal, for example, the superiority is determined according to the following judgment rules. That is, the change in position is compared, and the subject with the larger change in position is given an advantage. When the change in position is also equal, the change in size is then compared, and the subject with the larger change in size is given an advantage. When the change in size is also equal, the change in phase difference is then compared, and the subject with the larger change in phase difference is given an advantage. As described above, each option is compared in turn to determine the superiority. In addition, in this example, the example of determining the superiority in the order of position, size, and phase difference is explained, but it can also be set as a structure in which the user can arbitrarily set the order.

[0150] The display switching determination unit 116 determines whether the live view display needs to be switched based on the information on the amount of change in the position, size, and phase difference of each subject measured by the amount of change measurement unit 112 and the operation input information of the operation unit 22 .

[0151] Here, the information on the amount of change in the position, size, and phase difference of each subject measured by the amount of change measurement unit 112 is used to determine whether it is necessary to switch to the subject selection screen.

[0152] On the other hand, when the screen switches to the subject selection screen, the operation input information of the operation unit 22 is used to determine whether a subject has been selected and the selected subject. The operation input information of the operation unit 22 is used to determine whether the tracking zoom display function is turned on or off.

[0153] Whether to switch to the subject selection screen is determined by whether the subject has changed. Whether the subject has changed is determined as follows.

[0154] Figure 12 This is a flowchart showing an example of a procedure for determining whether or not there is a change in a subject.

[0155] First, the subject extraction unit 110 acquires information on the total number n of subjects extracted from an image (step S11 ).

[0156] Next, the value of k is set to 1 (step S12).

[0157] Next, information on the position, size, and phase difference change of subject k is obtained (step S13). Here, subject k is the number of the subject numbered in subject extraction unit 110. When k = 1, information on the position, size, and phase difference change of subject number 1 is obtained. Similarly, when k = 2, information on the position, size, and phase difference change of subject number 2 is obtained.

[0158] Next, the change in the position of the subject k is compared with a threshold value (position threshold value) to determine whether it is equal to or greater than the threshold value (step S14 ). The information on the threshold value (position threshold value) is stored in the ROM 26 , for example.

[0159] When the amount of change in the position of the subject k is equal to or greater than the threshold value (YES in step S14 ), it is determined that the subject has changed (step S18 ).

[0160] On the other hand, if the change in the position of the subject k is less than the threshold value ("No" in step S14), the change in the size of the subject k is then compared with the threshold value (the size threshold value) to determine whether it is greater than the threshold value (step S15). The threshold value information (the size threshold value information) is stored in the ROM 26, for example.

[0161] When the amount of change in the size of the subject k is equal to or greater than the threshold value (YES in step S15 ), it is determined that the subject has changed (step S18 ).

[0162] On the other hand, if the change in the size of the subject k is less than the threshold value ("No" in step S15), the change in the phase difference of the subject k is then compared with the threshold value (the phase difference threshold value) to determine whether it is greater than the threshold value (step S16). The threshold value information (the size threshold information) is stored in the ROM 26, for example.

[0163] When the amount of change in the phase difference of the subject k is equal to or greater than the threshold value (YES in step S16 ), it is determined that the subject has changed (step S18 ).

[0164] On the other hand, if the change in phase difference of subject k is less than the threshold ("No" in step S16), it is determined that the subject has not changed (step S17). In this case, all options of position, size, and phase difference are less than the threshold.

[0165] Once the determination of whether or not a subject has changed is completed, it is next determined whether the determination for all subjects is complete. Specifically, it is determined whether k = n (step S19). n is the total number of subjects extracted from the captured image. Therefore, if k = n, it can be determined that the determination for all subjects is complete. At this point (if "yes" in step S19), the process of determining whether or not a subject has changed is terminated.

[0166] When k=n, determination on all subjects has not been completed, so 1 is added to the value of k (step S20 ), and determination on the next-numbered subject is performed (returning to step S13 ).

[0167] If the above determination process determines that at least one subject has changed, the display switching determination unit 116 determines that it is necessary to switch to the subject selection screen. On the other hand, if it is determined that all extracted subjects have not changed, the display switching determination unit 116 determines that it is not necessary to switch to the subject selection screen.

[0168] When the screen switches to the subject selection screen, whether the subject to be enlarged is selected is determined based on the operation input of the operating unit 22. In this embodiment, subject selection is performed on the touch panel. Specifically, the subject to be enlarged is selected by touching the area (first display area A1 to fourth display area A4) where the subject to be enlarged is displayed.

[0169] If no selection is made within a predetermined time after switching to the subject selection screen, that is, if no touch on the touch panel is detected, the display switching determination unit 116 determines that no subject has been selected. The predetermined time for making this selection is an example of the first time.

[0170] As described above, the zoom display function is turned on and off using the tracking zoom display button included in operation unit 22. The tracking zoom display button is, for example, a push-button. Pressing the tracking zoom display button while the tracking zoom display function is off turns the tracking zoom display function on. Conversely, pressing the tracking zoom display button while the tracking zoom display function is on turns the tracking zoom display function off.

[0171] If the tracking zoom display function is turned on, the subject to be zoomed in on the live view image is selected and the selected subject is displayed in single zoom (single zoom display). At this time, for example, the selectable subject is displayed surrounded by a boundary frame (see Figure 6 ). Therefore, by selecting the bounding box (touching the subject within the bounding box), the object to be enlarged is selected.

[0172] Figure 13 and Figure 14 This is a flowchart showing the procedure for switching the live view display.

[0173] First, the live view is turned on or off (step S31). The live view is turned on or off by operating the live view button.

[0174] When the live view display is turned on (in the case of "yes" in step S31), first, the entire image is displayed as a live view (step S32). That is, the image captured by the camera unit 14 is directly displayed (refer to Figure 2 (A)).

[0175] In the state of the entire display, it is determined whether the tracking zoom display function is turned on or off (step S33). The tracking zoom display function is switched on or off by operating the tracking zoom display button.

[0176] If the tracking zoom display function is off (if step S33 returns "No"), the live view is turned on or off (step S37). If the live view display is off (if step S37 returns "Yes"), the live view display ends. On the other hand, if the live view display is on (if step S37 returns "No"), the entire display (step S32) continues.

[0177] In step S33, if the tracking zoom display function is on (if "yes"), the screen for selecting the zoom object is switched to step S34. As described above, in this selection screen, the selectable subject in the live view image displayed as a whole is surrounded by a bounding box and displayed (see Figure 6 ).

[0178] On the display of the enlargement object selection screen, it is determined whether the object to be enlarged is selected (step S35). The object is selected via the touch panel. The user touches the subject within the boundary frame displayed on the entire screen to select the object to be enlarged.

[0179] If no object has been selected ("No" in step S35), the tracking zoom display function is turned on or off (step S36). If the tracking zoom display function is off ("Yes" in step S36), the live view display is turned on or off (step S37). If the live view display is off ("Yes" in step S37), the live view display ends. On the other hand, if the live view display is on ("No" in step S37), the display switches to normal overall display (step S32). In other words, the overall display switches to a state where the bounding box is not displayed.

[0180] In step S35, if the object is selected (in the case of "Yes"), the display is switched to a display in which the selected object is enlarged as a single unit (single enlarged display) (see Figure 2 (B))(Step S41).

[0181] like Figure 14 As shown, if the display is switched to single magnification display, it is determined whether the subject has changed (step S42).

[0182] If the subject has not changed (NO in step S42 ), it is determined whether the tracking zoom display function is on or off (step S46 ).

[0183] When the tracking zoom display function is turned off (in the case of "Yes" in step S46), Figure 13 As shown, the live view display is turned on / off (step S37). If the live view display is turned off (if the answer is "yes" in step S37), the live view display ends. On the other hand, if the live view display is on (if the answer is "no" in step S37), the display switches to the normal overall display (step S32).

[0184] When the tracking zoom display function is on (in the case of "No" in step S46), Figure 14 As shown, single magnification display is continuously performed on the same subject (step S41).

[0185] In step S42, if the subject has changed (if "yes"), the single magnified display is switched to the subject selection screen (step S43). As described above, in the subject selection screen, the subject that is a candidate for magnification is magnified and displayed in each display area of ​​the screen divided into four equal parts (refer to Figure 4 (B)).

[0186] Once the screen switches to the subject selection screen, a determination is made as to whether an object to be enlarged has been selected (step S44). The user touches the display area showing the desired subject to be enlarged, selecting the subject to be enlarged. This selection operation indicates a change in the subject to be enlarged. If the candidate images include a subject that is displayed in a single magnified state, selecting the subject in the single magnified state does not actually change the subject, but it is included in the instruction to change the subject.

[0187] If a subject is selected in the subject selection screen (if "Yes" in step S44), the display switches to a magnified display of the selected subject (single magnified display) (refer to Figure 4 (C))(Step S41).

[0188] On the other hand, if the subject has not been selected (No in step S44), it is determined whether a predetermined time has passed since the subject selection screen was switched to (step S45). If the predetermined time has not passed, the subject selection screen is maintained (step S43).

[0189] On the other hand, if the predetermined time has elapsed (if "YES" in step S44), the on / off status of the tracking zoom display function is determined (step S46). If the tracking zoom display function is on (if "NO" in step S46), the subject selection screen is switched back to the original single zoom display (step S41). In other words, the zoom display is switched back to the single zoom display of the subject that was previously displayed before switching to the subject selection screen.

[0190] As described above, after switching to single magnified display, if a change of a specified amount or more is observed in the subject being photographed, the screen automatically transitions to the subject selection screen. Furthermore, if a subject is selected within a specified time on this subject selection screen, the selected subject is displayed in single magnified display. On the other hand, if no subject is selected within the specified time, the screen returns to the original single magnified display.

[0191] By automatically switching the display in this way, the user can concentrate on the focusing operation even when manually adjusting the focus, thereby improving the operability during manual focusing.

[0192] The display dynamic image data generation unit 118 generates display dynamic image data to be output to the display unit 16 based on the display switching determination result determined by the display switching determination unit 116. The display dynamic image data includes dynamic image data for overall display, dynamic image data for single magnification display, and dynamic image data for the subject selection screen during single magnification display. Furthermore, the subject selection screen dynamic image data includes initial selection dynamic image data and subject switching dynamic image data. The initial selection dynamic image data is dynamic image data displayed when the tracking magnification display function is turned on. That is, it is dynamic image data for selecting a subject to be displayed first in single magnification. As described above, the dynamic image data is generated by enclosing an image of a selectable subject in the overall displayed image with a boundary frame (refer to Figure 6 ). The subject switching dynamic image data is dynamic image data displayed when the subject being photographed changes by more than a predetermined amount in single magnification display. As described above, this dynamic image data is generated by dividing one screen into four equal parts and displaying an image of the subject selected as the magnification candidate in each display area (refer to Figure 4 (B) Figure 11 ).

[0193] The subject switching moving image data is an example of the second moving image data, and is moving image data in which the regions of a plurality of subjects are enlarged.

[0194] Furthermore, the single enlarged display moving image data before switching to the subject switching moving image data is an example of the first moving image data, and constitutes moving image data in which a region including a specific subject among a plurality of subjects is enlarged.

[0195] The single enlarged display moving image data after the subject is switched is an example of third moving image data, and constitutes moving image data obtained by enlarging a region including the subject selected as the enlargement target.

[0196] [Transition of the live view display]

[0197] Figure 15 and Figure 16 This is a diagram showing an example of the transition of the live view display.

[0198] If the live view display is turned on, first, Figure 15 (A) shows the whole display. If the tracking zoom display function is turned on in the whole display, Figure 15 As shown in (B), the screen switches to the subject selection screen to be displayed in a single magnified image. The user selects the subject to be displayed in a single magnified image on the subject selection screen. The subject selected here is an example of a specific subject. The selection is made by touching the object on the screen. If the subject to be displayed in a single magnified image is selected, the screen will be displayed as shown below. Figure 15 (C)(and Figure 16 As shown in (A), the selected subject (specific subject) is enlarged and displayed as a single body. In other words, it is enlarged and displayed alone. Figure 15 (C)(and Figure 16 (A) shows an example of a case where subject A is selected. Subsequently, the selected subject is displayed in a single magnified image. The moving image data displayed in a single magnified image is an example of first moving image data.

[0199] In the single magnified display, the changes of the subject being photographed are measured at a specified measurement cycle. If the change exceeds the specified amount, the Figure 16 As shown in (B), the screen switches to the subject selection screen. In this subject selection screen, four subjects selected for enlargement are displayed enlarged in each of the four equally divided display areas. The display in each display area tracks the movement of the subject. Specifically, in each display area, an enlarged image of each subject is displayed, with the subject positioned at (approximately) the center of the display area. The dynamic image data used in this subject selection screen is an example of the second dynamic image data.

[0200] The user selects a subject to be displayed in magnified form on the subject selection screen. The selection is made by touching the subject on the screen. By selecting the subject to be displayed in magnified form, a change of the subject is instructed.

[0201] If you select an object, Figure 16 As shown in (C), the selected subject is displayed in magnified form. Figure 16 (C) shows an example where subject B (the subject in the upper right display area of ​​the subject selection screen) is selected. Subsequently, the selected subject is displayed in a magnified form. The changed, magnified form of the moving image data is an example of the third moving image data.

[0202] Even after the subject is changed, the change of the subject being photographed is measured at a predetermined measurement cycle in the single magnified display. If the change exceeds the predetermined amount, the Figure 16 As shown in (B), the screen switches to the subject selection screen.

[0203] Furthermore, if the tracking zoom display function is turned off in the single zoom display, Figure 16 As shown in (D), switch to the overall display.

[0204] As described above, in the imaging device 10 of this embodiment, when tracking a specific subject and displaying a magnified image, if the subject being photographed changes by a specified amount or more, the screen automatically transitions to the subject selection screen. Furthermore, if a subject is selected in the subject selection screen, the selected subject is displayed in magnified form. On the other hand, if a specified time has passed without a subject being selected in the subject selection screen, the screen returns to the original magnified form. This allows the user to concentrate on the focusing operation even when manually adjusting the focus.

[0205] [Second embodiment]

[0206] In the imaging device of the above-described embodiment, when the display switches from a single magnified display to a subject selection screen, a change in the position, size, or phase difference of the subject being photographed serves as a trigger. In the imaging device of this embodiment, a configuration is provided in which the user can arbitrarily set the subject change that triggers the display switch. In other words, the user can arbitrarily set the conditions for determining whether the subject has changed. For example, the user can arbitrarily set a configuration in which the display switches from a single magnified display to a subject selection screen using only a change in position as a trigger, or a configuration in which the display switches from a single magnified display to a subject selection screen using only a change in position and size as triggers. In the imaging device of this embodiment, the trigger for the display switch is set on a predetermined setting screen.

[0207] Figure 17 This is a diagram showing an example of a setting screen for triggering display switching.

[0208] This setting screen is called from the menu screen, for example. Figure 17 As shown in the figure, for each option of "Position", "Size" and "Phase Difference", "On" and "Off" buttons are displayed. Select one of the "On" and "Off" buttons. Figure 17 In the example shown, the button with white background is the selected button. Figure 17 In the example shown, the "Position" option is set to "On," the "Size" option is set to "Off," and the "Phase Difference" option is set to "Off." In this case, the display switches from the single magnified display to the subject selection screen, triggered only by the change in position.

[0209] The CPU 24 receives a change in setting based on an operation input from the operation unit 22 (touch panel). The set information is stored in the storage unit 18.

[0210] The display switching determination unit 116 determines whether the subject has changed based on the set information and determines whether it is necessary to switch to the subject selection screen. For example, if only the location option is turned on, Figure 12 In the flowchart shown, the processing of step S15 (processing for determining whether the amount of change in size is equal to or greater than a threshold value) and the processing of step S16 (processing for determining whether the amount of change in phase difference is equal to or greater than a threshold value) are skipped.

[0211] As described above, by providing a configuration in which the user can arbitrarily set a change in the subject as a trigger, it is possible to prevent unnecessary switching of the display according to the user, thereby further improving operability.

[0212] In the above embodiment, the change in position, size, and phase difference (subject distance) is measured as the change in the subject. However, other changes may be measured instead of or in addition to these.

[0213] Furthermore, in the above embodiment, a configuration is used to determine that a subject has changed when at least one of its position, size, or phase difference (subject distance) has changed by a predetermined amount or more. However, a configuration may also be used to determine whether a subject has changed by combining multiple changes. For example, a configuration may be used to determine that a subject has changed when both the position change and the size change are greater than a threshold.

[0214] [Third embodiment]

[0215] In the camera device of the second embodiment described above, the following structure is set: the user can arbitrarily set the conditions for determining whether the subject has changed. In the camera device of this embodiment, the determination conditions are automatically optimized based on the past selection tendencies of the user. For example, in the case where there are multiple switchings of the display due to changes in position, and the subject is not selected once (switching of the subject), it is considered that the user does not want the switching of the display due to the change in position. Therefore, in this case, it is set not to detect the change of the subject due to the change in position. In this way, unnecessary switching of the display can be prevented, and the operability can be further improved. The following describes a structure in which the determination conditions are automatically optimized based on the past selection tendencies of the user.

[0216] Figure 18 This is a block diagram showing an example of functions implemented by the CPU when automatically optimizing the conditions for determining whether a subject has changed.

[0217] like Figure 18 As shown, the CPU 24 also functions as a selection result recording unit 120 and a determination condition correction unit 122 in order to automatically optimize the conditions for determining whether the subject has changed.

[0218] The selection result recording unit 120 records the subject selection results when switching from the single magnified display to the subject selection screen in the storage unit 18. The selection results are recorded by associating information about the subject's selected option, which triggered the display switch, with information about whether the selection was made. The selection results are recorded in the storage unit 18 as selection history data. This selection history data is updated each time the subject selection screen is displayed.

[0219] The determination condition correction unit 122 corrects the condition for determining whether the subject has changed based on the selection history data. The correction is performed, for example, as follows. That is, the switching rate R of the subject is calculated, and options whose switching rate R of the subject is below a threshold are excluded from the condition for determining whether the subject has changed. Here, the so-called switching rate R of the subject is calculated by R=n / N, if the number of times the single magnified display is switched to the subject selection screen is set to N and the number of times the subject is switched (the number of times the subject is selected in the subject selection screen) is set to n based on the change of the option to be calculated. That is, options whose ratio of actual subject switching is below a specified value are excluded from the condition for determining whether the subject has changed.

[0220] Figure 19 This is a flowchart showing a procedure for correcting conditions for determining whether a subject has changed.

[0221] Here, as a condition for determining that a subject has changed, it is determined that the change in at least one of position, size, and phase difference is greater than a threshold. The position option is set as the first option for determining a change, the size option is set as the second option for determining a change, and the phase difference option is set as the third option for determining a change.

[0222] like Figure 19 As shown, first, the variable i is set to 0 (step S51). Then, 1 is added to the value of i (step S52). Next, the selection history data of the i-th option to be determined is obtained (step S53). For example, in the case of i=1, the selection history data of the option at position is obtained.

[0223] Next, regarding the i-th option that is determined to have changed, it is determined whether the number of switching times is greater than a threshold value (step S54). Here, the number of switching times is the number of times (total number) that the single magnified display is switched to the selection screen of the subject according to the i-th option that is determined to have changed. If the number of switching times is less than the threshold value (the threshold value of the number of switching times), it is excluded from the current correction target. That is, at this time, it is determined that data that fully grasps the selection tendency has not been obtained, and it is excluded from the correction target. If the number of switching times is less than the threshold value (if "No" in step S54), the process proceeds to step S57. For example, if the threshold value of the number of switching times is set to 15 times, if the number of switching times of the i-th option that is determined to have changed is less than 15 times, it is excluded from the current correction target.

[0224] Regarding the i-th option for determining a change, if the number of switching times is greater than the threshold (if "yes" in step S54), it is then determined whether the switching rate R is less than the threshold. If the switching rate R is less than the threshold (the threshold value of the switching rate R) (if "yes" in step S55), the i-th option is excluded from the conditions for determining whether the subject has changed (step S56). In other words, at this time, although the display is switched, the subject is not selected (switched), so it is excluded from the conditions for determining whether the subject has changed. For example, if the threshold value of the switching rate R is set to 5%, if the switching rate R is less than 5% when the display is switched according to the i-th option for determining a change, the i-th option for determining a change is excluded from the conditions for determining whether the subject has changed.

[0225] On the other hand, if the switching rate R exceeds the threshold (switching rate threshold) (if "No" in step S55), a determination is made as to whether the variable i and j have the same value (i = j) (step S57). j represents the total number of determination options. If the determination options are position, size, and phase difference, j = 3. In other words, a determination is made as to whether the correction process has ended for all options. If the process has ended for all options, the correction process for determining whether the subject has changed ends.

[0226] Figure 20 This is a table showing an example of the number of switching times and switching rates for each option regarding the determination change.

[0227] Figure 20 An example is shown in which the screen switches from a single magnified display to a subject selection screen according to changes in the subject's position, size, and phase difference.

[0228] exist Figure 20 In the example shown, the display was switched 30 times based on position changes (number of switches). The proportion of subjects selected (switched) when the display was switched based on position changes (switching rate) was 0%. Furthermore, the display was switched 20 times based on size changes (number of switches). The proportion of subjects selected (switched) when the display was switched based on size changes (switching rate) was 60%. Furthermore, the display was switched 10 times based on phase difference changes (number of switches). The proportion of subjects selected (switched) when the display was switched based on phase difference changes (switching rate) was 3%.

[0229] Here, if the threshold value of the number of switching times is set to 15 times and the threshold value of the switching rate is set to 5%, the number of switching times for the phase difference is less than the threshold value and is therefore excluded from the correction target. In other words, it is used as the determination condition directly.

[0230] On the other hand, for position and size, the number of changes exceeded the change number threshold, making them candidates for correction. For size, the change rate exceeded the change rate threshold, making it a determination condition as it is. On the other hand, for position, the change rate was below the change rate threshold, making it excluded from the conditions for determining whether the subject has changed.

[0231] As described above, the determination condition modification unit 122 modifies the conditions for determining whether or not a subject has changed based on the switching rate R calculated from the past selection history. The modified condition information is stored in the storage unit 18. The display switching determination unit 116 determines whether or not a subject has changed based on the condition information stored in the storage unit 18, and determines whether or not the display needs to switch to the subject selection screen.

[0232] As described above, the imaging device of this embodiment automatically optimizes the conditions for determining whether a subject has changed. This prevents unnecessary display switching. Furthermore, since the settings are automatically made, convenience can be improved.

[0233] [Fourth embodiment]

[0234] The imaging device of this embodiment is configured to allow the user to adjust the threshold used to determine whether a subject has changed. Specifically, the user can freely increase or decrease the threshold setting. This adjustment is performed within a predetermined settings screen.

[0235] Figure 21 This is a diagram showing an example of a screen for setting an adjustment threshold.

[0236] Figure 21 The example of adjusting the thresholds of the respective options of the position, size, and phase difference of the subject as the setting of the variation characteristics is shown. This setting screen is called from the menu screen, for example.

[0237] like Figure 21 As shown, in the setting screen of this example, for each option of "position", "size" and "phase difference", an adjustment lever for the change characteristics is displayed separately. The adjustment lever is configured to be adjustable in 5 levels on the positive and negative sides with "0" as the center. If the lever is set to the "0" position, the threshold is set to the default value. If the lever is moved from the "0" position to the positive side, the threshold is increased in stages by a specified amount of change. On the other hand, if the lever is moved from the "0" position to the negative side, the threshold is lowered in stages by a specified amount of change. Therefore, for example, in the case of increasing the response (sensitivity) (in the case of determining that the change is made with a smaller amount of change), the lever is moved from the "0" position to the negative side. On the contrary, in the case of reducing the response (in the case of determining that the change is made with a larger amount of change), the lever is moved from the "0" position to the positive side.

[0238] For example, to increase the response to changes in position and decrease the response to changes in size and phase difference, move the "position" lever to the negative side and the "size" and "phase difference" levers to the positive side. This lowers the threshold for position relative to the default value, while increasing the threshold for size and phase difference relative to the default value.

[0239] The set information is stored in the storage unit 18. The display switching determination unit 116 determines whether the subject has changed based on the set information, and determines whether it is necessary to switch to the subject selection screen.

[0240] As described above, by adopting a configuration in which the user can freely adjust the threshold setting, it is possible to appropriately switch the display according to the imaging scene, etc. This further improves operability.

[0241] In addition, while this embodiment illustrates the user manually adjusting the threshold setting, a configuration in which the threshold is automatically adjusted can also be employed, similar to the imaging device of the third embodiment described above. For example, a configuration can be employed in which a switching rate is calculated for each option, and for options whose calculated switching rate is below a threshold (switching rate threshold), the threshold is increased by a predetermined amount. Alternatively, a configuration can be employed in which the threshold is varied based on the switching rate. For example, a configuration can be employed in which the threshold is increased as the switching rate decreases.

[0242] [Fifth embodiment]

[0243] In the imaging device of the first embodiment described above, a configuration is adopted in which one screen is divided into four equal parts as a subject selection screen, and a subject serving as a candidate for enlargement is displayed in each area in an enlarged manner.

[0244] In the imaging device of this embodiment, a whole image is displayed in one of the display areas divided into four equal parts. The whole image is an image showing the entire image captured by the imaging unit 14 and is displayed during normal live view.

[0245] Figure 22 This is a diagram showing an example of a subject selection screen displayed on the imaging device of this embodiment. Figure 22 An example of dividing one screen into four equal parts is shown.

[0246] like Figure 22 As shown, in the screen divided into four equal parts, the entire image is displayed in the fourth display area A4 in the lower right corner.

[0247] At this point, the enlargement candidate selection unit 114 selects three subjects as enlargement candidates in order of priority. Of the three selected subjects, the subject with the highest priority is displayed in the first display area A1, the subject with the second highest priority is displayed in the second display area A2, and the subject with the third highest priority is displayed in the third display area A3.

[0248] The display moving image data generating unit 118 generates Figure 21 The moving image data having the structure shown is moving image data for switching subjects (moving image data for a subject selection screen).

[0249] As described above, by displaying the entire image in a portion of the subject selection screen, the entire image can be viewed from above, making it possible to easily select a subject.

[0250] [Modification]

[0251] Figure 23 It is a diagram showing a modified example of the display of the subject selection screen.

[0252] exist Figure 23 In the example shown, the subjects displayed in the display areas A1 to A3 are surrounded by frames f1 to f3 so as to be distinguishable in the overall image.

[0253] exist Figure 23 In the example shown, subject A, subject B, and subject C in the entire image are selected as magnification candidates and are magnified and displayed in display areas A1 to A3, respectively. Figure 23 As shown, in the entire image displayed in the display area A4, subjects A, B, and C are surrounded and displayed by frames f1 to f3, respectively. Each frame f1 to f3 is displayed in the same range as the range displayed in the display areas A1 to A3 in an enlarged manner.

[0254] exist Figure 23 In the example shown, the following structure is set: further, in each display area A1 to A3, the periphery is surrounded by a frame F1 to F3. Each frame F1 to F3 is displayed in a different color. The frames f1 to f3 displayed in the overall image are also displayed in a different color. The color of each frame f1 to f3 is set to the same color as the color of the frame F1 to F3 of the corresponding display area A1 to A3. For example, the color of the frame F1 surrounding the first display area A1 is set to red, the color of the frame F2 surrounding the second display area A2 is set to blue, and the color of the frame F3 surrounding the third display area A3 is set to yellow. At this time, the subject displayed in the first display area A1 (in Figure 23 In the example of the subject A), the color of the frame f1 is set to red. Also, the subject (in the second display area A2) is surrounded by the frame f1. Figure 23 In the example of the subject B, the color of the frame f2 is set to blue. Also, the subject displayed in the third display area A3 (in Figure 23 In the example of the subject C, the color of the frame f3 is set to yellow. In the entire image, the subject surrounded by the frames f1 to f3 ( Figure 23 In the case of the example, subjects A to C) are examples of corresponding subjects.

[0255] As described above, in the overall image, the subject corresponding to each display area (corresponding subject) is surrounded by a frame of the same color as the frame of each display area. This allows the relationship between the subject displayed in each display area and the subject in the overall image to be understood at a glance, thereby further improving convenience.

[0256] Figure 24 This is a diagram showing still another modified example of the display of the subject selection screen.

[0257] In this example, a configuration is set such that even subjects not displayed in display areas A1 to A3 are surrounded by a frame fx in the entire image. The color of this frame fx is set to a different color from the colors of the frames f1 to f3 surrounding the subjects displayed in display areas A1 to A3. For example, it is set to black.

[0258] exist Figure 24 In the example shown, subjects A, B, and C are displayed in display areas A1 to A3, while subjects D, E, and F are not displayed in display areas A1 to A3. In this case, subjects A, B, and C are displayed surrounded by frames f1 to f3 of the same color as frames F1 to F3 surrounding the corresponding display areas A1 to A3. On the other hand, subjects D, E, and F are displayed surrounded by frames fx of a different color (e.g., black) than frames f1 to f3.

[0259] In this example, the subject can be selected even in the entire image. In this case, for example, by touching a position surrounded by frames f1 to f3 and fx in the entire image, a subject displayed in magnified form can be selected.

[0260] As described above, by adopting a configuration in which a subject can be selected also from the entire image, convenience and operability can be further improved.

[0261] In addition, Figure 23 and Figure 24 In the example shown, frames f1-f3 surrounding each subject within the entire image are displayed within the same range as the range displayed in the magnified display areas A1-A3. The display range (size) of frames f1-f3 does not necessarily need to be the same as the range displayed in the magnified display areas A1-A3. Any format is sufficient as long as the subject selected as a candidate for magnification can be visually recognized.

[0262] [Sixth embodiment]

[0263] The imaging device of this embodiment is configured so that the user can manually switch the display from a single magnified image to a subject selection screen. That is, while the display automatically switches from a single magnified image to a subject selection screen in the above-mentioned embodiment, the imaging device of this embodiment also allows manual switching. This further improves convenience and operability.

[0264] The CPU 24 switches the display from the single magnified display to the subject selection screen in response to a switching instruction from the operation unit 22. The switching instruction is provided by, for example, providing a dedicated button on the operation unit 22 and operating the button.

[0265] Similarly, a configuration may be provided in which the user can manually switch the display from the subject selection screen to the single magnified display. In this case, the CPU 24 also switches the display from the subject selection screen to the single magnified display in response to the switching instruction from the operation unit 22. Note that the single magnified display here is performed for the subject that was displayed in the single magnified display before switching to the subject selection screen. In other words, the original single magnified display is restored.

[0266] [Seventh embodiment]

[0267] In the imaging device of the first embodiment, a configuration is provided in which a predetermined priority is calculated for each subject extracted from an image based on the position, size, and phase difference change of the subject, and subjects are selected as candidates for enlargement in order of the calculated priority.

[0268] The imaging device of this embodiment is configured so that the user can set the conditions for calculating priority, that is, the conditions for selecting subjects as candidates for magnification. Specifically, the user can adjust the coefficients w1, w2, and w3 within a predetermined range in the formula for calculating priority. The imaging device of this embodiment is configured so that this adjustment is performed on a predetermined settings screen.

[0269] Figure 25 This is a diagram showing an example of a screen for adjusting coefficients of a calculation expression for calculation priority.

[0270] Figure 25 The example of adjusting the coefficients of each option of the position, size, and phase difference of the subject as the setting of the selection characteristics of the subject candidate for enlargement is shown. This setting screen is called from the menu screen, for example.

[0271] like Figure 25 As shown in FIG. 1 , in the setting screen of this example, an adjustment lever for selecting characteristics is displayed for each of the options "Position," "Size," and "Phase Difference." The adjustment lever is configured to be adjustable in stages within the range of "0" to "10."

[0272] For each option, if the lever is set to "0", the coefficient value is set to 0. By gradually raising the lever position, the coefficient increases step by step at a predetermined rate of change. The lever is set to the maximum value at "10".

[0273] When the lever is set to the "0" position, the option is excluded from the selection determination. In other words, the coefficient value is set to 0 at this time, so it is excluded from the selection determination.

[0274] Figure 25In the example shown, the lever is set to "10" for the "Position" option, "5" for the "Size" option, and "0" for the "Phase Difference" option. In this case, the selection of magnification candidates is based on the position and size of the subject, with phase difference excluded from the selection. Furthermore, the selection prioritizes position over size.

[0275] The set information (information of coefficients w1 to w3) is stored in the storage unit 18. The enlargement candidate selection unit 114 calculates the priority based on the set information and selects a subject as an enlargement candidate.

[0276] As described above, by allowing the user to set the conditions for calculating the priority (conditions for selecting subjects as enlargement candidates), subjects that meet the user's intention can be selected as enlargement candidates, thereby further improving convenience and operability.

[0277] [Eighth embodiment]

[0278] In the imaging device of the seventh embodiment described above, a configuration is adopted in which the user manually sets the conditions for calculating the priority.

[0279] In the imaging device of this embodiment, the conditions for calculating the priority are automatically set based on the user's past tendency to select subjects. In other words, the values ​​of the coefficients w1 to w3 are automatically optimized.

[0280] In the imaging device of this embodiment, the values ​​of coefficients w1 to w3 are set based on the switching rate. Specifically, the coefficients for each option increase as the switching rate for each option increases. A high switching rate indicates that the subject is frequently selected based on the change in that option. Therefore, in this case, the coefficients are set to large values. On the other hand, a low switching rate indicates that the subject is rarely selected based on the change in that option. Therefore, in this case, the coefficients are set to small values.

[0281] As described above, by automatically correcting the conditions for calculating the priority, a subject that meets the user's intention can be automatically selected as an enlargement candidate, thereby further improving convenience and operability.

[0282] [Ninth embodiment]

[0283] In the imaging device of the above embodiment, when the display switches from the single magnified display to the subject selection screen, if no subject is selected within a predetermined time, the display returns to the original single magnified display.

[0284] The subject is not selected, and it is considered that the display is not switched as expected by the user.

[0285] In the imaging device of this embodiment, if the display returns to the original single magnified display without selecting a subject, further display switching under the same conditions is prohibited for a predetermined period of time. For example, if the display switches from the single magnified display to the subject selection screen based on a change in position, and if no subject selection is made within a predetermined period of time, the display returns to the single magnified display, and further display switching due to the position change is prohibited for a predetermined period of time. The following describes the structure for implementing this processing.

[0286] Figure 26 This is a block diagram showing an example of the function implemented by the CPU when display switching due to a change in a specific option is prohibited.

[0287] like Figure 26 As shown, when no subject is selected, the CPU 24 further functions as a determination option setting unit 130 in order to prohibit switching of the display due to a change in a specific option.

[0288] The determination option setting unit 130 individually turns on / off the options used by the display switching determination unit 116 to determine whether or not a subject has changed, based on the determination results of the display switching determination unit 116. For example, for each option, a determination flag is set to "1" if it is used for determination, and to "0" if it is not used for determination. The determination flag is normally set to "1" but is set to "0" in specific circumstances. These specific circumstances are when no subject has been selected. In this case, the determination flag is set to "0" for the option that triggers the display switch, that is, for the option for which a change greater than a threshold value has been measured. For example, if the display switches from a single magnified display to a subject selection screen based on a change in position, and no subject selection is made within a specified time, the position determination flag is set to "0" after returning to the single magnified display. For options with the determination flag set to "0," the determination flag returns to "1" after a predetermined pause time has elapsed. The pause time is an example of the second time.

[0289] The display switching determination unit 116 determines whether the subject has changed, and whether it is necessary to switch to live view display, based on the determination use flag set by the determination option setting unit 130. Specifically, based on changes in the options whose determination use flags are set to "1," the determination of whether the subject has changed and whether it is necessary to switch to live view display is determined. For example, at a certain determination timing, the determination use flag for position is set to "0," the determination use flag for size is set to "1," and the determination use flag for phase difference is set to "1." In this case, the determination of whether the subject has changed due to changes in position is eliminated, and the determination of whether the subject has changed is based solely on changes in size and phase difference.

[0290] As described above, when no subject is selected on the subject selection screen, the option that triggers the display switch is set to "0" for a predetermined pause time after returning to single magnified display. While the determination flag is "0," that option is excluded from the determination of whether or not the subject has changed. In other words, display switching caused by such changes is prohibited.

[0291] As described above, by prohibiting the display from switching under the same conditions for a predetermined time when no subject is selected, it is possible to prevent the display from switching unexpectedly by the user, thereby further improving operability.

[0292] [Modification]

[0293] In the above embodiment, when no subject selection is made, the option that triggers display switching is excluded from the determination of whether the subject has changed during a predetermined pause time. Additionally or alternatively, a configuration may be employed in which the subject that triggers display switching, i.e., the subject that has been determined to have changed, is excluded from the determination of whether the subject has changed during the predetermined pause time. In this case, it is assumed that the user has not selected the subject for magnification. Therefore, by prohibiting display switching caused by the subject, display switching more in line with the user's intent can be achieved.

[0294] In this example, the CPU 24 sets a determination use flag for each subject. If no subject is selected, the determination use flag is set to "0" for the subject that triggers the display switching, i.e., the subject that is determined to have changed, after returning to single magnified display, and then for a predetermined pause time.

[0295] [Other embodiments]

[0296] [Image magnification ratio for single magnification display]

[0297] The magnification ratio of the image during single-magnification display may be a fixed value or may be arbitrarily changeable by the user, or may be automatically changed according to imaging conditions, etc.

[0298] As an example of a structure that automatically changes the magnification ratio, for example, a structure can be employed that changes the magnification ratio based on the depth of field during shooting. In this case, the magnification ratio is increased when the depth of field is shallow. A shallow depth of field requires fine focus adjustment. Therefore, in this case, the magnification ratio is increased, allowing the user to confirm the focus status in more detail. On the other hand, when the depth of field is deep, fine focus adjustment is deemed unnecessary, so the magnification ratio is reduced, allowing a bird's-eye view of the image.

[0299] For example, the CPU 24 obtains information necessary to calculate the depth of field and, if the depth of field is set to be narrower than a specific value, performs processing to increase the magnification ratio. For example, the magnification ratio is increased by a predetermined magnification ratio relative to a standard magnification ratio. Alternatively, a configuration may be employed in which the magnification ratio is changed in stages according to the depth of field.

[0300] As another example of a structure that automatically changes the magnification ratio, for example, a structure can be adopted that changes the magnification ratio based on the value of the high-frequency component in the magnified area. In this case, the structure is set as follows: when the value of the high-frequency component in the magnified area is low, the magnification ratio is reduced. The low value of the high-frequency component in the magnified area means that there are few edge components in the area. When there are few edge components, it is difficult to determine whether the focus is in place. Therefore, the structure is set as follows: when the value of the high-frequency component in the magnified area is low, that is, when there are few edge components, the magnification ratio is reduced, and the image can be checked from a bird's-eye view. In other words, more information is read, making focusing easier.

[0301] For example, the CPU 24 calculates the high-frequency component within the magnified area and, if the high-frequency component is lower than a specific value, performs processing to reduce the amplification factor. The amplification factor reduction process may, for example, be performed by reducing the amplification factor by a predetermined factor relative to a standard amplification factor. Alternatively, a configuration may be employed in which the amplification factor is changed in stages according to the value of the high-frequency component.

[0302] In the subject selection screen, the magnification factor of the image displayed in each display area can also be changed in accordance with the depth of field, the value of the high-frequency component, and the like.

[0303] Furthermore, the magnification of each display area can be manually adjusted individually. For example, if the display unit 16 includes a touch panel, the magnification can be changed by zooming in or out in each display area.

[0304] [Display destination for live view]

[0305] In the above embodiment, the live view is displayed on the display unit 16 provided in the device body, but it can also be displayed on an external display device connected to the device body via the connection unit 29. Furthermore, it can also be displayed on both.

[0306] [Magnified object]

[0307] In the above embodiment, a human face is extracted as a subject and the face area is magnified and displayed. However, the subject to be magnified is not limited to this. The entire person can also be magnified. Furthermore, in addition to people, a wide range of moving objects can be used as subjects.

[0308] The user may be configured to set the type of the subject to be photographed. For example, the subject to be photographed may be limited to a person's face or to the entire person.

[0309] [Selection of the magnified object when the tracking magnification display function is turned on]

[0310] In the above-described embodiment, a configuration may be employed in which a user manually sets a magnified subject (an example of a specific subject) when the tracking zoom display function is enabled. This processing may also be performed automatically. In this case, CPU 24 automatically selects a subject to be magnified from multiple subjects extracted from the image based on predetermined selection criteria. As an example of a selection criterion, a method may be employed to select the subject closest to the center of the image as the magnified subject. Alternatively, a method may be employed to select the subject with the shortest subject distance as the magnified subject. Alternatively, a method may be employed to select the subject with the smallest amount of defocus as the magnified subject. Alternatively, a method may be employed to select the subject in focus as the magnified subject. When selecting a focused subject as the magnified subject, if there are multiple focused subjects, a method may be employed to select the subject to be magnified by combining other criteria (e.g., the subject closest to the center of the image, the subject with the shortest subject distance, etc.). Furthermore, a configuration may be adopted in which a plurality of selection criteria are determined and a user can arbitrarily select a selection criterion to be used.

[0311] Furthermore, in the above-described embodiment, the user can manually select a subject to be magnified by surrounding the face of each subject extracted from the entire image with a bounding box, selecting the displayed bounding box, and selecting the subject to be magnified. The method for selecting a subject to be magnified is not limited to this. For example, a method can be employed in which subjects that are candidates for magnification are extracted and displayed in a list for user selection.

[0312] [Selection of subjects as candidates for enlargement]

[0313] As the subjects to be enlarged, the same number of subjects as the number of subjects displayed on the subject selection screen (the number of display areas) are selected. The number of subjects displayed on the subject selection screen is two or more.

[0314] If the number of subjects extracted from the image is less than the number of subjects displayed on the subject selection screen, the selection process for subjects as magnification candidates is not performed. In this case, a portion of the display area on the subject selection screen is set to a blank display (e.g., a black screen). Alternatively, the initial subject selection screen, described later, may be displayed.

[0315] Furthermore, not all subjects within the image are considered candidates for magnification. For example, it is possible to exclude subjects from at least one of the following: position, size, and distance (phase difference) of the subject within the image. For example, regarding the position of the subject within the image, subjects located at a distance greater than a specified distance from the center of the image can be excluded. Furthermore, regarding the size of the subject within the image, subjects smaller than a specified size can be excluded. Furthermore, regarding the distance to the subject, subjects located at a distance greater than a specified distance can be excluded.

[0316] Furthermore, it is also possible to select only pre-registered subjects as targets. For example, in the case of a person, personal authentication is performed based on the image, and the enlargement candidate is selected from pre-registered subjects.

[0317] Furthermore, in the above-described embodiment, a structure is provided in which a subject as a candidate for enlargement is selected based on a priority calculated based on a predetermined calculation formula, but the method of selecting a subject as a candidate for enlargement is not limited thereto. For example, a structure can be provided in which selection is made based on at least one of the position, size, and distance of the subject within the image. For example, a structure can be provided in which, when selecting based on position, subjects close to the center of the screen are selected in sequence. Furthermore, for example, a structure can be provided in which, when selecting based on size, subjects occupying a large area within the image are selected in sequence. Furthermore, for example, a structure can be provided in which, when selecting based on distance to the subject, subjects close to the imaging device are selected in sequence. Furthermore, selection can be made by combining these conditions.

[0318] [Display of the subject selection screen]

[0319] In the above embodiment, a configuration is employed in which a single screen is divided into four equal parts as a subject selection screen, with subjects selected for enlargement displayed in each display area. However, the number of subjects selected for enlargement and the manner in which they are displayed are not limited to this configuration. For example, a configuration in which the number of subjects selected for enlargement and the manner in which they are displayed are switched depending on the number of subjects extracted as enlargement candidates may also be employed. Furthermore, a configuration in which the number of subjects selected for enlargement is arbitrarily selected by the user may also be employed.

[0320] Furthermore, a configuration can be provided that automatically switches the number of displays depending on the display destination. For example, when displaying on an external display device, the number of displays can be increased compared to the display unit included in the device body. For example, while the display unit included in the device body has a display capacity of 4 (2×2), the external display device can have a display capacity of 9 (3×3).

[0321] Furthermore, in the above-described embodiment, a configuration is adopted in which one screen is divided equally to provide display areas for a plurality of subjects. However, the method of dividing the screen is not limited to this. Figures 27 to 29 This is a diagram showing another example of screen division in the subject selection screen. Figures 27 to 29 The example shown is an example of displaying the entire image larger than the subject that is a candidate for enlargement. Figure 27 This is an example of a screen with four display areas A1 to A4. The entire image is displayed in the large display area A1, located to the left of the center of the screen, and the enlarged candidate subjects are displayed in the smaller display areas A2 to A4, arranged vertically along the right side of the screen. Figure 28 This is an example of a screen with four display areas A1 to A4. The entire image is displayed in the large display area A1 located at the upper center of the screen, while the enlarged candidate subjects are displayed in the smaller display areas A2 to A4 arranged side by side along the lower edge of the screen. Figure 29 This is an example of six display areas A1 to A6 set on one screen. The entire image is displayed in the large display area A1 set in the upper left corner of the screen, and the enlarged candidate subjects are displayed in the smaller display areas A2 to A6 set side by side along the right and bottom sides of the screen.

[0322] Furthermore, a configuration may be adopted in which subjects serving as candidates for enlargement are sequentially switched and displayed on the subject selection screen. Figure 30 1 and 2 are diagrams showing an example of screen transition when switching and displaying a candidate subject for enlargement. Figure 30 (A) shows an example of the display of the first subject selection screen after switching to the subject selection screen. Figure 30 (B) shows an example of the display of the subject selection screen after a predetermined time has passed. Figure 30 (A) and Figure 30 As shown in (B), the subjects of the enlargement candidates displayed in the display areas A1 to A3 are switched. The switching is performed periodically. The user can arbitrarily set the switching period.

[0323] As described above, by switching the subjects as enlargement candidates displayed in the display areas A1 to A3 , the number of subjects that can be displayed as enlargement candidates can be increased.

[0324] In this example, the subject switching is performed periodically, but the subject to be displayed can also be switched according to a user instruction. For example, if the display unit includes a touch panel, the display can be switched by sliding or tapping the screen.

[0325] Furthermore, in this example, the screen is divided into a plurality of areas, and the image of the subject (enlarged image) displayed in each area is sequentially switched. However, a configuration in which the images are sequentially switched in one screen may be adopted.

[0326] [Subject Distance]

[0327] In the imaging device of the above embodiment, the phase difference between each subject is calculated to obtain subject distance information. However, the method of obtaining subject distance information is not limited to this. The subject distance information can also be obtained by using the measurement results of other known distance measurement mechanisms.

[0328] [Other examples of the operation unit]

[0329] In the above embodiment, operations such as selecting a subject to be magnified are performed using the touch panel included in the display unit 16. However, operations such as selecting a subject to be magnified can also be performed using key operations, lever operations, or dial operations. For example, a configuration can be adopted in which, when a subject to be magnified is selected using key operations, lever operations, or dial operations on a subject selection screen, a selection frame is displayed on the screen, and the frame is moved using key operations, lever operations, or dial operations to select the subject to be magnified.

[0330] [Tracking zoom display function on / off]

[0331] In the above embodiment, the tracking zoom display function is configured to be usable only in manual focus mode. However, it can also be configured to be usable in other focus modes. For example, in automatic focus mode, it can be used to select a subject to focus on. The same effect is also achieved in semi-automatic focus mode.

[0332] [Example of an imaging device]

[0333] The imaging device to which the present invention is applied includes not only imaging devices dedicated to video recording, such as so-called video cameras and movie cameras, but also imaging devices embedded in smartphones, personal computers, and the like.

[0334] [Hardware structure of the processing unit]

[0335] In the present invention, the hardware structure of the processing unit that performs each process is implemented by various processors. Among these various processors, there are general-purpose processors such as CPUs and / or GPUs (Graphic Processing Units) that execute programs and function as various processing units, FPGAs (Field Programmable Gate Arrays), and processors whose circuit structures can be changed after manufacturing, such as programmable logic devices (PLDs) and ASICs (Application Specific Integrated Circuits), which are processors with circuit structures specifically designed to perform specific processes, such as dedicated circuits. Programs and software have the same meaning.

[0336] One processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same type or different types. For example, one processing unit may be composed of multiple FPGAs, or a combination of a CPU and an FPGA. Furthermore, multiple processing units may be composed of one processor. As an example of a plurality of processing units composed of one processor, first, there is a method in which a processor is composed of a combination of one or more CPUs and software, as represented by computers used in clients, servers, etc., and the processor functions as multiple processing units. Secondly, there is a method in which a processor that realizes the functions of the entire system including multiple processing units is used by one IC (Integrated Circuit) chip, as represented by a system on chip (SoC). In this way, as a hardware structure, various processing units are composed of one or more of the above-mentioned various processors.

[0337] Explanation of symbols

[0338] 10-imaging device, 12-imaging optical system, 14-imaging unit, 16-display unit, 18-storage unit, 20-connection unit, 22-operation unit, 24-CPU, 26-ROM, 28-RAM, 29-connection unit, 110-object extraction unit, 112-change measurement unit, 112A-position change measurement unit, 112B-size change measurement unit, 112C-phase difference change measurement unit, 114-enlargement candidate selection unit, 116-display switching determination unit, 11 8-Dynamic image data generation unit for display, 120-Selection result recording unit, 122-Judgment condition correction unit, 130-Judgment option setting unit, A~F-Subject, A1~A6-Display area, BB-Bounding box, F1~F3-Frame, f1~f3-Frame, fx-Frame, S11~S20-Order for determining whether there is a change in the subject, S31~S46-Order for switching the live view display, S51~S57-Order for correcting the conditions for determining whether there is a change in the subject.

Claims

1. A camera device comprising a processor, The processor performs the following processing in the focusing mode: Extracting multiple subjects from the moving image data obtained by imaging. outputting first moving image data obtained by enlarging a region including a specific subject among the plurality of subjects to a display destination, When any one of the multiple subjects in the dynamic image data changes, the second dynamic image data in which the areas of the multiple subjects are magnified is output to the display destination instead of the first dynamic image data. When the selection of the subject is indicated, the third dynamic image data in which the area including the selected subject is magnified is output to the display destination instead of the second dynamic image data.

2. The imaging device according to claim 1, wherein After the processor outputs the second moving image data to the display destination instead of the first moving image data, if selection of the subject is not instructed within a first time, the processor outputs the first moving image data to the display destination instead of the second moving image data.

3. The imaging device according to claim 1 or 2, wherein: The plurality of subjects enlarged in the second moving image data include at least the subject that has changed.

4. The imaging device according to claim 1 or 2, wherein: The processor further performs the following processing: accepting the setting of a condition for determining whether the subject has changed.

5. The imaging device according to claim 1 or 2, wherein: The processor further performs the following processing: based on the history of instructions for selecting the subject, modifying the condition for determining whether the subject has changed. The imaging device according to claim 4 , wherein: The condition includes at least one of an option for determination and a threshold value for determination.

7. The imaging device according to claim 1 or 2, wherein: The processor determines whether the subject has changed according to a change in at least one of a position, a size, and a distance from the subject.

8. The imaging device according to claim 1 or 2, wherein: The processor outputs an image obtained by sequentially switching the enlarged images of the plurality of subjects to the display destination as the second moving image data.

9. The imaging device according to claim 1 or 2, wherein: The processor outputs an image including the entire image of the moving image data in addition to the enlarged images of the plurality of subjects to the display destination as the second moving image data.

10. The imaging device according to claim 9, wherein: The processor performs processing such that, in the second moving image data, the enlarged images are surrounded by frames of different colors, and corresponding subjects in the entire image are surrounded by frames of the same color.

11. The imaging device according to claim 1 or 2, wherein: The processor further performs the following processing: selecting an object to be enlarged in the second moving image data from a plurality of the objects extracted from the moving image data.

12. The imaging device according to claim 11, wherein: The processor further performs a process of accepting setting of a condition for selecting an object to be enlarged in the second moving image data.

13. The imaging device according to claim 11, wherein: The processor further performs processing for correcting a condition for selecting an object to be enlarged in the second moving image data based on a history of instructions for selecting the subject.

14. The imaging device according to claim 11, wherein The processor selects an object to be enlarged in the second moving image data based on a change in at least one of a position, a size, and a subject distance.

15. The imaging device according to claim 1 or 2, wherein: After the processor outputs the second dynamic image data to the display destination, if no selection of the subject is indicated within the first time, the subject determined to have changed when the first dynamic image data is replaced by the second dynamic image data is excluded from the objects for detecting changes during the second time after the first time.

16. The imaging device according to claim 1 or 2, wherein: In the case of determining whether the subject has changed for a plurality of options, if the processor does not indicate the selection of the subject within a first time after outputting the second dynamic image data to the display destination, the option determined to have changed when the first dynamic image data is replaced by the second dynamic image data is excluded from the conditions for detecting the change during a second time after the first time.

17. The imaging device according to claim 1 or 2, wherein: The processor increases the magnification ratio of the subject during enlarged display when setting is made such that the depth of field during imaging becomes narrower than a specific value.

18. The imaging device according to claim 1 or 2, wherein: The processor reduces a magnification ratio of the subject during magnified display when a value of a high-frequency component in the magnified area is lower than a specific value.

19. The imaging device according to claim 1 or 2, wherein: The display destination is a display unit included in the device body and / or an external display device connected via a connection unit included in the device body.

20. An image processing method, wherein: Extracting multiple subjects from dynamic image data obtained by camera, outputting first moving image data obtained by enlarging a region including a specific subject among the plurality of subjects to a display destination, When any one of the multiple subjects in the dynamic image data changes, the second dynamic image data in which the areas of the multiple subjects are magnified is output to the display destination instead of the first dynamic image data. When the selection of the subject is indicated, the third dynamic image data in which the area including the selected subject is magnified is output to the display destination instead of the second dynamic image data.

21. A non-volatile computer-readable recording medium storing a program for causing a computer to execute the following processing: Extracting multiple subjects from dynamic image data obtained by camera, outputting first moving image data obtained by enlarging a region including a specific subject among the plurality of subjects to a display destination, When any one of the multiple subjects in the dynamic image data changes, the second dynamic image data in which the areas of the multiple subjects are magnified is output to the display destination instead of the first dynamic image data. When the selection of the subject is indicated, the third dynamic image data in which the area including the selected subject is magnified is output to the display destination instead of the second dynamic image data.

Images