Imaging device, its control method, and program

Abstract

The present invention provides an imaging device that enables subject changes and exposure tracking while obtaining appropriate imaging effects without unnatural blur fluctuations when switching subjects using manual focus. [Solution] An imaging device 10 is equipped with a sensor unit 103 for capturing images and an AF processing unit 108 for adjusting the focal position during imaging, and is capable of manual focus by adjusting the focal position according to user operation, and is characterized by comprising: a subject detection unit 107 for detecting a subject from an image; an in-screen distance calculation unit 110 for calculating the depth of field that fits each detected subject; a control unit 102 for performing control based on a program diagram relating to exposure during imaging; and a modification unit that controls the modification of the program diagram when the focal position is adjusted by manual focus and it is determined that multiple subjects that fit within each of the calculated multiple depths of field satisfy predetermined conditions.

Description

Technical Field

[0005] , ,

[0001] The present invention relates to an imaging device capable of manually adjusting the focus position of a subject, a control method thereof, and a program.

Background Art

[0002] When performing photometry control in a digital camera or the like, a method is known in which the screen is divided into grid-like blocks, the luminance value in each block is acquired, and exposure control is performed based on an evaluation value obtained from the average value of the acquired luminance values. An exposure correction step for converging the captured image to an appropriate luminance value is obtained from this evaluation value, and the exposure can be appropriately maintained by feeding back to exposure control such as aperture, shutter speed, and ISO.

[0003] In addition, there is also a method of detecting the face or head of a subject in a captured image and calculating an exposure correction value that results in an appropriate luminance for each obtained face or head, and it is possible to calculate the same exposure correction value for a plurality of detected subjects. Also, when a plurality of subjects are present within the angle of view, there are cases where the subjects are arranged at positions shifted in the depth direction. In such cases, a method of displaying the depth information of the subjects so as to smoothly move the focus position between the subjects using manual focus has also been disclosed (see, for example, Patent Document 1).

Prior Art Documents

[0006] Next, consider the case where the focus shifts from a darker subject in the background (Figure 4(a)) to a brighter subject in the foreground. In this case, since the main subject shifts to the brighter subject in the foreground (Figure 4(b)), the aperture moves to a smaller aperture to properly adjust the exposure, and ultimately the depth of field deepens as shown in Figure 4(c). As a result, the effect of blurring the subject after the focus shift is reduced. In other words, when focus shift and exposure tracking operate simultaneously, moving the aperture to a smaller aperture to adjust the exposure to the brighter subject in the foreground ultimately deepens the depth of field, and the effect of blurring the subject after the focus shift is halved.

[0007] The object of the present invention is to provide an imaging device, a control method, and a program that enable changes in the subject being focused on and stable exposure tracking, while obtaining an appropriate imaging effect without unnatural blur fluctuations when switching subjects to focus on with manual focus. [Means for solving the problem]

[0008] To achieve the above objective, one aspect of the present invention is an imaging device that includes an imaging means for capturing an image and a focusing means for adjusting the focal position during imaging, and is capable of manual focusing, where the focal position can be manually adjusted according to user operation, and is characterized by comprising: a detection means for detecting a subject from the image; a calculation means for calculating the depth of field that includes each detected subject; a control means for performing control based on a program diagram relating to exposure during imaging; and a modification means for changing the program diagram when the focal position is adjusted by manual focusing and it is determined that a plurality of subjects that fit within each of the calculated plurality of depths of field satisfy predetermined conditions. [Effects of the Invention]

[0009] According to the present invention, when switching subjects to focus on using manual focus, it is possible to change the subject to focus on and maintain stable exposure tracking while obtaining appropriate imaging effects without unnatural blur fluctuations. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of an imaging device according to an embodiment of the present invention. [Figure 2] This is an overall flowchart illustrating the operation of the imaging device according to an embodiment of the present invention. [Figure 3] This is a flowchart showing the process for changing the program diagram in an imaging device according to an embodiment of the present invention. [Figure 4] This is a diagram illustrating the challenges of conventional technology. [Figure 5] This is an explanatory diagram of a distance map. [Figure 6] This is a diagram illustrating the task. [Figure 7] This is a program diagram illustrating the problem. [Figure 8] This is an explanatory diagram of an image obtained by integrating brightness values ​​in a grid pattern, and the corresponding subject area. [Figure 9] This is an explanatory diagram of a program diagram of an embodiment of the present invention. [Figure 10] This is an explanatory diagram illustrating the relationship between depth of field and the corresponding aperture. [Figure 11] This is an explanatory diagram of the program diagram for this embodiment. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described in detail below with reference to the drawings. However, the configurations described in the following embodiments are merely illustrative, and the scope of the present invention is not limited by the configurations described in the embodiments. Also, identical or similar configurations in the attached drawings are given the same number, and redundant explanations are omitted. The term "depth of field" as described below refers to the range within the distance range from near to far, centered on the focal distance, where the image is in focus and appears sharp. The configuration and operation of the imaging device 10 according to the embodiment of the present invention will be described below.

[0012] (Figure 1: Block diagram of imaging device 10) Figure 1 is a block diagram showing an example configuration of the imaging device 10. In this embodiment, a digital camera is used as an example of the imaging device 10, but the imaging device 10 is not limited to this. For example, the imaging function of various electronic devices such as smartphones and tablets can also be implemented in the imaging device 10.

[0013] (Configuration of imaging device 10) The imaging device 10 includes an operation unit 101, a control unit 102, a sensor unit 103, an A / D conversion unit 104, an image processing unit 105, a brightness calculation unit 106, a subject detection unit 107, an in-screen distance calculation unit 110, an AF processing unit 108, and an AE processing unit 109. The imaging device 10 further includes a memory unit 111, an encoder unit 112, an image recording unit 113, an external connection unit 114, and a display unit 115. Each component of the imaging device 10 is connected to the control unit 102 so as to be able to communicate the required information.

[0014] (Operation unit 101: Control unit 102) The operation unit 101 is for the operator to input various instructions and is composed of switches, buttons, etc. The operation unit 101 includes a shutter switch and a touch sensor, and this touch sensor is a sensor that can be operated by touching the display unit 115. The control unit 102 is composed of a CPU 121, a non-volatile memory 122, and a RAM 123. The CPU 121 can realize various functions required in this embodiment by executing the program stored in the non-volatile memory 122. As an example, the control unit 102 controls the operations of each part shown in FIG. 1 according to the instructions given from the operation unit 101. Also, the control unit 102 performs various controls based on the program diagram related to the exposure at the time of imaging described later. Furthermore, the operation unit 101 includes an operator that can perform a manual focus (MF) operation, etc., and the optical system and the control unit 102 are constructed to perform an imaging operation according to the MF operation. That is, the imaging device 10 is configured to be able to manually adjust the focus position according to the user operation.

[0015] (Sensor unit 103: A / D conversion unit 104: Image processing unit 105) The sensor unit 103 receives the light incident through an optical system including a mechanical mechanism 1091 such as a lens 1081 and an aperture, and outputs an analog image signal by outputting an electric charge corresponding to the amount of the light. Therefore, the sensor unit 103, the lens 1081, and the mechanical mechanism 1091 constitute an imaging unit (imaging means) in the imaging device 10. The sensor unit 103 is realized by an imaging element such as a CMOS image sensor in which pixels are arranged in a matrix form, for example. The A / D conversion unit 104 performs sampling, gain adjustment, A / D conversion, etc. on the analog image signal output from the sensor unit 103, and outputs a digital image signal. The image processing unit 105 performs various image processes on the digital image signal output from the A / D conversion unit 104 and outputs it as a processed digital image signal. For example, the image processing unit 105 converts the digital image signal received from the A / D conversion unit 104 into a YUV image signal and outputs it.

[0016] (Luminance calculation unit 106: Subject detection unit 107) The luminance calculation unit 106 calculates a luminance evaluation value from the screen area using the image signal obtained from the image processing unit 105. The subject detection unit 107 (detection means) detects a subject using the digital image signal obtained from the image processing unit 105. The subject detected by the subject detection unit 107 is assumed to be a person, and the areas of the face and head within the screen are acquired. The following methods (1) to (3) are exemplified as subject detection methods. (1) A method of extracting an area by pattern matching from the contour shape of the human body. (2) A method of detecting important organs with features such as eyes, nose, and mouth and detecting the head area including them. (3) A known method such as a method using an algorithm learned by machine learning for the face area of a person, etc.

[0017] For example, in a method using machine learning, learning is performed by associating the concept of each granularity from the overall image of the object to the details as a hierarchical structure. When learning a person, it is possible to learn using person images including various races / / ages / genders / face orientations / hair. In the present embodiment, a person is described as a subject, but animals, stationary objects, etc. may also be used as subjects, and the subject detection method is not limited to the above (1) to (3), and other subject detection methods may be adopted.

[0018] (In-screen distance calculation unit 110: Figure 5: Explanation diagram of distance map) The in-screen distance calculation unit 110 (calculation means) calculates distance information from the subject. In the example of Figure , as shown in Figure 5(b), distance information is calculated for each area obtained by dividing the screen shown in Figure (a) into a grid pattern. In the present embodiment, it is assumed that a distance map is acquired from the defocus information obtained using imaging surface phase difference pixels. This is a case where distance measurement imaging surface phase difference pixels are incorporated in the sensor unit 103, and it is possible to acquire a "distance map" by obtaining the distance measurement result for each pixel simultaneously with the sensor unit 103 acquiring an imaging signal. The in-screen distance calculation unit 110 acquires distance information from each subject and also calculates the depth of field of each of the plurality of subjects. Specifically, each of the numbers "5 to 100" in Figure 5(b) indicates distance information.

[0019] (AF processing unit 108:AE processing unit 109) The AF processing unit 108 (focusing means) drives and controls the lens 1081 based on the image obtained by the image processing unit 105, and drives the focus lens to focus on the detected subject. In addition, if manual focus is selected, the user can freely control the position of the focus lens manually. The AE processing unit 109 calculates the difference from the appropriate brightness from the image obtained by the image processing unit 105, and controls the mechanical mechanism 1091 so that this difference is eliminated. The mechanical mechanism 1091 consists of an aperture, a shutter, etc. The aperture adjusts the amount of light by adjusting the opening of the aperture blades, and the shutter is located in front of the sensor unit 103, and the amount of light is adjusted by the time it opens and closes, thereby adjusting the exposure value.

[0020] (Memory section 111: Encoder section 112: Image recording section 113) The memory unit 111 temporarily stores image data being processed by the control unit 102, image processing unit 105, encoder unit 112, etc. The encoder unit 112 converts the format of the output digital image signal (image data) to a format such as JPEG and outputs it to the image recording unit 113. The image recording unit 113 records the format-converted image data output from the encoder unit 112 to a memory (not shown) within the imaging device 10 or to an external memory such as a memory card installed in the imaging device 10.

[0021] (External connection unit 114: Display unit 115) The external connection unit 114 connects to external devices such as an external monitor or a personal computer. For example, by connecting an external monitor to the external connection unit 114, the screen displayed on the display unit 115 can be displayed on the external monitor. The display unit 115 is composed of a liquid crystal display device or the like and displays images (digital image signals) output from the image processing unit 105.

[0022] The image processing unit 105, brightness calculation unit 106, subject detection unit 107, and in-screen distance calculation unit 110 may be implemented by the CPU 121 executing software, or by dedicated hardware such as an ASIC. In Figure 1, the image processing unit 105, brightness calculation unit 106, subject detection unit 107, and in-screen distance calculation unit 110 are configured separately from the control unit 102, but the system is not limited to this configuration. At least some of the functions of the image processing unit 105, brightness calculation unit 106, subject detection unit 107, and in-screen distance calculation unit 110 may be included in the control unit 102. In this case, the functions included in the control unit 102 may be implemented, for example, by the CPU 121 executing a program stored in the non-volatile memory 122.

[0023] <Problems to be solved by this invention> Here, we will explain the problem that this invention solves. Assume there are two subjects, A and B, located at multiple positions offset in the depth direction (left-right direction in Figure 6). Now, suppose there is a case where we want to capture a video that transitions from a state where subject A is prominent before movement to a state where subject B is prominent after movement, utilizing the "blurring effect" due to the difference in depth of field, as shown in Figure 6(a) to Figure 6(b). In other words, the left side of Figure 6 is the background, and the user wants to capture a video that transitions from a state where the focus is on subject A in the background to a state where the focus is on subject B in the foreground. In this case, the depth of field at the start of capture only includes subject A in the background, and does not include subject B in the foreground. Next, when moving from a situation where the focus is on the darker subject A in the background to a position where the focus is on the brighter subject B in the foreground, the main subject shifts to the brighter subject B, resulting in exposure fluctuations.

[0024] Here, we will explain how to calculate the correct exposure within the screen in this embodiment. First, as shown in Figure 8, the entire screen (left side of Figure 8) is divided into grid-like blocks (center side of Figure 8), and the "brightness value (Bv value)" obtained in each block is multiplied by the weight for each region to obtain the "average brightness" (block integration). The weights in each block are set numerically in a matrix as shown in the upper right side of Figure 8. The CPU 121 calculates the brightness difference between this average brightness and the target brightness, and uses aperture, shutter speed, ISO sensitivity, etc. to control the exposure so as to eliminate that difference.

[0025] As shown in Figure 8(b), when a main subject is present, the weighting of the subject area is set to be heavier than other areas. In other words, in order to set an additional weight for the subject when calculating the photometric value by adding the overall screen brightness and the subject brightness, the weight values ​​of each of the eight blocks hatched in black in the right side of Figure 8(b), which correspond to the main subject, are set to be higher. For this reason, the calculated average brightness is easily affected by the subject brightness, and the exposure tends to fluctuate in the direction of decreasing exposure when the subject brightness is bright, and increasing exposure when the subject brightness is dark. Note that in Figure 8(b), for ease of understanding, only the position of the main subject is indicated in the above eight blocks, and the way in which the weights are described is not particularly different, and the weights are followed from Figure 8(a).

[0026] Furthermore, exposure tracking in this embodiment is controlled by a combination of aperture, shutter speed, and ISO sensitivity (hereinafter referred to as the "program diagram") that is pre-designed to vary according to the brightness value (Bv value). Figure 7 is an example of a program diagram used when controlling the exposure of the camera assumed in this embodiment, with the vertical axis being "F-number (aperture value)" and the horizontal axis being "shutter speed," and the ISO fixed at "100." As mentioned above, depth of field has the characteristic that the smaller the aperture value (F-number), the shallower the depth of field, and the larger the F-number, the deeper the depth of field.

[0027] Furthermore, the smaller the aperture value, the narrower the depth of field and the greater the blur, while the larger the aperture value, the wider the depth of field and the less blur there is. The program diagrams described below are graphs showing the relationship between shutter speed and aperture value, with the vertical axis representing "F-number (aperture value)" and the horizontal axis representing "shutter speed," and also show straight lines corresponding to the subject brightness values ​​(ambient light brightness) Bv3(a) and Bv8(b). In the following, the program diagram for live viewing will simply be referred to as the "LV diagram," and the "program diagram" will simply be referred to as the "diagram."

[0028] When the focal point is moved using manual focus, the main subject moves, causing an exposure fluctuation from "Bv3" to "Bv8" as shown in Figure 7. In such cases, the F-number moves from "F2.8" to "F8" by referring to the program diagram mentioned above. This increases the depth of field, resulting in a situation where both subjects A and B are at the same depth of field, as shown in Figure 6(c), which can reduce the highlighting effect due to bokeh by half. In this embodiment, exposure control is performed based on the subject area and depth of field information, etc., solving the problem of the highlighting effect due to bokeh being halved as described above, and enabling the acquisition of appropriate imaging effects without unnatural bokeh fluctuations when switching subjects using manual focus. The operation of the imaging device 10 will be described in detail below.

[0029] (Figure 2: Overall flowchart showing the imaging operation) (Step S201) Next, the imaging operation of the imaging device 10 will be explained with reference to Figure 2. First, in step S201, when the operator (user) turns on the power switch included in the control unit 101, the control unit 102 detects this and supplies power to each component of the imaging device 10.

[0030] (Step S202) Next, in step S202, when power is supplied to each part of the imaging device 10, the shutter opens, and the sensor unit 103 receives light incident through the lens 1081 and mechanical mechanism 1091 located on the front of the camera. The aperture, shutter speed, and ISO at startup are set to pre-registered values ​​(initial AvTvSv settings), and the program diagram used is the initially set LV (Live View) diagram (initial LV diagram setting). Live View refers to displaying the image from the sensor unit 103 on the LCD monitor on the back of the camera, allowing the user to take images while checking the subject displayed on the LCD monitor instead of the viewfinder. It is also assumed that the LV diagram used is the solid line diagram with thick lines shown in Figure 9.

[0031] (Step S203) In step S203, the sensor unit 103 reads the charge accumulated according to the amount of light received and outputs it as an analog image signal to the A / D conversion unit 104. The A / D conversion unit 104 performs sampling, gain adjustment, etc. on the analog image signal output from the sensor unit 103 and outputs it as a digital image signal. The image processing unit 105 performs various image processing on the digital image signal output from the A / D conversion unit 104 and outputs it as a processed digital image signal (live image data) (live image acquisition).

[0032] (Step S204) In step S204, the luminance calculation unit 106 divides the entire screen into grid-like blocks using the obtained image data and calculates the block integral by multiplying the luminance of each block by its weight (Block integral calculation: see Figure 8).

[0033] (Step S205) In step S205, the subject detection unit 107 detects a subject from the image data acquired in step S203 (subject detection). Here, the subject detection unit 107 detects a person in the image and acquires their face region or head region. These regions are converted to coordinates on the block calculated in step S204, and together with the brightness evaluation value calculated by the brightness calculation unit 106, the brightness of the face can be calculated. As shown in Figure 8, the image obtained by averaging the screen in a grid is "Figure 8(a) Block Integration Image". Furthermore, subject detection is performed using the same original image (left side of Figure 8) to extract the above regions, it is determined which block contains the subject, and the brightness of the block corresponding to that coordinate is added and averaged. This gives us Figure 8(b) Subject Region Brightness.

[0034] (Step S206) In step S206, the luminance calculation unit 106, subject detection unit 107, in-screen distance calculation unit 110, and AE processing unit 109 perform AE processing using depth-of-field information to obtain exposure values ​​to be used for imaging. The AE processing using depth-of-field in step S206 is shown in Figure 2(b), so the AE processing will be explained with reference to this.

[0035] (Figure 2(b): AE processing using depth of field: Steps S220:S221:S222) As shown in Figure 2(b), first, in step S220, the subject detection unit 107 acquires the face information (position, size) of the subjects (acquisition of face information for subjects A and B). In step S221, the luminance calculation unit 106 uses the face information and block-integrated luminance to acquire the "overall screen luminance evaluation value" and the "subject luminance evaluation value". In this embodiment, the focus shift is assumed to occur between multiple people (multiple subjects), so the subject evaluation value is calculated for each person detected. In step S222, the final "photometric value (Bv value)" is calculated. At this time, the in-screen distance calculation unit 110 and the subject detection unit 107 determine the person who is finally in focus as the "main subject", and the AE processing unit 109 combines the "subject evaluation value" and the "overall screen evaluation value", which give the greatest weight to that person (subject), to perform photometry.

[0036] (Step S223: Step S224) In step S223, distance information is calculated for each region of the screen that has been divided into a grid, as shown in Figure 5. In this embodiment, it is assumed that a "distance map" is obtained from defocus information obtained using image sensor phase-difference pixels. An image sensor phase-difference pixel for distance measurement is incorporated into the sensor unit (image sensor) 103, and a "distance map" is obtained by obtaining the distance measurement result for each pixel at the same time as obtaining the imaging signal. In this "distance map," the value is set to be smaller the closer the subject is and larger the farther away it is, and it covers the entire screen. Therefore, in step S224, if there are multiple subjects in the screen, the in-screen distance calculation unit 110 calculates the depth of field for each subject (A, B).

[0037] (Step S225: Step S226: Step S227: Step S228) Next, in step S225, the AE processing unit 109 makes a decision on changing the diagram, but the details will be described later with reference to Figure 3, so the explanation will be omitted here. In step S226, the AE processing unit 109 makes the above decision on changing the diagram. If it is decided to change the diagram (Yes in step S226), in step 227, the AE processing unit 109 changes the LV diagram using the diagram calculated in the above process (S202, etc.) (moving image diagram change). On the other hand, if it is decided not to change the diagram (No in step S226), in step S228, the AE processing unit 109 stores the currently used diagram in the memory unit 111, etc. (diagram retention).

[0038] (Step S229) Then, in step S229, the AE processing unit 109 calculates the exposure value to be imaged using the photometric value calculated above and the set diagram.

[0039] (Step S207: Step S208) After calculating the exposure value in step S229, the process returns to Figure 2(a), and in step S207, it is determined whether or not the video recording start button was pressed. If it is determined that the button was pressed (Yes), the process proceeds to step S208; otherwise, it returns to step S203. In other words, in step S208, the control unit 102 uses the exposure value obtained by the AE processing in step S229 to instruct the start of video recording and acquire a video image. On the other hand, if it is determined that the video recording start button was not pressed, the process returns to step S203, and steps S203 to S206 are repeated.

[0040] (Steps S208~S211: Step S112) Next, the imaging device 10 enters motion recording mode. However, the operation from motion image acquisition to the end of motion recording is the same as the operation during live view standby described in steps S203 to S206 above, so a redundant explanation is omitted (steps S208 to S211). Then, in step S212, the control unit 120 determines whether or not a motion recording termination instruction has been given. If the control unit 102 determines that a motion recording termination instruction has been given (Yes), imaging ends. On the other hand, if it determines that a motion recording termination instruction has not been given (No), the process returns to step S208. In the case of motion recording termination, the encoder unit 112 converts the digital signal output from the image processing unit 105 into a format such as MPEG and outputs it to the image recording unit 113. The image recording unit 113 records the format-converted image data into an external memory such as a memory card.

[0041] (Figure 3: Flowchart showing the process of changing the program diagram) Next, referring to Figure 3, we will explain the process of determining whether to change the diagram using the subject information and distance information in step S225. Figure 3 is a flowchart of the process for changing the program diagram.

[0042] (Step S301) First, in step S301, the luminance calculation unit 106 determines whether or not a scene change has occurred. In this embodiment, the transition of depth of field is assumed to be captured with a stable angle of view, and if the captured scene changes significantly, the effect will be reduced, so it is necessary to check for such changes. For example, the luminance calculation unit 106 determines that a scene change has occurred if the difference from the previously held photometric value is greater than or equal to a predetermined number of stops. In other words, if the luminance fluctuation within the screen changes by a predetermined value or more, it is determined that a scene change has occurred. Alternatively, the difference in the gyro signals output by the gyro sensor built into the imaging device 10 may be used to determine if a scene change has occurred when that difference is greater than a predetermined value.

[0043] If it is determined in step S301 that a scene change has occurred (Yes), the process proceeds to step S310; otherwise, it proceeds to step 302.

[0044] (Step S310: Step S302: Step S303) In step S310, if it is determined that a scene change has occurred, there is a high probability that exposure fluctuations will occur as a result, so the AE processing unit 109 changes the program diagram to the initial diagram (change to initial diagram). Next, in step 302, the in-screen distance calculation unit 110 determines whether or not there are multiple subjects within the field of view. If it is determined that there are multiple subjects (Yes), the process proceeds to S303; otherwise, the process proceeds to step S309. Next, in step 303, the in-screen distance calculation unit 110 determines whether or not the multiple subjects are located at multiple positions separated by a predetermined distance or more in the depth direction. If it is determined that the subjects are located at multiple positions separated by a predetermined distance or more in the depth direction (Yes), the process proceeds to S304; otherwise, the process proceeds to step S309.

[0045] In this embodiment, an arbitrary fixed distance is assumed for determining the subject distance in the depth direction, but additional conditions may be added, such as the arrangement of subjects so that their depths of field do not overlap. Thus, in step S303, the in-screen distance calculation unit 110 determines that there are multiple subjects and that the subjects are arranged at multiple positions separated by a predetermined distance or more in the depth direction.

[0046] (Step S309) In step S309, if the subject is alone or its distance in the depth direction is shorter than a predetermined distance, the AE processing unit 109 maintains the current diagram (maintain current diagram).

[0047] (Step S304) In step S304, the AE processing unit 109 determines whether the selectable lens F-number is greater than or equal to a predetermined value. If it is determined that the selectable lens F-number is greater than or equal to a predetermined value (Yes), the process proceeds to step S309; ​​otherwise, it proceeds to step S306. For example, as shown in Figure 11, if the position of "Current Bv" on the program diagram is "F5.6" and the only movable aperture on the program diagram is "F8", the difference in aperture is only one stop, so the difference in depth of field is also small and the expected effect cannot be obtained.

[0048] Therefore, in this embodiment, the process proceeds to step 309, and if the maximum aperture is "F5.6" or higher, the current diagram is maintained. In addition, lenses with a focal length of 24mm or less are generally called wide-angle lenses, and the depth of field becomes deeper with wider-angle lenses, resulting in a situation similar to that described above when the aperture difference is small, so the current diagram is maintained in the case of wide-angle lenses as well. Thus, in this embodiment, processing is performed to restrict changes to the program diagram according to the type of lens, and processing is performed to restrict changes to the program diagram if the F value of the target lens is above a predetermined value. One form of processing to restrict changes to the program diagram refers to maintaining the current program diagram as is without changing it (S309: Maintain current diagram).

[0049] (Step S305: Step S306) In step S305, the control unit 102 determines whether manual focus (MF) is selected. If it is determined to be selected (Yes), the process proceeds to step S306; otherwise, it proceeds to step S309. Next, in step S306, the AE processing unit 109 calculates the aperture for each subject that will keep only that subject within the depth of field. In other words, in steps S302, S305, and S306, if there are multiple subjects and manual focus is selected, the AE processing unit 109 calculates the aperture for each detected person (subject) that will keep only that person (subject) within the depth of field.

[0050] (Step S307: Step S308: Figure 9: Figure 10) Next, in step S307, the AE processing unit 109 selects the aperture of the object determined to be the "main subject," and then in step S308, it calculates a modified program diagram using the selected aperture value (modification means). Then, the control unit 102 performs exposure control of the imaging device 10 based on the modified program diagram.

[0051] (Figure 9: Explanatory diagram of the program line diagram; Figure 10: Explanatory diagram of the relationship between depth of field and corresponding aperture) An example of the processing in steps S307 and S308 will be explained with reference to Figures 9 and 10. Figure 10 is an explanatory diagram when the subject distance from the imaging device 10 is measured and the apertures that bring subjects A and B within the corresponding depth of field "a" and "b" are "X" and "Y". For example, as shown in Figure 10, when the apertures X and Y of subjects A and B are both "F2.8", a diagram prioritizing "F2.8" is drawn in the diagram shown in Figure 9 (see the dotted line in Figure 9). Therefore, the program diagram in Figure 9, which is a modified version of the program diagram in Figure 7, is used, and exposure tracking from "Figure 9(a):Bv3" to "Figure 9(b):Bv8" becomes possible while maintaining the highlighting effect due to blur.

[0052] (Structure and effects of the present invention) From the above, according to the present invention, the imaging device 10 comprises a sensor unit 103 (imaging means) for capturing images and an AF processing unit 108 (focusing means) for adjusting the focal position during imaging, enabling manual focus by adjusting the focal position according to user operation. The subject detection unit 107 (detection means) detects subjects from the captured image, the in-screen distance calculation unit 110 (calculation means) calculates the depth of field that fits each detected subject, and the control unit 102 (control means) performs control based on a program diagram relating to exposure during imaging. The AE processing unit 109 (modification means) executes the process in S308, and when the focal position is adjusted with manual focus (S305), the program diagram is changed when it is determined that multiple subjects that fit within each of the calculated multiple depths of field satisfy predetermined conditions. One aspect of these predetermined conditions is that multiple subjects are located at multiple positions that are separated by a predetermined distance or more in the depth direction.

[0053] With the above configuration, the program diagram used can be appropriately changed and controlled when switching subjects using manual focus operation. As a result, subject changes and exposure tracking become possible while obtaining imaging effects such as highlighting with a "bokeh" that matches the user's intention, without unnatural blur fluctuations.

[0054] Furthermore, as shown in steps S306 to S308, when the AF processing unit 108 moves the focus position from one subject to another, it controls the aperture value so that the other subject after the move is within the depth of field, and the subject before the move is outside the depth of field. In addition, when the "main subject" is changed based on the movement of the focus position, the program diagram is changed to prioritize the use of an aperture with a large F-number among the apertures that allow each subject to be within the depth of field. As an example of how the program diagram may be changed, when the main subject is changed based on the movement of the focus position, the program diagram may be changed to prioritize the use of an aperture with a large F-number among the apertures that allow each subject to be within the depth of field.

[0055] <Addendum> This embodiment includes the following configurations, methods, and programs. (Configuration 1) An imaging device comprising an imaging means for capturing an image and a focusing means for adjusting the focal position during imaging, and capable of manual focus, where the focal position is adjusted according to user operation, A detection means for detecting a subject from the aforementioned image, A calculation means for calculating the depth of field required to contain each detected subject, A control means that performs control based on a program diagram relating to exposure during imaging, An imaging device characterized by comprising: a modification means for modifying the program diagram when adjusting the focus position with manual focus, and when it is determined that multiple subjects that fit within each of the multiple calculated depths of field satisfy predetermined conditions. (Configuration 2) The imaging device according to Configuration 1, characterized in that the predetermined conditions are that multiple subjects are located at multiple positions that are separated by a predetermined distance or more in the depth direction. (Configuration 3) The focusing means is The imaging device according to configuration 1 or 2, characterized in that when the focal position is moved from one subject to another, the aperture value is controlled so that the other subject after the movement is within the depth of field, and the subject before the movement is outside the depth of field. (Configuration 4) The modification means is, The imaging device according to configuration 1 or 2, characterized in that, when changing the main subject based on the movement of the focal position, the program diagram is changed to prioritize the use of an aperture with a large F-number among a plurality of apertures that allow each subject to fall within the depth of field. (Configuration 5) The modification means is The imaging apparatus according to configuration 1 or 2, characterized in that, when it is determined that a second predetermined condition is met, the program diagram is changed to the initial program diagram. (Configuration 6) The imaging apparatus according to Configuration 5, characterized in that the second predetermined condition is that the brightness fluctuation within the screen changes by a predetermined value or more. (Configuration 7) The imaging device according to Configuration 5, characterized in that the second predetermined condition is when the amount of variation measured by a gyro sensor built into the device shows a change of a predetermined value or more. (Configuration 8) The modification means further, The imaging apparatus according to configuration 1 or 2, characterized in that it restricts the modification of the program diagram according to the type of lens. (Configuration 9) The modification means further, The imaging apparatus according to configuration 1 or 2, characterized in that the change of the program diagram is restricted when the F-number of the target lens is greater than or equal to a predetermined value. (Configuration 10) The modification means further, The imaging apparatus according to configuration 1 or 2, characterized in that if multiple subjects are not detected by the detection means, or if the predetermined conditions are not met, the change to the program diagram is restricted. (Method) A control method for an imaging device comprising an imaging means for capturing an image and a focusing means for adjusting the focal position during imaging, wherein manual focus is possible, and the focal position can be adjusted according to user operation. A detection step for detecting a subject from the aforementioned image, A calculation process for calculating the depth of field required to contain each detected subject, A control process that performs control based on a program diagram regarding exposure during imaging, A control method for an imaging device, characterized by comprising: a modification step of modifying the program diagram when adjusting the focus position with manual focus, and when it is determined that multiple subjects that fit within each of the multiple calculated depths of field satisfy predetermined conditions. A program that causes a computer to execute a control method for an imaging device that includes an imaging means for capturing an image and a focusing means for adjusting the focal position during imaging, and which is capable of manual focus by adjusting the focal position according to user operation, The control method described above is A detection step for detecting a subject from the aforementioned image, A calculation process for calculating the depth of field required to contain each detected subject, A control process that performs control based on a program diagram regarding exposure during imaging, A program characterized by having a modification step that controls the modification of the program diagram when adjusting the focus position with manual focus and it is determined that multiple subjects that fit within each of the multiple calculated depths of field satisfy predetermined conditions.

[0056] Although preferred embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of its gist. For example, the present invention can also be realized by supplying a program that implements one or more of the functions of the above embodiments to a system or device via a network or recording medium, and the processor of the computer in that system or device reads and executes the program. Furthermore, the present invention can also be realized by a circuit (such as an ASIC) that implements one or more functions. [Explanation of symbols]

[0057] 10 Imaging device 101 Operation section 102 Control Unit 103 Sensor unit (image sensor) 105 Image Processing Unit 106 Brightness Calculation Unit 107 Subject detection unit 108 AF Processing Unit 109 AE Processing Unit 110 In-screen distance calculation unit 115 Display section 121 CPU 1081 Lens

Claims

1. An imaging device comprising an imaging means for capturing an image and a focusing means for adjusting the focal position during imaging, and capable of manual focusing by adjusting the focal position according to user operation, A detection means for detecting a subject from the aforementioned image, A calculation means for calculating the depth of field required to contain each detected subject, A control means that performs control based on a program diagram relating to exposure during imaging, An imaging device characterized by comprising: a modification means for modifying the program diagram when adjusting the focus position with manual focus, and when it is determined that multiple subjects that fit within each of the multiple calculated depths of field satisfy predetermined conditions.

2. The imaging apparatus according to claim 1, characterized in that the predetermined conditions are that multiple subjects are located at multiple positions separated by a predetermined distance or more in the depth direction.

3. The focusing means is The imaging device according to claim 1 or 2, characterized in that when the focal position is moved from one subject to another, the aperture value is controlled so that the other subject after the movement is within the depth of field, and the original subject before the movement is outside the depth of field.

4. The aforementioned modification means is, The imaging apparatus according to claim 1 or 2, characterized in that, when changing the main subject based on the movement of the focal position, the program diagram is changed to prioritize the use of an aperture with a large F-number among a plurality of apertures that fit within the depth of field for each subject.

5. The aforementioned modification means further, The imaging apparatus according to claim 1 or 2, characterized in that, if it is determined that a second predetermined condition is met, the program diagram is changed to the initial program diagram.

6. The imaging apparatus according to claim 5, characterized in that the second predetermined condition is that the brightness fluctuation within the screen changes by a predetermined value or more.

7. The imaging device according to claim 5, characterized in that the second predetermined condition is when the amount of variation measured by a gyro sensor built into the device shows a change of a predetermined value or more.

8. The aforementioned modification means further, The imaging apparatus according to claim 1 or 2, characterized in that the change of the program diagram is restricted according to the type of lens.

9. The aforementioned modification means further, The imaging apparatus according to claim 1 or 2, characterized in that it restricts the modification of the program diagram when the F-number of the target lens is equal to or greater than a predetermined value.

10. The aforementioned modification means further, The imaging apparatus according to claim 1 or 2, characterized in that if multiple subjects are not detected by the detection means, or if the predetermined conditions are not met, the change to the program diagram is restricted.

11. A control method for an imaging device comprising an imaging means for capturing an image and a focusing means for adjusting the focal position during imaging, wherein manual focus is possible, allowing the focal position to be adjusted according to user operation, A detection step for detecting a subject from the aforementioned image, A calculation process for calculating the depth of field required to contain each detected subject, A control process that performs control based on a program diagram regarding exposure during imaging, A control method for an imaging device, characterized by comprising: a modification step of modifying the program diagram when adjusting the focus position with manual focus, and when it is determined that multiple subjects that fit within each of the multiple calculated depths of field satisfy predetermined conditions.

12. A program that causes a computer to execute a control method for an imaging device that includes an imaging means for capturing an image and a focusing means for adjusting the focal position during imaging, and which is capable of manual focus by adjusting the focal position according to user operation, The control method described above is A detection step for detecting a subject from the aforementioned image, A calculation process for calculating the depth of field required to contain each detected subject, A control process that performs control based on a program diagram regarding exposure during imaging, A program characterized by having a modification step that controls the modification of the program diagram when adjusting the focus position with manual focus and it is determined that multiple subjects that fit within each of the multiple calculated depths of field satisfy predetermined conditions.

Images