An imaging device capable of focusing on an intended subject, a control method for the imaging device, and a program.

Abstract

To improve the accuracy of focusing on the intended subject. [Solution] An imaging device for imaging multiple subjects, comprising: a defocus range estimation means for estimating the defocus range of each of the multiple subjects; a target determination means for setting a combination of the multiple defocus ranges; a defocus range setting means for setting a controlled defocus range from the combination of the multiple defocus ranges based on the setting; and a control means for controlling the imaging conditions of the imaging device based on the controlled defocus range.

Description

Technical Field

[0001] The present invention relates to an imaging device capable of focusing on an intended subject, a control method for the imaging device, and a program.

Background Art

[0002] There is known an imaging device that performs focus adjustment so as to focus on an intended subject based on a plurality of focus detection results within an imaging range. Patent Document 1 discloses a method of adjusting the aperture and focus of a camera based on the detected distance information of a plurality of subjects to capture an image that is in focus on a plurality of subjects simultaneously.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, the depth direction spread of the subject is not considered for the distance information of the subject, and there are cases where accurate focusing cannot be achieved. The present invention has been made in view of the above problems, and an object thereof is to improve the focusing accuracy on an intended subject.

Means for Solving the Problems

[0005] One aspect of the present invention is an imaging device that images a plurality of subjects, and includes a defocus range estimation unit that estimates a defocus range for each of the plurality of subjects, a target determination unit that sets a combination of the plurality of defocus ranges, a defocus range setting unit that sets a control defocus range from the combination of the plurality of defocus ranges based on the setting, and a control unit that controls imaging conditions of the imaging device based on the control defocus range.

[0006] Another aspect of the present invention is a control method for an imaging device, comprising: a defocus range estimation step of estimating the defocus range of each of a plurality of subjects; a target determination step of setting a plurality of combinations of the defocus ranges; a defocus range setting step of setting a control defocus range from the combinations of the defocus ranges of the plurality of subjects based on the setting; and a control step of controlling the imaging device based on the control defocus range. [Effects of the Invention]

[0007] According to the present invention, it is possible to improve the accuracy of focusing on the intended subject. [Brief explanation of the drawing]

[0008] [Figure 1] This is a hardware configuration diagram of the imaging device according to the present invention. [Figure 2] This is a diagram of the imaging optical system for explaining the amount of defocus in the first embodiment. [Figure 3] This diagram illustrates the configuration of the imaging device 10 in the first embodiment. [Figure 4] This is a diagram illustrating the defocus range in the first embodiment. [Figure 5] This is an overall flowchart of the first embodiment. [Figure 6] This diagram illustrates the processing of the target determination unit in the first embodiment. [Figure 7] This is a flowchart of the target determination process in the first embodiment. [Figure 8] This is a flowchart of the range setting process in the first embodiment. [Figure 9] This figure illustrates the process of setting the defocus range in the first embodiment. [Figure 10] This figure illustrates the configuration of the imaging device 10 in the second embodiment. [Figure 11]It is a flowchart of the range setting process in the second embodiment. [Figure 12] It is a flowchart of the priority setting process in the second embodiment. [Figure 13] It is a diagram for explaining the priority setting process in the second embodiment. [Figure 14] It is an approximate formula for calculating the depth of field in the second embodiment.

Mode for Carrying Out the Invention

[0009] Hereinafter, based on the preferred embodiments of the present invention, an explanation will be given with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[0010] (First Embodiment) As an example of an imaging device according to the present invention, an interchangeable-lens digital camera will be described.

[0011] The first embodiment of the present invention will be described with reference to FIGS. 1 to 9. FIG. 1 is a block diagram showing the main part of the system of the imaging device 10. The imaging device 10 is, for example, an interchangeable-lens digital camera, and includes a camera body 100 and a lens unit 200 that guides incident light to an image sensor 101 included in the camera body 10.

[0012] The camera body 100 has an image sensor 101, a system control unit 102, a shutter 103, a memory 104, a power switch 105, a mode switching unit 106, a rear monitor 107, a touch panel 108, and a finder display unit 109. The camera body 100 further has an eyepiece lens 110, an eyepiece detection unit 111, a shutter control unit 112, and a lens mount mechanism 113.

[0013] The imaging device 101 is, for example, a CMOS image sensor, which converts an optical signal, which is an optical image, into an electrical signal. The light rays incident on the photographing lens 201 in the lens unit 200 form an optical image on the imaging device 101 through the aperture 202 and the shutter 103.

[0014] The system control unit 102 incorporates a well-known CPU or the like and controls the camera body 100. The system control unit 102 includes an image processing unit for the video signal obtained by the imaging device 101. Further, the system control unit 102 includes a phase difference AF unit that performs a focus detection process by the phase difference detection method based on the focus detection image data (signal for phase difference AF) obtained from the imaging device 101 and the image processing unit. More specifically, the image processing unit generates a pair of image data formed by light fluxes passing through a pair of pupil regions of the imaging optical system as focus detection image data. The phase difference AF unit detects the amount of focus shift based on the shift amount of the pair of image data. Thus, the phase difference AF unit of the present embodiment performs imaging surface phase difference AF based on the output of the imaging device 101 without using a dedicated AF sensor.

[0015] The memory 104 stores programs, variables, constants, etc. for the operation of the system control unit 102. Further, this memory 104 also includes an electrically erasable and rewritable non-volatile memory. Various parameters, setting values such as ISO sensitivity, shooting modes, and various correction data, etc. are stored in the memory 104.

[0016] The power switch 105 switches between the power-on / off modes of the camera body 100. The mode switching unit 106 is a switch for switching and setting various shooting modes such as live view shooting and video shooting.

[0017] The rear monitor 107 is composed of a liquid crystal device, an LED, or the like that displays operation states such as characters, images, and sounds, and shooting information such as messages according to the execution of the program in the system control unit 102.

[0018] The touch panel 108 is positioned in an area approximately the same size as the rear monitor 107, detects contact with a finger or pen, notifies the system control unit 102 of the contact position relative to the rear monitor 107, and executes an operation or function associated with the contact position.

[0019] The viewfinder display unit 109, like the rear monitor 107, displays shooting information in accordance with the execution of a program in the system control unit 102, and together with the eyepiece lens 110, constitutes an electronic viewfinder (EVF).

[0020] The eyepiece detection unit 111 allows the system control unit 102 to selectively display the aforementioned shooting information on the rear monitor 107 or the viewfinder display unit 109, depending on the photographer's eyepiece state.

[0021] Next, the configuration of the lens unit 200 will be described. The camera body 100 and the lens unit 200 are mechanically and electrically connected via a lens mount mechanism 113, and the lens unit 200 is detachable from the camera body 100. The lens unit 200 consists of a photographic lens 201, an aperture 202, a lens drive circuit 203, an aperture control circuit 204, and a lens control unit 205. In Figure 1, only one photographic lens 201 is shown for simplification, but in reality, the photographic lens 201 is composed of multiple photographic lens groups.

[0022] The aperture 202 is a mechanism for adjusting the amount of light incident on the image sensor 101 through the lens, and is controlled by the aperture control circuit 204.

[0023] The lens drive circuit 203 is a drive circuit for moving the lens along the optical axis in order to adjust the focus position of the shooting screen.

[0024] The lens control unit 205 controls the entire lens unit 200. The lens control unit 205 includes a memory (not shown) that stores various constants, variables, and programs for lens operation. It also includes a non-volatile memory that stores information specific to the lens unit, such as maximum and minimum aperture values ​​and focal length.

[0025] The system control unit 102 of the camera body 100 calculates the amount of defocus using the output information of the image sensor 101. The system control unit 102 communicates via the lens control unit 205 of the lens unit 200 and adjusts the focus by controlling the lens drive circuit 203 based on the calculated amount of defocus.

[0026] Here, the amount of defocus used as image depth information in the present invention will be explained with reference to Figure 2. Figure 2 is a diagram illustrating the relationship between the amount of defocus of the imaging optical system and the phase difference (image shift amount) between the first focus detection signal and the second focus detection signal acquired from the image sensor.

[0027] An image sensor (not shown) is positioned on the imaging surface 300 in Figure 2, and the exit pupil of the imaging optical system is divided into two parts: a first pupil region 311 and a second pupil region 312. The defocus amount d represents the distance from the imaging position C of the light beam from the subject (321, 322) to the imaging surface 300. Let |d| be the absolute value of this distance. When the imaging position C is on the subject side of the imaging surface 300, it is called a front-focus state, and the defocus amount is represented by a negative value (d<0). When the imaging position C is beyond the imaging surface 300 and on the opposite side from the subject, it is called a back-focus state, and the defocus amount is represented by a positive value (d>0). In the in-focus state, where the imaging position C is on the imaging surface 300, d=0. The imaging optical system shown in Figure 2 is in focus (d=0) with respect to subject 321 and in a front-focus state (d<0) with respect to subject 322. The combination of a front-focused state (d<0) and a back-focused state (d>0) is called a defocused state (|d|>0).

[0028] In the front-focused state (d<0), a portion of the light beam from the subject 322 passes through the first pupil region 311 and is focused, forming a blurred image on the imaging surface 300 with a width Γ1 centered on the centroid position G1 of the light beam. This blurred image is received by each first focus detection pixel on the image sensor, and a first focus detection signal is generated. In other words, the first focus detection signal is a signal representing a subject image on the imaging surface 300 in which the subject 322 is blurred by a blur width Γ1 at the centroid position G1 of the light beam. Similarly, a portion of the light beam from the subject 322 passes through the second pupil region 312 and is focused, forming a blurred image on the imaging surface 300 with a width Γ2 centered on the centroid position G2 of the light beam. This blurred image is received by each second focus detection pixel on the image sensor, and a second focus detection signal is generated. In other words, the second focus detection signal is a signal that represents the subject image in which the subject 322 is blurred by a blur width Γ2 at the centroid position G2 of the light beam on the imaging surface 300.

[0029] The blur widths Γ1 and Γ2 of the subject image increase in roughly proportional proportion to the increase in the magnitude of the defocus amount d |d|. Similarly, the magnitude of the image shift amount p |p| between the first focus detection signal and the second focus detection signal, which is the difference in the centroid positions of the light beam (G1-G2), also increases in roughly proportional proportion to the increase in the magnitude of the defocus amount d |d|.

[0030] In a back-focused state, the direction of image shift between the first and second focus detection signals is opposite to that in a front-focused state. The relationship between the amount of defocus, blur width, and image shift is the same as in the front-focused state.

[0031] As described above, the magnitude of the image shift amount p between the first focus detection signal and the second focus detection signal increases as the magnitude of the defocus amount d |d| increases. In this embodiment, image plane phase difference detection is performed to calculate the defocus amount d from the image shift amount p between the first focus detection signal and the second focus detection signal obtained using the image sensor 101. Therefore, the phase difference AF unit of the system control unit 102 converts the image shift amount p into a detected defocus amount d. The conversion coefficient is calculated based on the baseline length, given the relationship that the magnitude of the image shift amount |p| between the first focus detection signal and the second focus detection signal increases as the magnitude of the defocus amount |d| of the imaging signal increases. In this embodiment, the unit of the defocus amount d is the product of the aperture F value and the allowable circle of confusion diameter δ in the imaging device optical system during image capture [Fδ].

[0032] The operation of the imaging device according to this embodiment will be explained with reference to Figures 3 to 9. Figure 3 is a diagram illustrating the configuration of the imaging device 10 according to this embodiment. The imaging device 10 is composed of a defocus range estimation unit 401, a target determination unit 402, a range setting unit 403, and a control unit 404.

[0033] The defocus range estimation unit 401 distinguishes between different types of subjects and estimates the defocus range for each part of the subject. The defocus range is the range of values ​​for the amount of defocus that the subject has.

[0034] The target determination unit 402 determines a combination of multiple subjects by combining the defocus ranges estimated by the defocus range estimation unit 401.

[0035] The range setting unit 403 changes the control defocus range, which is a combination of defocus ranges, based on the combination determined by the target determination unit 402. In other words, the range setting unit 403 sets a new defocus range.

[0036] The control unit 404 calculates the amount of drive for the photographic lens 201 based on the control defocus range changed by the range setting unit 403, and controls the focus position. The aperture 202 adjusts the depth of field (DoF). In other words, the control unit 404 controls the imaging conditions based on the control defocus range.

[0037] The defocus range estimation unit 401 is described in detail below. Various models can be considered for the defocus range estimation unit 401. For example, a neural network using a convolutional neural network (CNN), a Vision Transformer (ViT), or a Support Vector Machine (SVM) combined with a feature extractor. In this embodiment, the defocus range estimation unit 401 uses the defocus map calculated by the phase difference AF unit of the system control unit 102, image data including the subject detection area, and the position and size of the subject detection area as input for defocus range estimation. The defocus map is information on the defocus amount distribution to which a certain amount of defocus is assigned to a certain number of pixels on the imaging surface. For example, when the subject is a person, the image data of the area encompassing the subject detection area is used as the input image data. The portion of the defocus map corresponding to the subject detection area is also used as input.

[0038] The defocus range estimation unit 401 according to this embodiment includes a subject detection unit (not shown). The subject detection unit performs subject detection processing, described later, based on a subject detection signal generated by the system control unit 102. The subject detection processing detects a subject detection area indicating the position and size of the subject for each type, state, and part of the subject. The defocus range estimation unit 401 takes as input image information including region information within the image of the subject detected by the subject detection unit, and a defocus map detected by the phase difference AF unit of the system control unit 102, and outputs the defocus range of each detected subject.

[0039] The object detection unit is a known object detector composed of a machine-learned CNN that performs whole-area detection and local area detection of a specific object. The objects that can be detected whole-area and locally are those defined in advance when the object detector is machine-learned. The object detection unit may be implemented using a graphics processing unit (GPU) or a circuit specialized for estimation processing by a CNN.

[0040] Machine learning of a CNN can be performed using any method. For example, a predetermined computer, such as a server, may perform machine learning of the CNN, and the imaging device 10 may acquire the trained CNN from the predetermined computer. In this embodiment, the CNN of the subject detection unit is trained by performing supervised learning, where the predetermined computer takes training image data as input and the position information of the subject corresponding to the training image data as training data. Note that the CNN training may also be performed by the imaging device 10.

[0041] As described above, the subject detection unit includes a trained model which is a CNN trained by machine learning. The subject detection unit takes image data as input, estimates the position, size, confidence level, etc., of the subject, and outputs the estimated information. The CNN may be, for example, a network in which a fully connected layer and an output layer are connected to a layer structure in which convolutional layers and pooling layers are stacked alternately. In this case, for example, backpropagation may be applied to train the CNN. Alternatively, the CNN may be a neocognitive CNN with a set of feature detection layers (S layer) and feature integration layers (C layer). In this case, a training method called "Add-if Silent" may be applied to train the CNN.

[0042] The subject detection unit may use any pre-trained model other than a pre-trained CNN. For example, a pre-trained model generated by machine learning, such as a support vector machine or a decision tree, may be applied to the subject detection unit. Furthermore, the subject detection unit does not have to be a pre-trained model generated by machine learning. For example, any subject detection method that does not use machine learning may be applied to the subject detection unit.

[0043] Here, the subject type refers to the classification of each subject corresponding to the defocus range estimation by the defocus range estimation unit 401. For example, this could be a classification of subjects such as people, animals, or vehicles, and animals could be further subdivided into horses, birds, etc. The subject part refers to the defined area of ​​the defocus range estimation target that the defocus range estimation unit 401 corresponds to, such as the overall area or local area.

[0044] The overall area may be defined as the area that includes the entire subject, or as the area that includes the main part of the subject. For example, the overall area of ​​a subject related to "vehicles" can be defined for each type of subject, such as the "body" of a car or motorcycle, the "front car" of a train, or the "fuselage" of an airplane.

[0045] A local region refers to a part of a subject identified in the overall region. For example, a local region might be "a person's pupil" in relation to "an entire person's face" as the overall region, or "an animal's pupil" in relation to "an entire animal's face" as the overall region. It is set as a local region that is included within the overall region. In other words, a local region may not be included in the overall region; for example, a local region might be "a driver's helmet" that extends beyond the body of a motorcycle in relation to "the entire body of a motorcycle." The type and part of the subject described above correspond to the detection results of the subject detection unit and are linked to the position and size of the subject detection region.

[0046] The defocus range estimation unit 401 distinguishes between different types of subjects, such as a person's pupil or a horse's face, based on the position and size of the subject detection area, and estimates the defocus range for each part of the subject. The defocus range estimation unit 401 outputs the possible range of defocus values ​​for each subject detection area as the defocus range of the subject. For example, for a subject detection area (pupil), two values ​​are output: the maximum and minimum values ​​of the pupil's defocus amount.

[0047] Figure 4 is a diagram illustrating the defocus range, and Figure 4(a) shows the process of photographing a person 811 using the imaging device 10. In Figure 4(a), 812 represents the person's pupil, 813 represents the person's face, and 814 represents the extent of the person's torso as an object in the depth direction as seen from the imaging device 10. 815 is the focus position in the imaging device 10, indicating that the imaging device 10 is in focus at the position of the person's pupil 812.

[0048] Figure 4(b) schematically shows the defocus range of the person's pupil 812, face 813, and torso 814. In Figure 4(b), the horizontal axis represents the amount of defocus, and the length of the line segment represents the defocus range, which is the range of values ​​for the amount of defocus. The side closer to the imaging device 10 is called the near side, and the side further away is called the far side.

[0049] In Figure 4(a), for example, the extent of the human torso 814 as an object in the depth direction as seen from the camera is such that the closest point is, for example, the tip of the person's nose, and the furthest point is, for example, the tip of the person's shoulder. Therefore, the maximum value of the defocus amount of the human torso 814 (closest value) is the defocus amount that corresponds to the tip of the person's nose, and the minimum value of the defocus amount (farthest value) is the defocus amount that corresponds to the tip of the person's shoulder. The range of values ​​defined from these maximum and minimum values ​​is the defocus range of the human torso 814. In Figure 4(b), the line segment corresponding to the human torso represents the relationship of the defocus amounts, with the closest defocus value being, for example, 0.2Fδ, and the farthest value being, for example, -1.4Fδ. In this way, the defocus range estimation unit 401 estimates the defocus range by taking into account the perspective relationship in the depth direction of the estimation target, such as the eyes, face, and torso of the person.

[0050] Figure 5 shows a flowchart of the entire imaging device according to this embodiment.

[0051] When the imaging device 10 starts capturing images, in step S501, the defocus range estimation unit 401 estimates the defocus range for multiple subjects within the image-capable area. The defocus range estimation is performed by a machine learning model that takes the defocus map, image data including the subject detection area, and the position and size of the subject detection area as input and outputs the defocus range for each subject area. This model is created by training it with pairs of data: the defocus map, image data including the subject detection area, and the position and size of the subject detection area, and the correct data for the defocus range of each subject area. Existing machine learning methods such as deep learning can be used.

[0052] Next, in step S502, the target determination unit 402 determines a combination of multiple types of subjects whose defocus range was estimated in step S501. Figure 6 is a diagram illustrating the process in step S502. Figure 7 shows a flowchart of the process in step S502.

[0053] Figure 6(a) illustrates the case where there are multiple subjects of different types in the image 800 captured in step S501. Image 800 contains subjects such as person A801, dog A802, person B803, and dog B804. In addition, the local areas of the subject detection region acquired in step S501, namely the face of person A 805, the face of dog A 806, the face of person B 807, and the face of dog B 808, are displayed as detection frames.

[0054] Figure 6(b) is a schematic representation of the defocus range estimation results obtained in step S501 for person A801, dog A802, person B803, and dog B804 in Figure 6(a), showing the defocus range of each subject's face, which is a local area. The horizontal axis represents the magnitude of the defocus amount, and the length of the line segment represents the defocus range. Figure 6(b) shows the defocus ranges for person A's face 805, dog A's face 806, person B's face 807, and dog B's face 808. For example, the left end of the line segment corresponding to person A's face 805 indicates the front-focused defocus value among the pixels in the area of ​​person A's face 805. Similarly, the right end of the line segment corresponding to person A's face 805 indicates the back-focused defocus value among the pixels in the area of ​​person A's face 805, and the entire line segment represents the defocus range of person A's face.

[0055] Figure 7 illustrates the process of determining the combination of defocus ranges in step S502.

[0056] In step S601, the target determination unit 402 determines a combination of multiple subjects of different types. In this embodiment, the user selects a combination from a preset list of candidate combinations using the touch panel 108. The candidate combinations are two or more selections made by the defocus range estimation unit 401 from the corresponding classifications of defocus range estimation targets, such as a person and a horse, or a person, a horse, and a bird. The presets cover all combinations of the classifications of defocus range estimation targets that the defocus range estimation unit 401 supports, and if the number of classifications supported by the defocus range estimation unit 401 is n, then 2 n- There is one possible combination. The number of candidates for selecting a combination may be limited by setting an upper limit on the number of classifications n. Alternatively, the user may be able to select a combination from a selection of preset combinations to suit their needs. The user may also be able to specify combination candidates in advance so that the desired combination is selected without manual selection. In step S601, the target determination unit 402 determines a combination of multiple subjects of different types, for example, a combination of a person and a dog.

[0057] In step S602, the target determination unit 402 determines which part of the subject whose defocus range will be combined. In this step, the target determination unit 402 determines which part of the defocus range to combine from among the defocus ranges of multiple subjects of different types estimated by the defocus range estimation unit 401. The part of the defocus range to be combined can be any part that can limit the defocus range of each subject, and in this embodiment, local areas of each subject are selected as a priority.

[0058] Let's take the example of combining the defocus range of the person and the defocus range of the dog, based on the combination of person and dog determined in step S601. Assume that the defocus range estimation unit 401 has obtained the defocus range of the person's torso, which is the entire area of ​​the person, and the defocus range of the person's face, which is a local area of ​​the person. Similarly, assume that the defocus range of the dog's torso, which is the entire area of ​​the dog, and the defocus range of the dog's face, which is a local area of ​​the dog, have been obtained. In this case, the parts to combine the defocus ranges in step S602 are determined to be the person's face and the dog's face, which are local areas of each subject. Here, the parts to combine the defocus ranges may be specific parts that the user has specified in advance. If multiple defocus ranges of local areas have been obtained, they may be combined and considered as one part. For example, if the defocus range of the left pupil and the defocus range of the right pupil have been obtained as local areas of the person, the combined defocus range may be considered as the defocus range of the person's pupils. The pupils of the person, from which the defocus range has been obtained through the combination, may be used as the part to combine with the defocus range of other subjects.

[0059] In step S603, the target determination unit 402 determines whether there are multiple combinations of the same subject in the image. If there are multiple combinations of the same subject, it distinguishes between them and determines the target to which the defocus range will be combined.

[0060] Let's take the case where it is decided in step S601 to combine the defocus range of the person and the defocus range of the dog. If there are two or more combinations of person and dog in the image, proceed to step S604; otherwise, terminate the processing of the target determination unit and proceed to the processing of S503. In this embodiment, if there are multiple defocus ranges of the subject determined in step S601 in the defocus range estimation result obtained in step S501, it is determined that the same combination of subject exists.

[0061] In step S604, if it is determined in step S603 that there are multiple combinations of the same subject, the target determination unit 402 performs subject linking processing so that the defocus ranges are combined using the combinations determined in step S601. In this embodiment, subjects with the largest IoU (Intersection over Union) of the subject detection area and a common defocus range are linked together.

[0062] For example, consider the case in Figure 7(a) where, in step S601, the subjects to be combined in the defocus range are determined to be a person and a dog, and in step S602, the parts to be combined in the defocus range are the person's face and the dog's face. In S604, the objects to be verified for association are all combinations of the subject detection area representing the person's face and the subject detection area representing the dog's face. In the case of Figure 7(a), the IoU is calculated for face 806, which is the subject detection area of ​​dog A802, or face 808, which is the subject detection area of ​​dog B804, for face 805, which is the subject detection area of ​​person A801, and face 807, which is the subject detection area of ​​person B803. The IoU for person A801 ​​and dog B804, person B803 and dog A806, and person B803 and dog B804 is 0, but the IoU for person A801 ​​and dog A802 is, for example, 0.2.

[0063] Furthermore, it is determined whether the combination of subjects for which IoU was calculated has a common defocus range. In the case of Figure 6(b), it is determined whether the defocus range of dog A802's face or dog B804's face overlaps with the defocus range of person A801's face and the defocus range of person B803's face. In the case of Figure 6(b), there is no overlapping defocus range for the combination of person A801's face and dog B804's face, and for the combination of person B803's face and dog A802's face. On the other hand, it is determined that there is an overlapping defocus range for the combination of person A801's face and dog A802's face, and for the combination of person B803's face and dog B804's face.

[0064] In the linking process of step S604, the target determination unit 402 determines the combination of the defocus range of person A801's face and the defocus range of dog A802's face as the linking target. Similarly, in the linking process of step S604, the target determination unit 402 determines the combination of the defocus range of person B803's face and the defocus range of dog B804's face as the linking target. This linking process is performed in order to uniquely determine the focus target from the linked targets and to control the imaging device 10. For example, if the faces of person A801 ​​and dog A802 are determined to be the focus targets, the imaging device 10 is controlled so that the new defocus ranges based on their defocus ranges fall within the focus range. This makes it possible to capture an image in focus on both person A801 ​​and dog A802.

[0065] In this embodiment, subjects with the largest IoU and overlapping defocus ranges are linked; however, for example, subjects with the closest distance between the center coordinates of the subject detection area may also be linked. In addition to whether the defocus ranges overlap, subjects with the closest maximum or minimum defocus ranges, or subjects with the closest midpoint defocus range, may also be linked.

[0066] In step S605, a unique target for combining defocus ranges is determined from the subject combinations linked in step S604. In this embodiment, the defocus ranges of the subject combinations linked in S604 are compared, and the target is determined to be the combination with the closest defocus range. For example, consider the case where in step S604, the combination of the defocus range of person A801's face and the defocus range of dog A806's face is linked, and the combination of the defocus range of person B803's face and the defocus range of dog B804's face is linked. In Figure 6(b), the closest (maximum) value of the defocus range between person A801's face and dog A806's face, linked in S604, is compared with the closest (maximum) value of the defocus range between person B803's face and dog B804's face. It can be determined that the maximum defocus range of the face of person A801 ​​and the face of dog A is greater than the maximum defocus range of the face of person B803 and the face of dog B804. As a result, in Figure 6, the face of person A801 ​​and the face of dog A806 are selected as the subject to combine the defocus ranges. At this point, the target determination process in step S502 is completed.

[0067] In step S503, the defocus range is set in the range setting unit 403 based on the combination of subjects set in the target determination unit 402.

[0068] Figure 8 shows the flowchart of the process in step S503.

[0069] In step S701, the range setting unit 403 determines how to combine the defocus ranges based on the combination of subjects determined by the target determination unit 402. In this embodiment, the range setting unit 403 combines the defocus ranges from the closest to the furthest defocus range among the defocus ranges of each of the multiple subjects.

[0070] In step S702, the range setting unit 403 acquires the defocus range of the parts of the subject whose combination has been determined by the target determination unit 402.

[0071] Figure 9 illustrates the defocus range setting process of the range setting unit 403. Figure 9(a) shows the defocus ranges of each part of the subject determined by the target determination unit 402 as line segments, when the combination of subjects is a person's face and a dog's face. For example, the figure shows the case where the acquired defocus range is 0.1Fδ to 0.3Fδ for the person's face and 0.2Fδ to 0.4Fδ for the dog's face.

[0072] In step S703, the range setting unit 403 combines the defocus ranges acquired in step S702 according to the method determined in step S701. In this embodiment, the maximum values ​​and minimum values ​​of each defocus range acquired in step S702 are compared to determine the combined defocus range. In other words, the target determination unit 402 compares the defocus ranges of the parts whose combinations have been determined and selects the maximum and minimum values ​​of all the defocus ranges that can be combined. That is, the nearest defocus amount and the furthest defocus amount included in the combination of defocus ranges set by the target determination unit 402 are set as the maximum and minimum values ​​of the new defocus range.

[0073] Figure 9(b) shows the resulting defocus range, obtained by combining the defocus ranges acquired in Figure 9(a) in this step, illustrated by a line segment. As mentioned above, in Figure 9(a), the defocus range for the person's face is 0.1Fδ to 0.3Fδ, and the defocus range for the dog's face is 0.2Fδ to 0.4Fδ. Of these defocus ranges, the closest defocus amount corresponds to the dog's face at 0.4Fδ, and the furthest defocus amount corresponds to the person's face at 0.1Fδ. Therefore, when the person and dog are combined, the defocus range is 0.1Fδ to 0.4Fδ.

[0074] In S504, the control unit 404 controls the imaging device 10 based on the result of the range setting unit 403. The control unit 404 controls the photographic lens 201 and aperture 202 based on the new defocus range set in S503. In this embodiment, the drive amount required to control the photographic lens 201 is calculated using the defocus amount at the center of the defocus range set in S503, and the focus is controlled. In addition, aperture control not only adjusts the amount of incident light but also adjusts the depth of field (DoF). Using the defocus range information set in S503, it is possible to adjust the aperture to determine how much of a different type of subject is included in the depth of field. For example, by adjusting the aperture so that the defocus range of each subject falls within the unit depth determined by the allowable circle of confusion, camera control that keeps each of the different types of subjects in focus can be achieved.

[0075] In step S505, based on the control result of S504, the imaging device 10 performs imaging. The imaging device according to this embodiment can take into account the depth of the subject and capture images that are accurately in focus on multiple subjects.

[0076] In this embodiment, the defocus range estimation unit 401 is configured to include a subject detection unit, but this configuration is merely an example. For example, the defocus range estimation unit 401 may be configured to estimate the position, region, and defocus range for each part of the subject.

[0077] (Variation 1) In the first embodiment, the user manually selected candidate subject combinations in step S601. However, candidate subject combinations may be set without manual selection.

[0078] For example, the combination of subjects may be set based on the user's shooting tendencies. If the user selects the same combination of subjects a certain number of times consecutively in step S601, in subsequent imaging steps S601, that combination will be considered the combination desired by the user and the combination will be set accordingly. For example, if the user selects the combination of a person and a horse 10 or more times consecutively in step S601 and takes pictures, in the next step S601, the combination of a person and a horse will be determined to be included in the defocus range without any user intervention.

[0079] Alternatively, the combination of subjects may be set according to the estimation results obtained by the defocus range estimation unit 401 and the results detected by the subject detection unit.

[0080] For example, when the defocus range estimation result is obtained in step S501, if the subject detection unit can authenticate a specific subject, the subject combination is set in step S601 based on the authentication result. Authentication refers to the process of identifying a registered person in an image by pre-storing the image of the person in question in memory 104, etc., if the subject is a person. If two specific people are authenticated by the subject detection unit in step S501, the two people authenticated in step S501 are set as the subject combination to combine the defocus range in the next step, S601.

[0081] Furthermore, when the defocus range estimation result is obtained in step S501, if the subject detection unit obtains the defocus range estimation result for a specific subject a certain number of times or more, it is determined that shooting is being performed under certain shooting conditions, and a pre-set combination of subjects is set. For example, if the defocus range estimation result for a person and a horse is obtained 10 or more times consecutively in step S501, it is determined that the shooting is taking place at a racecourse, and in the next step S601, a pre-set combination of person and horse is determined. The combination to be determined may also be set in advance by the user.

[0082] This modification reduces the effort required for the user to manually select combinations of subjects. Furthermore, the subject combination settings described above may also include recommendations for combinations to the user. The user then manually selects their desired combination from the recommended options. This recommendation system prevents the user from selecting combinations they did not intend.

[0083] (Modification 2) In step S701 of the first embodiment, the range from the closest to the furthest subject among the defocus ranges of multiple subjects is set as the new defocus range. However, a defocus range common to multiple subjects may also be set as the new defocus range.

[0084] In this modified example, when comparing the defocus ranges acquired in step S702 in step S703, common defocus ranges are determined from among the defocus ranges of each subject acquired in step S702. That is, the defocus ranges of parts whose combinations have been determined are compared, and the range in which all defocus ranges overlap is determined. For example, let's consider the case where the defocus ranges acquired in S702 are -0.1Fδ to 0.2Fδ corresponding to a person's face and 0.1Fδ to 0.7Fδ corresponding to a dog's face. As a result of comparing each defocus range in step S703, the resulting defocus range is 0.1Fδ to 0.2Fδ.

[0085] By setting a defocus range common to multiple subjects as the new defocus range, it is possible to set a narrower range as the new defocus range than when setting the defocus range from the closest to the furthest of multiple subjects as the new defocus range. As a result, when controlling the aperture to 202 in step S504, it becomes possible to shoot with a shallower depth of field.

[0086] (Second embodiment) The second embodiment will be described using Figures 10 to 14. Explanations common to the first embodiment will be omitted, and the differences from the first embodiment will be explained primarily. In this embodiment, a priority is set for the defocus range, and the defocus ranges are combined according to the priority.

[0087] Figure 10 is a diagram illustrating the imaging device 10 according to this embodiment. The defocus range estimation unit 401, target determination unit 402, and control unit 404 in this embodiment are the same as those in Figure 2 of the first embodiment.

[0088] In the imaging device 10 according to this embodiment, the range setting unit 403 includes a priority determination unit 4031. The priority determination unit 4031 determines the priority of the defocus ranges when the range setting unit 403 combines the defocus ranges of multiple subjects of different types.

[0089] Figure 11 shows a flowchart of the processing of the range setting unit 403 in this embodiment. From the process of determining the defocus range combination method in step S902 to the process of combining each defocus range in step S904, the process is the same as that shown in steps S701 to S703 in Figure 8 of the first embodiment. The priority setting process in step S901 will be explained in detail with reference to Figures 12 and 13.

[0090] Figure 13 illustrates the priority setting process in step S901. Figure 13(a) shows a case where there are multiple subjects of different types in image 1100, and image 1100 contains a dog 1101 and a person 1102 as subjects. In addition, among the subject detection areas acquired by the subject detection unit of the defocus range estimation unit 401, the left pupil 1106 of the dog 1101 and the right pupil 1104 of the person 1102 are displayed as detection frames in the image as local areas of the dog 1101. Similarly, the torso 1105 of the dog 1101 and the torso 1103 of the person 1102 are displayed as detection frames in the image as the entire area of ​​the dog 1101.

[0091] Figure 13(c) is a schematic diagram showing the results of defocus range estimation performed by the defocus range estimation unit 401 for the subject shown in Figure 13(a). The horizontal axis represents the magnitude of the defocus amount, and the length of the line segment represents the defocus range. Figure 13(c) shows the defocus range of the local areas, namely the dog's left pupil 1106 and the person's right pupil 1104, and the defocus range of the overall area, namely the dog's body 1105 and the person's body 1103. DoF represents the depth of field in Figure 13(a).

[0092] Figure 13(b) illustrates the case where multiple subjects of different types are shown in close-up within image 1110. The subjects are a dog 1111 and a person 1112. In addition, from the subject detection area acquired by the subject detection unit of the defocus range estimation unit 401, the dog's left pupil 1115 is displayed as a local area, and the dog's body 1114 and the person's body 1113 are displayed as detection frames within the image as overall areas.

[0093] Figure 13(d) is a schematic diagram showing the results of defocus range estimation performed by the defocus range estimation unit 401 on Figure 13(b). The defocus range of the dog's left pupil 1115 is shown as a local area, and the defocus range of the dog's body 1114 and the human body 1113 are shown as the overall area.

[0094] Figures 13(e) and 13(f) schematically represent the results of assigning priority to the defocus range estimation results in Figure 13(d).

[0095] Figure 12 shows a flowchart of the priority setting process in this embodiment.

[0096] In step S1001, the defocus range estimation unit 401 acquires the position and size of the subject detection area detected by the subject detection unit. For example, in Figure 13(a), for dog 1101, information on the subject detection area of ​​the left pupil 1106 as a local area and the body 1105 as an overall area is acquired. Similarly, for person 1102, information on the subject detection area of ​​the right pupil 1104 as a local area and the body 1103 as an overall area is acquired. On the other hand, in Figure 13(b), for dog 1111, information on the subject detection area of ​​the left pupil 1115 as a local area and the body 1114 as an overall area is acquired. For person 1112, the right pupil is hidden by dog ​​1111, so the right pupil cannot be detected as a local area. As a result, the left pupil 1115 and body 1114 of dog 1111, and the body 1113 of person 1112 are acquired as subject detection area information.

[0097] Next, in step S1002, the priority determination unit 4031 selects a part for priority determination from the subject detection area acquired in step S1001. In this embodiment, the entire area of ​​the subject detection area acquired in step S1001 is selected as the part for priority determination. In the case of person 1102 in Figure 13(a), since the position and size of the right pupil 1104 in the local area and the torso 1103 in the overall area are acquired as the subject detection area in step S1001, the torso 1103 is selected as the part for priority determination. Similarly, in the case of dog 1101 in Figure 13(a), the torso 1105 is selected as the part for priority determination. On the other hand, in the case of person 1112, the torso 1113 is selected as the part for priority determination. Similarly, the torso 1114 of dog 1111 is selected.

[0098] In step S1003, the priority determination unit 4031 determines whether to set a priority for the defocus range based on the priority determination area selected in step S1002. In this embodiment, whether or not to perform priority determination is determined by the area ratio of the subject detection area to the image. The priority determination unit 4031 calculates the area from the size information of the subject detection area of ​​the priority determination area selected in step S1002 and calculates the area ratio of each area by dividing it by the image size. If the calculated area ratio is greater than or equal to a threshold, the priority determination unit 4031 performs the process in step S1004. If the calculated area ratio is less than a threshold, or if no priority determination area was selected in step S1002, the priority determination unit 4031 terminates the priority setting process in step S901. The threshold to be set is, for example, 0.5.

[0099] In Figure 13(a), the parts selected for priority determination in step S1002 are the torso of the person 1103 and the torso of the dog 1105. For example, if the threshold is set to 0.5, the area ratio of the subject detection area to the image is less than or equal to the threshold for both. Therefore, the priority setting process in step S901 is terminated. On the other hand, in Figure 13(b), the parts selected for priority determination in step S1002 are the torso of the person 1113 and the torso of the dog 1114. If the area ratio is calculated in the same way as in Figure 13(a), the area ratio of the torso of the person 1113 is greater than or equal to the threshold of 0.5, and the process proceeds to step S1004.

[0100] A situation where the subject detection area for priority determination occupies a large proportion of the image is when the subject is shown in close-up. When the subject is shown in close-up, it is highly likely that the subject is the main subject intended by the user, that is, the subject that the user wants to focus on preferentially. Considering this, step S1003 determines whether to set the priority of the defocus range.

[0101] In step S1004, the priority determination unit 4031 sets a priority for the defocus range of the parts whose combination was determined in step S602. In this embodiment, the priority determination unit 4031 sets the priority by changing the defocus range of the parts whose combination was determined in step S602, based on the subject having parts whose area ratio was greater than or equal to a threshold in step S1003. More specifically, the priority determination unit 4031 changes the defocus range of all parts among the defocus ranges of the parts whose combination was determined in step S602 so that it is the same as the reference defocus range. Here, the defocus range of the subject having parts whose area ratio was greater than or equal to a threshold in step S1003 is used as the reference. That is, in this step, the defocus range used to control the imaging device 10 is changed so that it focuses on the subject having parts whose area ratio was greater than or equal to a threshold.

[0102] Figure 13(e) is a schematic diagram showing the result of setting the priority of this step on the defocus range estimation result in Figure 13(d). Each line segment in Figure 13(e) represents the depth of field (DoF) calculated according to the calculation formula in Figure 14, and the result of setting the priority on the defocus range of the parts to be combined in step S602. For example, consider the case where the parts to be combined in S602 are the dog's left pupil 1115 and the person's torso 1113, and it is determined in step S1003 to set priority on the person 1112. In this step, priority is set on the person's torso 1113 by changing the defocus range of the dog's left pupil 1115 to be the same as the defocus range of the person's torso 1113, which is the reference. In the case of Figure 13(e), the priority setting in this step makes the defocus range of the person's torso 1113 and the defocus range of the dog's left pupil 1115 the same. Therefore, from step S902 onward in the range setting unit 403, processing is performed based on the defocus range of the person's torso 1113, and the control unit 404 controls the shooting lens 201 to focus on the person's torso 1113. In this way, by determining the defocus range of a reference subject, the user can select the subject they want to focus on from among multiple subjects.

[0103] The priority of the defocus range may be set by comparing the defocus range with the depth of field and changing the defocus range used to control the imaging device 10. The method for calculating the depth of field will be explained with reference to Figure 14. Figure 14 is a calculation formula (approximate formula) for calculating the depth of field. Depth of field is the distance at which a photograph appears to be in focus, and is calculated in the system control unit 102 using the calculation formula shown in Figure 14(a). Information that serves as a criterion for whether or not the subject is in focus, namely the allowable circle of confusion diameter δ, and the set aperture value F (F number at the time of shooting) are stored in memory 104. In addition, information on the focal length f and the object distance L differs depending on the position of the photographic lens 201 and is stored in the memory in the lens control unit 205. For this reason, the imaging device 10 receives this information from the photographic lens 201 via communication. The imaging device 10 calculates the depth of field based on the received information. In the following explanation, the depth of field calculated by increasing the aperture F value by a certain amount according to the method in Figure 14 is compared with the defocus range of the area selected in step S1002, and a priority is set between the depth of field and the defocus range.

[0104] Figure 13(f) is a schematic diagram showing the result of setting priorities for the defocus range estimation results in Figure 13(d) in step S1004. The line segments in Figure 13(f) represent the depth of field (DoF) calculated according to the formula in Figure 14 and the depth of field (DoF(+1 stop)) calculated by increasing the aperture F value by one stop. The DoF and DoF+1 stop values ​​are converted according to the formula in Figure 14(b) for comparison with the defocus range estimation results.

[0105] The line segment representing the defocus range of the torso of person 1112 in Figure 13(f) represents the result of setting a priority based on a depth of field of DoF+1 stop relative to the defocus range of the torso of person 1112 in Figure 13(b). In other words, in Figure 13(f), priority is set for the defocus range by restricting the defocus range of the torso of person 1112 to fall within a depth of field of DoF+1 stop. By setting a priority for the defocus range in this way, it becomes possible to control focus while considering that the defocus range of the subject will be estimated to be wider the larger the proportion of the subject detection area of ​​the priority determination part occupies in the image and the closer the subject is to the subject. When the subject area is close up, it is thought that the distance between the subject and the imaging device 10 is short and the focal length of the lens unit 200 is long, so it is assumed that the shooting conditions are shallow depth of field. In that case, even a slight extension of the subject in the depth direction will result in a large difference in the amount of defocus, and it is thought that the defocus range estimation result of the defocus range estimation unit 401 will be wide. If the defocus ranges shown in S501 to S503 are combined based on a wider defocus range estimation result, and the control processing in S504 is performed, the accuracy of focus and aperture control may decrease. By setting a priority for the defocus range, it becomes possible to take into account the depth of the subject and capture an image in focus on any subject.

[0106] (Other embodiments) Furthermore, the present invention can also be realized by performing the following process: that is, supplying software (program) that realizes the functions of the above-described embodiment to a system or device via a network or various storage media, and having the computer (or CPU or MPU, etc.) of that system or device read and execute the program.

[0107] It should be noted that the above embodiments are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features.

[0108] Furthermore, this disclosure includes the following components.

[0109] (Composition 1) An imaging device for capturing images of multiple subjects, A defocus range estimation means for estimating the defocus range of each of the aforementioned multiple subjects, A target determination means for setting multiple combinations of the defocus ranges, A defocus range setting means sets a control defocus range from the combination of the plurality of defocus ranges based on the above setting, An imaging device characterized by having control means for controlling the imaging conditions of the imaging device based on the control defocus range.

[0110] (Configuration 2) The imaging device has a detection means that determines the position of each of the multiple subjects within the range that can be imaged, The imaging apparatus according to configuration 1, characterized in that the defocus range estimation means performs the estimation based on the detection result.

[0111] (Composition 3) The imaging device has a detection means that determines the size of each of the multiple subjects within the range that the imaging device can capture, The imaging apparatus according to configuration 1 or 2, characterized in that the defocus range estimation means performs the estimation based on the detection result.

[0112] (Composition 4) The imaging apparatus according to any one of configurations 1 to 3, characterized in that the target determination means sets a combination based on the user's selection from a plurality of pre-set combinations of defocus ranges.

[0113] (Composition 5) The imaging apparatus according to any one of configurations 1 to 4, characterized in that the target determination means sets the combination according to the user's shooting tendencies.

[0114] (Composition 6) The imaging apparatus according to any one of configurations 1 to 5, characterized in that the target determination means sets the combination according to the estimation result of the defocus range estimation means.

[0115] (Composition 7) The defocus range estimation means estimates the defocus range for each of the multiple parts of the multiple subjects, The aforementioned plurality of parts include a local region and an overall region that is wider than the local region. The imaging apparatus according to any one of configurations 1 to 6, characterized in that the target determination means combines the defocus ranges corresponding to the local region from among the defocus ranges of a plurality of parts to determine the target.

[0116] (Composition 8) The imaging apparatus according to configuration 2 or 3, characterized in that the target determination means performs a linking process that combines multiple subjects of different types based on the IoU of the subjects and the defocus range corresponding to each subject.

[0117] (Composition 9) The imaging apparatus according to configuration 8, characterized in that, when there are multiple combinations that can be linked, the linking process prioritizes selecting combinations of subjects whose defocus range is closest to the subject.

[0118] (Composition 10) The imaging apparatus according to any one of configurations 1 to 9, characterized in that the defocus range setting means controls the defocus range from the closest to the furthest of the plurality of subjects.

[0119] (Composition 11) The imaging apparatus according to any one of configurations 1 to 10, characterized in that the defocus range setting means sets a common range among the defocus ranges of the plurality of subjects as a control defocus range.

[0120] (Composition 12) The imaging apparatus according to any one of configurations 1 to 11, characterized in that the control means controls the lens of the imaging apparatus and controls the focus position.

[0121] (Composition 13) The imaging apparatus according to any one of configurations 1 to 12, characterized in that the control means adjusts the depth of field by controlling the aperture.

[0122] (Composition 14) The system includes a determination means for determining the priority of the defocus range for each of the multiple subjects, The imaging apparatus according to any one of configurations 1 to 13, characterized in that the defocus range setting means sets the controlled defocus range based on the result of the determination in the determination means.

[0123] (Composition 15) A method for controlling an imaging device, A defocus range estimation step that estimates the defocus range of each of multiple subjects, A target determination step of setting multiple combinations of the defocus ranges, A defocus range setting step in which a control defocus range is set from a combination of the defocus ranges of the multiple subjects based on the above setting, A control method characterized by comprising: a control step of controlling the imaging device based on the control defocus range.

[0124] (Composition 16) A program that causes a computer to function as an image processing device as described in any of configurations 1 to 14.

[0125] (Composition 17) A program that causes a computer to implement the control method described in Configuration 15. [Explanation of symbols]

[0126] 10 Imaging device 401 Defocus Range Estimation Unit 402 Target Determination Section 403 Range setting section 404 Control Unit

Claims

1. An imaging device for capturing images of multiple subjects, A defocus range estimation means for estimating the defocus range of each of the aforementioned multiple subjects, A target determination means for setting multiple combinations of the defocus ranges, A defocus range setting means sets a control defocus range from the combination of the plurality of defocus ranges based on the above setting, An imaging device characterized by having control means for controlling the imaging conditions of the imaging device based on the control defocus range.

2. The imaging device has a detection means that determines the position of each of the multiple subjects within the range that can be imaged, The imaging apparatus according to claim 1, characterized in that the defocus range estimation means performs the estimation based on the detection result.

3. The imaging device has a detection means that determines the size of each of the multiple subjects within the range that the imaging device can capture, The imaging apparatus according to claim 1, characterized in that the defocus range estimation means performs the estimation based on the detection result.

4. The imaging apparatus according to claim 1, characterized in that the target determination means sets a combination from a plurality of pre-set combinations of defocus ranges based on the user's selection.

5. The imaging apparatus according to claim 1, characterized in that the target determination means sets the combination according to the user's shooting tendencies.

6. The imaging apparatus according to claim 1, characterized in that the target determination means sets the combination according to the estimation result of the defocus range estimation means.

7. The defocus range estimation means estimates the defocus range for each of the multiple parts of the multiple subjects, The aforementioned plurality of parts include a local region and an overall region that is wider than the local region. The imaging apparatus according to claim 1, characterized in that the target determination means combines the defocus ranges corresponding to the local region from among the defocus ranges of a plurality of parts to determine the target.

8. The imaging apparatus according to claim 2, characterized in that the target determination means performs a linking process that combines a plurality of subjects of different types based on the IoUs of the subjects and the defocus range corresponding to each subject.

9. The imaging apparatus according to claim 3, characterized in that the target determination means performs a linking process that combines a plurality of subjects of different types based on the IoUs of the subjects and the defocus range corresponding to each subject.

10. The imaging apparatus according to claim 8, characterized in that, when there are multiple combinations that can be linked, the linking process prioritizes selecting combinations of subjects whose defocus range is closest to the subject.

11. The imaging apparatus according to claim 9, characterized in that, when there are multiple combinations that can be linked, the linking process prioritizes selecting combinations of subjects whose defocus range is closest to the subject.

12. The imaging apparatus according to claim 1, characterized in that the defocus range setting means controls the defocus range from the closest to the furthest of the plurality of subjects.

13. The imaging apparatus according to claim 1, characterized in that the defocus range setting means sets a common range among the defocus ranges of the plurality of subjects as a control defocus range.

14. The imaging apparatus according to claim 1, characterized in that the control means controls the lens of the imaging apparatus and controls the focus position.

15. The imaging apparatus according to claim 1, characterized in that the control means adjusts the depth of field by controlling the aperture.

16. The system includes a determination means for determining the priority of the defocus range for each of the multiple subjects, The imaging apparatus according to claim 1, characterized in that the defocus range setting means sets the controlled defocus range based on the result of the determination in the determination means.

17. A method for controlling an imaging device, A defocus range estimation step that estimates the defocus range of each of multiple subjects, A target determination step of setting multiple combinations of the defocus ranges, A defocus range setting step in which a control defocus range is set from a combination of the defocus ranges of the multiple subjects based on the above setting, A control method characterized by comprising: a control step of controlling the imaging device based on the control defocus range.

18. A program that causes a computer to function as an image processing device according to any one of claims 1 to 16.

19. A program that causes a computer to perform the control method described in claim 17.

Images