Information processing device, information processing method, and program

Abstract

To improve focusing accuracy based on auto-focusing in an image sensor in which an exposure condition can be set for each area.SOLUTION: Auto-focusing of an imaging apparatus in which a condition of exposure can be set for each partial area of a light receiving surface of an image sensor is performed. A first partial area in which imaging is performed using a first exposure condition and a second partial area which is adjacent to the first partial area and in which imaging is performed using a second exposure condition different from the first exposure condition are set on the light receiving surface. A third partial area which does not include an exposure-level difference which is a boundary portion between the first partial area and the secondary partial area is set as an area used for the auto-focusing. The auto-focusing of the imaging apparatus is controlled according to contrast of an image based on an image signal of the third partial area.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is a technology for changing exposure conditions for each region within an image sensor of an imaging device. Patent Document 1 discloses an imaging device that allows exposure conditions (exposure time and analog gain settings) to be changed for each region of the image sensor.

[0003] Furthermore, contrast AF is often adopted for autofocus (hereinafter referred to as AF) in imaging devices. In contrast AF, a focus lens, which is part of the imaging optical system and acts to adjust focus, is moved over a focus adjustment range while capturing an image, and high-frequency components are extracted from the output image signal within a partial area to sequentially calculate a contrast evaluation value for focusing. The position of the focus lens where this contrast evaluation value is maximized is then set as the in-focus position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-136205 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the technology described in Patent Document 1, because exposure conditions are set for each exposure area, there are cases where areas set to high gain (high gain areas) and areas set to low gain (low gain areas) are mixed within the image. In this case, an exposure step, where a sharp change in gain occurs, occurs at the boundary between the high gain area and the low gain area. If this exposure step is included in the area referenced by AF, the edge of the exposure step portion will unintentionally increase the contrast evaluation value, making it difficult to accurately focus on the plane.

[0006] An object of the present invention is to improve the focusing accuracy of autofocus in an image sensor that allows exposure conditions to be set for each area. [Means for solving the problem]

[0007] To achieve the object of the present invention, for example, an information processing apparatus according to one embodiment includes the following arrangement: That is, an information processing apparatus for performing autofocusing on an imaging device capable of setting exposure conditions for each partial region of a light receiving surface of an image sensor, comprising: a first setting means for setting, on the light receiving surface, a first partial region for performing imaging under first exposure conditions and a second partial region adjacent to the first partial region for performing imaging under second exposure conditions different from the first exposure conditions; a second setting means for setting, as a region to be used for the autofocusing, a third partial region that does not include an exposure step that is a boundary between the first partial region and the second partial region; and a control means for controlling the autofocusing of the imaging device in accordance with the contrast of an image based on an image signal of the third partial region. a display control means for displaying the area used for autofocus, which is set by the second setting means so as not to include the exposure step; The present invention is characterized by comprising: [Effects of the Invention]

[0008] To improve autofocus accuracy in an image sensor capable of setting exposure conditions for each area. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the functional configuration of an imaging system including an imaging device according to a first embodiment. [Figure 2] 4A to 4C are diagrams showing examples of images in which exposure differs for each region of the imaging device according to the first embodiment. [Figure 3] 5 is a flowchart showing an example of information processing performed by the imaging device according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of an AF area reset display of the imaging device according to the first embodiment. [Figure 5] FIG. 4 is a diagram showing an example in which an AF area of ​​the imaging device according to the first embodiment is set without any problems. [Figure 6] 10 is a flowchart showing an example of information processing performed by the imaging device according to the second embodiment. [Figure 7] 10A and 10B are diagrams for explaining correction of an AF area by an imaging device according to a second embodiment. [Figure 8] 10A to 10C are diagrams showing examples of AF areas before and after correction by an imaging device according to a second embodiment. [Figure 9] 10A and 10B are views showing another example of the AF area before and after correction by the imaging device according to the second embodiment. [Figure 10] 10A and 10B are views showing another example of the AF area before and after correction by the imaging device according to the second embodiment. [Figure 11] 11 is a flowchart showing an example of information processing by the imaging device according to the third embodiment. [Figure 12] 10A and 10B are diagrams showing an example of setting a plurality of AF areas by an imaging apparatus according to a third embodiment. [Figure 13] FIG. 4 is a diagram illustrating how an AF area is set based on a subject position by the imaging device according to the first embodiment. [Figure 14] FIG. 10 is a diagram illustrating how an AF area is set based on a subject position using an imaging device according to a third embodiment. [Figure 15] FIG. 10 is a diagram showing the functional configuration of a computer according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] [Embodiment 1] FIG. 1 is a block diagram showing an example of an imaging system including an imaging device, which is an information processing device according to this embodiment. The imaging device 100 is an imaging device that captures images, and is, for example, a digital camera, or a smartphone or tablet terminal equipped with a camera. In this embodiment, each process performed by the imaging device 100 is performed by a processor built into the imaging device 100. The client device 110 is a device that allows a user to check images captured by the imaging device 100 and operate the imaging device, and is, for example, a personal computer, a smartphone, or a tablet terminal. In this embodiment, the imaging device 100 and the client device 110 will be described as separate devices, but these devices may also be implemented as the same device.

[0012] The client device 110 according to this embodiment includes an input unit 111, a display unit 112, an operation unit 113, and an output unit 114. The display unit 112 is, for example, a liquid crystal display, and displays an image captured by the imaging device 100 and a UI for performing processing by each functional unit. The operation unit 113 is, for example, composed of a keyboard and a mouse, or a touch panel, and receives input from a user. The user can perform various inputs to the imaging device 100, such as setting an AF area, which will be described later, via the operation unit 113. The input unit 111 acquires various information from the imaging device 100, such as a captured image, and the output unit 114 transmits various information, such as setting an AF area, to the imaging device 100.

[0013] The imaging device 100 according to this embodiment includes an image sensor unit 101, a lens 102, an image processing unit 103, an output unit 104, a calculation unit 105, a lens control unit 106, a condition setting unit 107, and an input unit 108. The imaging device 100 forms an image on the light receiving surface of the image sensor unit 101, which is an image sensor, based on image light incident on the lens 102, and outputs image data. The image processing unit 103 can convert the image data output by the image sensor unit 101 into a format readable by a client device 110. The output unit 104 transmits each piece of data generated by the imaging device 100 to the client device 110. In this embodiment, the output unit 104 can output the image data converted by the image processing unit 103 to the client device 110.

[0014] The condition setting unit 107 sets exposure conditions for the image sensor unit 101 for each partial region of the light receiving surface. Here, the exposure conditions (hereinafter also referred to as exposure conditions) refer to setting values ​​related to the brightness of image data, such as the exposure time (shutter speed) or gain (analog gain or digital gain) value of the image sensor for each pixel. The image sensor unit 101 according to this embodiment forms an image of image light based on the exposure conditions set by the condition setting unit 107. The condition setting unit 107 can set exposure conditions for each pixel of the light receiving surface, but may also set exposure conditions for each unit region 203 obtained by equally dividing the light receiving surface of the image sensor unit 101 (here, divided into 16 equal parts, 4 × 4), corresponding to, for example, a screen 201 in FIG. 2 described below. The following description will be given assuming that the condition setting unit 107 sets exposure conditions for each partial region of the light receiving surface.

[0015] The condition setting unit 107 also sets an exposure step between areas with different exposure conditions. The exposure step according to this embodiment refers to the boundary between adjacent areas on the light-receiving surface that have different exposure conditions. Edges are likely to occur in the exposure step due to differences in brightness resulting from differences in exposure conditions. For this reason, the condition setting unit 107 sets a partial area on the light-receiving surface that does not include an exposure step as an area (AF area) where pixel values ​​are referenced in contrast AF. Note that the exposure step refers to the boundary between pixel areas with different exposure conditions (i.e., not a pixel area), but to more reliably exclude edge areas from the AF area, a pixel area, such as an area several pixels wide centered on the boundary, may be set as the exposure step.

[0016] The imaging device 100 performs contrast AF according to the contrast of the image based on the pixel values ​​of the AF area. Specifically, the imaging device 100 captures an image while adjusting the position of the lens 102 over the focus adjustment range, extracts high-frequency components from the image signal within the AF area, and sequentially calculates an evaluation value of the contrast for focusing (hereinafter, the contrast evaluation value). The imaging device 100 then sets the position of the lens 102 where the sequentially calculated contrast evaluation value is maximized as the focus position.

[0017] The lens control unit 106 controls the position of the lens 102 for contrast AF. The image processing unit 103 acquires image data from the image sensor unit 101 for each position of the lens 102 and transmits the output image signal within the AF area to the calculation unit 105. The calculation unit 105 calculates a contrast evaluation value for the chorus from the output image signal of the AF area for each position of the lens 102. Next, the calculation unit 105 sets the position of the lens 102 where the calculated contrast evaluation value is maximized as the focus position, and the lens control unit 106 controls the lens position to that focus position.

[0018] In this embodiment, autofocus is performed based on a contrast evaluation value of the output image signal of the AF area. The contrast evaluation value is an intra-image parameter for evaluating the level of contrast within the AF area. While any parameter used in a general contrast evaluation method may be used as the contrast evaluation value, the following description will be given assuming that the contrast evaluation value according to this embodiment is the sum of high-frequency components of the output image signal of the AF area.

[0019] 2 shows an example of display on the UI of the client device 110 of image data based on exposure conditions set by the condition setting unit 107, which is transmitted to the client device 110 by the output unit 104 according to this embodiment and displayed to the user. In this embodiment, an analog gain value (hereinafter simply referred to as gain) is set as the exposure condition, but as described above, different conditions such as exposure time may also be used. In the example of FIG. 2, the condition setting unit 107 sets exposure conditions for each unit area 203 obtained by dividing the screen 201 into 16 parts as described above, and the screen 201 is displayed in a color-coded manner according to the exposure conditions, with the color becoming darker as the gain value decreases. Each of the areas 204 to 207 is an area in which the same gain is set within the area (hereinafter referred to as a gain area).

[0020] The condition setting unit 107 sets an exposure step 210 at the boundary between each of the gain areas 204 to 207. In the example of FIG. 2, the condition setting unit 107 sets an area 202 on the screen 201 as an AF area, and this area 202 includes an exposure step 210 within the area. When performing contrast AF by referring to an area including an exposure step, the calculated contrast evaluation value often becomes unintentionally high due to the edge of the exposure step. Therefore, by performing contrast AF by setting an AF area so as not to include the exposure step, it is possible to improve the AF focusing accuracy. Note that in the screens shown in FIGS. 4 to 5, 7 to 10, and 12 to 14, which will be described later, the same exposure conditions as those in FIG. 2 are set for each area on the screen.

[0021] Here, the condition setting unit 107 may set all boundaries between regions with different exposure steps as exposure steps, or may set a condition for setting only those boundaries that satisfy a condition as exposure steps. For example, the condition setting unit 107 may set the boundary between those regions as an exposure step only when the difference in the set gain values ​​is equal to or greater than a predetermined threshold, taking into account cases where the difference in gain between the regions is slight and the resulting edges can be ignored. This setting can omit the setting of unnecessary exposure steps and reduce the processing load. In the following description, two regions having "the same exposure condition" will be considered to include not only cases where the exposure condition settings between the regions are the same, but also cases where the difference in the settings does not satisfy the condition for setting an exposure step.

[0022] In this example, the user can set an AF area on the screen 201 via a UI 208 displayed on the display unit 112 of the client device 110. The condition setting unit 107 may set the AF area in response to, for example, a touch operation by the user or a click operation using a mouse. Here, if an AF area including an exposure step is set by the user's input, the condition setting unit 107 may prompt the user to reset the AF area by notifying the user of this fact, or may set the AF area so that it does not include the exposure step while following the user's input. In addition, an AF setting frame 109 is displayed on the screen 201, and the user can select whether to set the AF area manually or automatically (auto) by operating the AF setting frame 109. The AF area may be displayed with a frame (e.g., a dotted line, a solid line, or a blinking line) on the screen 201 for a predetermined period after the user's input, depending on the user's input. Alternatively, the AF area may be displayed with the frame until the AF area setting is completed, or only AF may be performed without displaying the AF area.

[0023] When the AF area is set automatically, the condition setting unit 107 sets a partial area on the light receiving surface that is automatically acquired so as not to include an exposure step as the AF area. In this case, the condition setting unit 107 may set the AF area from a region of interest, or may set a partial area randomly acquired from the entire light receiving surface as the AF area. In this embodiment, the region of interest may be a region designated by the user, or may be a region where a subject to be tracked is located when the imaging device 100 has a function for tracking a subject in a captured image, and is not particularly limited as long as it is a region where focusing is expected.

[0024] If the imaging device 100 has a subject tracking function and the subject being tracked is located on an exposure step, the condition setting unit 107 can set an AF area based on the tracking position of the subject. For example, the condition setting unit 107 may set the tracking position of the subject as the AF area, or may set the entire or part of the gain area in which the subject was located immediately before moving onto the exposure step as the AF area. The condition setting unit 107 may also estimate the gain area to which a subject residing on the exposure step will next move, and set the entire or part of the estimated gain area as the AF area. Here, the condition setting unit 107 can estimate the gain area to which the subject will next move from among multiple gain areas in which the subject is located across the exposure step, using a known tracking technique. The process of manually setting the AF area will be described later.

[0025] Fig. 13 is a diagram illustrating an AF area determined by the image capture device 100 that tracks a subject, depending on the tracking position of the subject. In this figure, a pull-down list for selecting whether or not to perform tracking is displayed on the UI 208, and the tracking mode is selected by the user. In Fig. 13, a subject 1301 moving in the direction from the gain area 206 to the gain area 204 is located within a unit area that constitutes the gain area 206. Therefore, the condition setting unit 107 sets an AF area 1302 within the unit area of ​​the gain area 206 in which the subject 1301 is located.

[0026] 3 to 5, the process of setting an AF area performed by the imaging device 100 according to this embodiment when capturing an image including at least two partial areas with different exposure conditions will be described. In this example, at least two partial areas with different exposure conditions are set as described above, but if the exposure conditions for the entire light receiving surface are set to be the same, normal contrast AF is performed (i.e., no exposure step is set).

[0027] 3 is a flowchart showing an example of processing by the image capturing apparatus 100 according to this embodiment. In S301, the condition setting unit 107 sets the exposure conditions of the image capturing element unit 101 for each partial region of the light receiving surface.

[0028] In S302, the condition setting unit 107 determines whether exposure conditions have been set for the entire area of ​​the light receiving surface of the image sensor. If so, the process proceeds to S303; if not, the process returns to S301.

[0029] In S303, the condition setting unit 107 sets areas on the light receiving surface that have the same exposure condition. In the example of Fig. 2, the condition setting unit 107 sets areas 204 to 207 as areas that have the same exposure condition.

[0030] In S304, the condition setting unit 107 sets exposure steps based on each region that has the same exposure condition. In the example of Fig. 2, the condition setting unit 107 sets each boundary between regions 204 to 207 as exposure step 210. As described above, if the exposure condition between regions is smaller than a predetermined threshold, the condition setting unit 107 does not need to set an exposure step at that boundary.

[0031] In S305, the condition setting unit 107 determines whether to set the AF area automatically or manually. In this example, the condition setting unit 107 makes the determination by referring to a user input to the AF setting frame 209 in Fig. 2, but, for example, the AF area may be set to be set manually (or automatically) as an initial setting. If the AF area is to be set automatically, the process proceeds to S306, and if it is to be set manually, the process proceeds to S307.

[0032] In S306, the condition setting unit 107 automatically sets the AF area so that the exposure step is not included, and the process proceeds to S311. In step S307, the condition setting unit 107 presents the exposure step to the user, and the process proceeds to S308. The condition setting unit 107 may present the exposure step 210 by displaying it as a dotted line, as shown in FIG. 2, for example, but the method of presentation is not particularly limited as long as it is a display format that is recognizable to the user, such as displaying it as a line of a predetermined color (such as red). Note that S307 may be omitted, and the process may proceed to setting the AF area without displaying the exposure step.

[0033] In S308, the condition setting unit 107 acquires user input from the client device 110 and sets the AF area based on the acquired input. In S309, the condition setting unit 107 determines whether the AF area set in S308 includes an exposure step. If an exposure step is included, the process proceeds to S310, and if not, the process proceeds to S311.

[0034] In S310, the condition setting unit 107 notifies the user that the AF area to be set includes an exposure step, and prompts the user to reset the AF area. FIG. 4 is a diagram showing an example of a message displayed in S310. In the example of FIG. 4, a message prompting the user to reset the AF area, stating "Please reset the AF area," is displayed on the UI 208. Note that the method of prompting the user to reset the AF area by the condition setting unit 107 is not limited to this. For example, the condition setting unit 107 may notify the user by voice, or may display a message indicating that the setting failed (for example, an "X" mark) and return to the setting screen. Next, the process returns to S308, and the processes of S308 to S310 are repeated until the AF area is set within the area with the same exposure condition as shown in FIG. 5. FIG. 5 is a diagram showing a case where the AF area 501 has been set within the area 206 with the same exposure condition on the screen of FIG. 2.

[0035] In S311, the calculation unit 105 calculates a contrast evaluation value from the output image signal of the AF area, and the lens control unit 106 drives the lens 102 to a position where the calculated contrast evaluation value becomes the highest.

[0036] This process allows the AF area to be set while avoiding the positions of the exposure gaps, which can prevent high-frequency components from appearing at the edges of the exposure gaps, which can cause noise in the calculation of the AF contrast evaluation value, thereby improving focusing accuracy.

[0037] [Embodiment 2] The imaging device according to the first embodiment sets an AF area so as not to include an exposure step, and performs AF by calculating a contrast evaluation value from the output image signal of the AF area. The imaging device according to this embodiment acquires a user's designation of an AF area and sets the AF area by correcting the designated area. Here, if the designated area includes an exposure step, the condition setting unit 107 sets the corrected area so as not to include the exposure step as the AF area. In this embodiment, the imaging device 100 and the client device 110 have the same configuration as those shown in FIG. 1. Furthermore, the exposure step setting process and the processes subsequent to the setting of the AF area are performed in the same manner as in the first embodiment, and therefore redundant description will be omitted.

[0038] In this embodiment, the user inputs a designation of an AF area via the operation unit 113 of the client device 110. Next, the condition setting unit 107 sets an AF area that does not include an exposure step, based on the AF area (designated area) designated by the user input. Here, if the designated area does not include an exposure step, the condition setting unit 107 may set the designated area as the AF area as is. On the other hand, if the designated area includes an exposure step, the condition setting unit 107 sets the AF area by correcting the designated area so that the exposure step is not included in the area.

[0039] The following description will be given assuming that the specified area basically includes an exposure step. The condition setting unit 107 can extract an area where the specified area and a gain area overlap on the light receiving surface of the image sensor, and set the extracted area as an AF area. Furthermore, if the specified area has an area that overlaps with two or more gain areas, the condition setting unit 107 may set the area that overlaps with one of the gain areas as the AF area, or may set areas that overlap with each of the multiple gain areas as the AF area. A detailed description of the case where multiple AF areas are set will be given in the third embodiment. Furthermore, here, the condition setting unit 107 may set the entire area where the specified area and the gain area overlap as the AF area, or may set an area that excludes a width of a predetermined number of pixels from a boundary that constitutes an exposure step as the AF area in order to reduce the possibility of being affected by an exposure step.

[0040] 7 is a diagram illustrating an example in which the condition setting unit 107 according to this embodiment sets an AF area based on a specified area. In Fig. 7, an AF area 701 is set as the AF area. Here, the area 202 in Fig. 2 is specified as the specified area, and the area 202 includes the exposure step 210. Therefore, the condition setting unit 107 sets the area 702 where the area 202 and the gain area 204 overlap as the AF area.

[0041] Furthermore, the condition setting unit 107 may set an overlapping area between a specified area and a gain area as an AF area if the area satisfies a predetermined condition. An example of setting an area satisfying the condition as an AF area will be described below. For example, the condition setting unit 107 may set an overlapping area between a specified area and a gain area as an AF area if the area is equal to or greater than a threshold (note that the area here refers to the area on the light receiving surface). The threshold value for the area set here is not particularly limited. For example, the threshold value for the area may be set as a desired value by the user, or may be equal to or greater than half the area of ​​the gain area, or may be the average area of ​​the overlapping areas between each gain area and the specified area (area of ​​the specified area / number of gain areas having an overlapping area with the specified area). In this case, if the area of ​​any of the overlapping areas between the specified area and the gain area is less than the threshold, the condition setting unit 107 notifies the user to urge them to reset the AF area. Ensuring a sufficient area for the AF area improves the reliability of the contrast evaluation value calculated from the AF area.

[0042] Furthermore, for example, when there are multiple areas where the designated area and the gain area overlap, the condition setting unit 107 may set the area with the largest area among them as the AF area. FIG. 8 is a diagram illustrating an example in which the area with the largest area is set as the AF area. FIG. 8(a) is a diagram showing a designated area 801 indicated by a dotted line, and FIG. 8(b) is a diagram showing an AF area 802 set based on the designated area 801. The designated area 801 overlaps with three gain areas 204 to 206. Here, the condition setting unit 107 selects the gain area 204 from the three gain areas as the area with the largest overlap with the designated area, and sets the area where the designated area 801 and the gain area 204 overlap as the AF area 802.

[0043] Furthermore, for example, if the exposure condition (here, the amount of gain) of a gain area overlapping the specified area is smaller than a threshold, the condition setting unit 107 may set the area where the gain area overlaps with the specified area as an AF area. The gain threshold set here may be set as a value desired by the user, may be the average value of the gain amounts of each gain area, or may be the average value of the gain within the specified area, and is not particularly limited. In this case, if a gain amount equal to or greater than the threshold is set in any of the gain areas that overlap with the specified area, the condition setting unit 107 notifies the user to urge them to reset the AF area. By sufficiently reducing the gain value of the AF area, it is possible to reduce the possibility that contrast will not be evaluated correctly due to excessive gain.

[0044] For example, when there are multiple regions where the designated region and the gain region overlap, the condition setting unit 107 may set, as the AF region, a region where the designated region overlaps with a region with the smallest amount of gain among the multiple gain regions. Furthermore, for example, the condition setting unit 107 may set, as the AF region, a region where a gain region that is a region of interest among the gain regions overlaps with the designated region. Furthermore, the condition setting unit 107 may set, as the AF region, a region where a region selected by the user among the multiple gain regions overlaps with the designated region.

[0045] The condition setting unit 107 may set an area satisfying one of the above-described conditions as the AF area, or may set an area satisfying two or more of these conditions as the AF area. Fig. 9 shows an example in which, among areas where a designated area and a gain area overlap, an area whose area is equal to or greater than a threshold and whose gain value is the smallest is set as the AF area. Fig. 9(a) shows a designated area 901 indicated by a dotted line, and Fig. 9(b) shows an AF area 902 set based on the designated area 901. The designated area 901 overlaps with three gain areas 204 to 206. Of the three gain areas, area 204 and area 206 are the two areas whose areas overlap with the designated area are equal to or greater than the threshold, and the area where area 206, which has the smallest gain value, overlaps with the designated area is set as the AF area 902.

[0046] Here, an example has been described in which an area overlapping with a gain area from a designated area is extracted as an AF area, that is, an example in which the designated area is corrected by shrinking the designated area. However, the condition setting unit 107 may set an AF area by enlarging the designated area as the correction. This may be a process of enlarging the designated area as is, or a process of enlarging an area extracted from the designated area where the designated area and gain area overlap. Hereinafter, the process of enlarging such a corrected designated area will be referred to as a process of "enlarging a designated area."

[0047] For example, the condition setting unit 107 may enlarge the designated area when a user's instruction to enlarge the AF area is received, or when the designated area satisfies an enlargement condition. Here, the condition for enlarging the designated area is not particularly limited, and may be, for example, when the area of ​​the designated area is equal to or smaller than a predetermined value, or when an improvement in AF accuracy in the designated area is expected. Furthermore, the user's instruction to enlarge the AF area may be received by, for example, pressing an enlargement execution button (not shown) provided on the UI 208. Furthermore, for example, the instruction to enlarge the AF area may be received by a user's input to the screen 201 (for example, designating an area by a drag operation using a cursor or a swipe operation via a touch panel).

[0048] The condition setting unit 107 enlarges the designated region so that the exposure step is not included within the region, and sets the enlarged designated region as an AF region. Here, the condition setting unit 107 may enlarge the designated region according to a user input, or may automatically enlarge the designated region (for example, so that the entire gain region where the designated region overlaps becomes the AF region). FIG. 10 is a diagram illustrating an example of a designated region enlarged in this manner. In FIG. 10(a), a user inputs an instruction to enlarge an AF region 1001 to a region 1002. In the example of FIG. 10(a), the exposure conditions are set uniformly throughout the entire region, so that the exposure conditions are also uniform throughout the region 1002. Therefore, the condition setting unit 107 sets the AF region to the region 1002. On the other hand, in FIG. 10(b), the region 1002 spans multiple gain regions, so the AF region is not enlarged to the region 1002. Therefore, in response to an instruction to expand the AF area 1001, the condition setting unit 107 limits the expansion range to within the gain area 204 and sets the area 1003 as the AF area. This is an example showing an AF area expanded to the maximum extent from the area 1001, and correction may be performed to satisfy desired conditions, such as setting an AF area obtained by excluding a predetermined number of pixels width from the boundary portion that forms the exposure step 210 from the AF area 1003.

[0049] Fig. 6 is a flowchart showing an example of processing by the imaging device 100 according to this embodiment. The processing in Fig. 6 is performed in the same manner as the processing shown in Fig. 3 of the first embodiment, except that S601 is added as processing subsequent to S307, and processing of S602 and S603 is performed instead of S308 and S310, respectively, and therefore a duplicated description will be omitted.

[0050] In S601, the imaging device 100 acquires a designated area input by the user via the client device 110. Subsequently, in S602, the condition setting unit 107 sets an AF area by referring to the acquired designated area, and the process proceeds to S309. Here, the AF area is set by correcting the designated area, for example, as described above.

[0051] In S603, if it is determined in S309 that the AF area includes an exposure step, the condition setting unit 107 reduces (corrects) the AF area so that it does not include the exposure step. For example, as described above, the condition setting unit 107 can correct the AF area by extracting an area that overlaps with the gain area. After S603, the process proceeds to S311.

[0052] This process corrects the area specified by the user and sets an AF area that does not include exposure steps. Therefore, it is possible to avoid high-frequency components due to the edges of the exposure step positions, which become noise in the calculation of the AF contrast evaluation value, thereby improving focusing accuracy. Furthermore, by correcting the specified area, the AF area can be set so that the area or exposure conditions satisfy predetermined conditions, allowing AF to be performed under more optimal conditions.

[0053] Note that, if the specified area includes an exposure step within the area, the condition setting unit 107 may set the specified area as an AF area as is by uniformly changing the exposure conditions over the entire specified area only when calculating the AF contrast evaluation value. That is, the condition setting unit 107 may set an AF area that does not include an exposure step within the area by temporarily erasing the exposure step when calculating the contrast evaluation value. By performing processing in this manner, it becomes possible to set an AF area that does not include an exposure step using the area specified by the user, and perform AF processing.

[0054] [Embodiment 3] In the first and second embodiments, AF is performed by setting one AF area and then controlling the lens to a lens position that maximizes the contrast evaluation value of that AF area. Image capture device 100 according to this embodiment sets multiple AF areas and performs AF based on the contrast evaluation value of each of the multiple AF areas that have been set.

[0055] The condition setting unit 107 according to this embodiment acquires a designated area in the same manner as in the second embodiment. Next, if the designated area includes an exposure step within the area, the condition setting unit 107 sets multiple AF areas that do not include an exposure step within the area based on the designated area. Here, for example, if the designated area has areas that overlap with two or more gain areas, the condition setting unit 107 extracts areas that overlap with the designated area for each of the gain areas in the same manner as in the second embodiment. Next, the condition setting unit 107 selects multiple of the extracted areas as AF areas and controls AF based on the contrast evaluation value calculated from each of the selected areas. Here, the condition setting unit 107 may set AF areas from all gain areas that overlap with the designated area, or may set thresholds for the gain value or the area of ​​the area that overlaps with the designated area, and set only areas that satisfy conditions for the thresholds as AF areas. The gain value threshold and area threshold here are set in the same manner as in the second embodiment, but may also be set to values ​​different from those in the second embodiment.

[0056] The calculation unit 105 according to this embodiment calculates a contrast evaluation value from output image signals of the multiple AF areas, and the lens control unit 106 drives the lens 102 to a position where the calculated contrast evaluation value is highest. The calculation unit 105 may, for example, calculate a contrast evaluation value from each of the multiple AF areas in the same manner as in embodiment 1, and calculate the contrast evaluation value according to this embodiment based on each calculated value. For example, the calculation unit 105 may use, as the contrast evaluation value used for AF, the average value, median value, or weighted average value of the contrast evaluation values ​​calculated from each AF area, weighted according to the area of ​​the AF area on the light receiving surface.

[0057] 12 is a diagram illustrating an example in which multiple AF areas are set. Here, area 202 in FIG. 2 is specified as the specified area, and area 202 includes an exposure step. Therefore, condition setting unit 107 sets the area where area 202 and gain area 204 overlap as AF area 1201, and the area where area 202 and gain area 206 overlap as AF area 1202. In the example of this diagram, area 202 is divided to set two AF areas that do not include an exposure step, but the specified area may be used as the AF area as is, and the exposure conditions within the AF area may be changed uniformly only when the AF contrast evaluation value is calculated.

[0058] Furthermore, if the imaging device 100 has a subject tracking function and the subject being tracked is located on an exposure step, the condition setting unit 107 may set multiple AF areas from multiple gain areas in which the subject is located across the exposure step, based on the tracking position of the subject. For example, the condition setting unit 107 can set AF areas from the gain area in which the subject was located immediately before moving onto the exposure step and the gain area to which the subject on the exposure step is estimated to move next. When AF is performed by referring to the gain area in which the subject was located immediately before, it is possible to maintain the focus position even while the subject is moving on the exposure step. When AF is performed by referring to the gain area to which the subject is estimated to move, it is possible to adjust the focus position to a position where the subject will be in focus after moving off the exposure step. When AF areas are set using both of these areas, it is possible to perform focusing while taking into account both the subject's movements before and after moving onto the exposure step.

[0059] 14 is a diagram illustrating an AF area determined by the imaging device 100 tracking a subject according to the tracking position of the subject. In FIG. 14, the subject 1301 similar to that in FIG. 13 is located at the boundary (exposure step 210) between the gain area 206 and the gain area 204. In this case, the condition setting unit 107 may set an AF area 1401 in the gain area 206 in which the subject 1301 was located immediately before it was located on the exposure step 210. The condition setting unit 107 may also set an AF area 1402 in the gain area 204 in which it is estimated that the subject 1301 will move next based on the tracking result, or may perform AF using both the AF area 1401 and the AF area 1402.

[0060] The processing performed by the imaging device 100 according to this embodiment will be described below with reference to Fig. 11. Fig. 11 is a flowchart showing an example of processing by the imaging device 100 according to this embodiment. The processing in Fig. 11 is performed in the same manner as the processing shown in Fig. 6 of the second embodiment, except that S1101 to S1103 are performed as processing subsequent to S309 instead of S603, and therefore a duplicated description will be omitted.

[0061] In S309, the condition setting unit 107 determines whether the AF area set in S602 includes an exposure step within the area. If an exposure step is included, the process proceeds to S1101, and if not, the process proceeds to S311.

[0062] In S1101, the condition setting unit 107 sets multiple AF areas based on the AF area set in S602. The condition setting unit 107 sets multiple AF areas by dividing the AF area (designated area), for example, as shown in FIG.

[0063] In S1102, the condition setting unit 107 calculates a contrast evaluation value from each of the set AF areas. In this process, the same calculation as in S311 is performed for each AF area.

[0064] In S1103, the condition setting unit 107 calculates a contrast evaluation value to be used for AF based on each of the contrast evaluation values ​​calculated in S1102. As described above, the condition setting unit 107 can calculate, for example, the average value of each evaluation value as the final contrast evaluation value. The lens control unit 106 drives the lens 102 to a position where the contrast evaluation value calculated in S1103 is the highest.

[0065] This process allows multiple AF areas to be set so as to avoid the positions of exposure gaps, which can prevent high-frequency components from appearing at the edges of exposure gaps, which can cause noise in the calculation of AF contrast evaluation values, thereby improving focusing accuracy.

[0066] [Embodiment 4] In the above-described embodiments, each processing unit shown in, for example, FIG. 1 is realized by dedicated hardware, such as a camera serving as an information processing device, as described above. However, some or all of the processing units of the imaging device 100 (and the client device 110) may be realized by a computer. In this embodiment, at least some of the processing according to each of the above-described embodiments is executed by a computer.

[0067] FIG. 15 is a diagram showing the basic configuration of a computer. In FIG. 15, processor 1501 is, for example, a CPU, and controls the operation of the entire computer. Memory 1502 is, for example, a RAM, and temporarily stores programs, data, etc. Computer-readable storage medium 1503 is, for example, a hard disk or a CD-ROM, and stores programs, data, etc. long-term. In this embodiment, programs that realize the functions of each unit, which are stored in storage medium 1503, are read into memory 1502. Then, processor 1501 operates in accordance with the programs on memory 1502, thereby realizing the functions of each unit.

[0068] 15, an input interface 1504 is an interface for acquiring information from an external device. When the client device 110 is an external device of the imaging device 100, the input interface 1504 may acquire information input by a user from the client device 110. Furthermore, an output interface 1505 is an interface for outputting information to an external device. A bus 1506 connects the above-mentioned units and enables data exchange.

[0069] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0070] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0071] 100: imaging device, 101: imaging element section, 102: lens, 103: image processing section, 104: output section, 105: calculation section, 106: lens control section, 107: condition setting section, 108: input section, 110: client device, 111: input section, 112: display section, 113: operation section, 114: input section

Claims

1. An information processing device that performs autofocusing on an imaging device that can set exposure conditions for each partial area of ​​the light receiving surface of an image sensor, a first setting means for setting, on the light receiving surface, a first partial area for which an image is captured under a first exposure condition, and a second partial area adjacent to the first partial area for which an image is captured under a second exposure condition different from the first exposure condition; a second setting means for setting a third partial area that does not include an exposure step that is a boundary between the first partial area and the second partial area as an area to be used for autofocus; a control unit that controls autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region; a tracking means for tracking a subject in an image captured by the imaging device, the second setting means sets the third partial area from an area of ​​the subject on the light receiving surface; The information processing device is characterized in that, when the subject is located on the exposure step, the second setting means sets the third partial area from the partial area, of the first partial area and the second partial area, in which the subject was located immediately before moving onto the exposure step.

2. the first exposure condition and the second exposure condition are exposure time or gain value, The information processing apparatus according to claim 1 , wherein a difference between the first exposure condition and the second exposure condition is equal to or greater than a predetermined threshold value.

3. a first acquisition means for acquiring a user input specifying an area to be used for the autofocus; 3. The information processing apparatus according to claim 1, wherein the second setting means sets the third partial region that does not include the exposure step based on the user input.

4. 4. The information processing device according to claim 3, wherein the second setting means, when a fourth partial area including the exposure step on the light receiving surface is specified by the user input, sets an area obtained by correcting the fourth partial area so that the exposure step is not included within the area as the third partial area.

5. 5. The information processing device according to claim 4, wherein the second setting means, when there is an area that exceeds a predetermined area among the overlapping area between the fourth partial area and the first partial area and the overlapping area between the fourth partial area and the second partial area, sets the area that exceeds the predetermined area as the third partial area.

6. 6. The information processing device according to claim 4, wherein the second setting means sets the larger area of ​​the overlapping area between the fourth partial area and the first partial area and the overlapping area between the fourth partial area and the second partial area as the third partial area.

7. 5. The information processing device according to claim 4, characterized in that the second setting means, when there is an area where the amount of gain is smaller than a predetermined threshold value among the area where the fourth partial area and the first partial area overlap and the area where the fourth partial area and the second partial area overlap, sets the area where the amount of gain is smaller than the predetermined threshold value as the third partial area.

8. 8. The information processing device according to claim 4, wherein the second setting means sets the area with a smaller amount of gain as the third partial area from among the area where the fourth partial area and the first partial area overlap and the area where the fourth partial area and the second partial area overlap.

9. a second acquisition means for acquiring a user's instruction to enlarge the area used for the autofocus from the third partial area; 9. The information processing device according to claim 1, wherein the control means controls the autofocus of the imaging device in accordance with the contrast of an image based on an image signal of a fifth partial area of ​​the light receiving surface expanded from the third partial area based on a user instruction acquired by the second acquisition means.

10. a third acquisition means for acquiring a user input specifying an area to be used for the autofocus; 3. The information processing device according to claim 1, wherein the control means controls the autofocus of the imaging device based on the contrast of an image based on an image signal of the sixth partial area when exposure conditions are set uniformly across the entire sixth partial area when a sixth partial area including the exposure step is specified on the light receiving surface by user input acquired by the third acquisition means.

11. the second setting means further sets a seventh partial area of ​​the light receiving surface, which is different from the third partial area and does not include the exposure step, as the area to be used for the autofocus; 11. The information processing device according to claim 1, wherein the control means controls autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region and the contrast of an image based on the image signal of the seventh partial region.

12. The information processing device according to claim 11, characterized in that the control means controls the autofocus of the imaging device by adjusting the position of the lens of the imaging device so that the average value of the evaluation value of the contrast of the image based on the image signal of the third partial region and the evaluation value of the contrast of the image based on the image signal of the seventh partial region is maximized.

13. further comprising a tracking means for tracking a subject in an image captured by the imaging device, 3. The information processing apparatus according to claim 1, wherein the second setting means sets the third partial area from an area of ​​the subject on the light receiving surface.

14. An information processing device that performs autofocusing of an imaging device that can set exposure conditions for each partial area of ​​a light receiving surface of an image sensor, a first setting means for setting, on the light receiving surface, a first partial area for which an image is captured under a first exposure condition, and a second partial area adjacent to the first partial area for which an image is captured under a second exposure condition different from the first exposure condition; a second setting means for setting a third partial area that does not include an exposure step that is a boundary between the first partial area and the second partial area as an area to be used for autofocus; a control unit that controls autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region; a tracking means for tracking a subject in an image captured by the imaging device; an estimation means for estimating a partial region to which the subject will next move from the first partial region and the second partial region when the subject is located on the exposure step, The information processing device is characterized in that the second setting means sets the third partial area from the area of ​​the subject on the light receiving surface, and sets the third partial area from the partial area to which the subject will next move, estimated by the estimation means.

15. the second setting means further sets an eighth partial area of ​​the light receiving surface, which is different from the third partial area and does not include the exposure step, from a partial area of ​​the first partial area and the second partial area where the subject was located immediately before moving onto the exposure step; 15. The information processing device according to claim 14, wherein the control means controls autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region and the contrast of an image based on the image signal of the eighth partial region.

16. The information processing device according to claim 1 , wherein the information processing device is built into the imaging device.

17. An information processing method for performing autofocusing on an imaging device capable of setting exposure conditions for each partial area of ​​a light receiving surface of an image sensor, comprising: setting, on the light receiving surface, a first partial area in which an image is captured under a first exposure condition, and a second partial area adjacent to the first partial area in which an image is captured under a second exposure condition different from the first exposure condition; setting a third partial area that does not include an exposure step that is a boundary between the first partial area and the second partial area as an area to be used for autofocus; controlling autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region; and tracking a subject in an image captured by the imaging device, the third partial area is set from an area of ​​the subject on the light receiving surface, An information processing method characterized in that, when the subject is located on the exposure step, the third partial area is set from the partial area between the first partial area and the second partial area in which the subject was located immediately before moving onto the exposure step.

18. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 16.

Images