IMAGING APPARATUS, FOCUS CONTROL METHOD, AND FOCUS CONTROL PROGRAM

Abstract

The present invention provides an imaging device, a focus control method, and a focus control program which make it possible to improve focus accuracy.　A digital camera (100) detects a subject area as a second area e2 from pixel data D2 obtained at a time t2 by an imaging element (5) for capturing an image of a subject through an imaging lens (1). Further, the digital camera (100) outputs, to a lens drive mechanism of the imaging lens (1), a control signal generated selectively using a first focus position based on data of the second area e2 in the pixel data D2 obtained at the time t2 and a second focus position based on data of the second area e2 in pixel data D5 obtained at a time t3 later than the time t2 by the imaging element (5).

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, a focus control method, and a focus control program. [Background technology]

[0002] In recent years, with the increasing resolution of imaging elements such as CMOS (Complementary Metal Oxide Semiconductor) image sensors, the demand for information devices with imaging functions such as digital still cameras, digital video cameras, and smartphones is rapidly increasing. Note that such information devices with imaging functions are called imaging devices. These imaging devices employ a contrast AF (Auto Focus) method or a phase difference AF method as a focus control method for focusing on a main subject.

[0003] Patent document 1 describes an imaging device that uses a focus assist function, which is an auxiliary function for manual focus that indicates to a user the extent to which the main subject area is out of focus based on the defocus amount, to update the defocus amount of the main subject area multiple times within one frame period by reading out only the main subject area from the image sensor in addition to the normal readout from the image sensor within one frame period.

[0004] Patent document 2 describes an autofocus adjustment device that sets a detection area corresponding to an area including a subject image detected for each frame image represented by a read-out image signal, detects a defocus amount based on the image signal corresponding to the set detection area, and determines a lens drive amount for driving a focus lens based on the detected defocus amount. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2017-125985 [Patent Document 2] Japanese Patent Publication No. 2010-101972 Summary of the Invention

[0006] One embodiment of the technique of the present disclosure provides an imaging apparatus, a focus control method, and a focus control program capable of improving focus accuracy. [Means for solving the problem]

[0007] An imaging device according to one embodiment of the technology disclosed herein is an imaging device comprising an imaging element that images a subject through an imaging optical system, and a processor, wherein the processor detects a subject area as a second area from pixel data obtained by the imaging element at a second time, and outputs a control signal to a lens driving mechanism of the imaging optical system, selectively using a first focus position based on data for the second area in the pixel data obtained at the second time and a second focus position based on data for the second area in pixel data obtained by the imaging element at a third time after the second time.

[0008] A focus control method according to one embodiment of the technology disclosed herein is a focus control method using an imaging device including an imaging element that images a subject through an imaging optical system and a processor, in which the processor detects a subject area as a second area from pixel data obtained by the imaging element at a second time, and outputs a control signal to a lens driving mechanism of the imaging optical system, selectively using a first focus position based on data for the second area in the pixel data obtained at the second time and a second focus position based on data for the second area in pixel data obtained by the imaging element at a third time after the second time.

[0009] A focus control program of one embodiment of the technology disclosed herein is a focus control program for an imaging device that includes an imaging element that images a subject through an imaging optical system and a processor, and causes the processor to execute a process of detecting a subject area as a second area from pixel data obtained by the imaging element at a second time, and outputting a control signal to a lens driving mechanism of the imaging optical system, the control signal being generated selectively using a first focus position based on data for the second area in the pixel data obtained at the second time and a second focus position based on data for the second area in pixel data obtained by the imaging element at a third time after the second time. Effect of the Invention

[0010] According to the present invention, it is possible to provide an imaging apparatus, a focus control method, and a focus control program capable of improving focus accuracy. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing a schematic configuration of a digital camera 100 which is an embodiment of an imaging apparatus of the present invention. [Diagram 2] 1 is a schematic plan view showing an example of the configuration of an image sensor 5 mounted on a digital camera 100. FIG. [Diagram 3] 3 is a partial enlarged view of one AF area 53 shown in FIG. 2. FIG. [Figure 4] 4 is a diagram showing phase difference detection pixels constituting an arbitrary pair row shown in FIG. 3. FIG. [Diagram 5] 2 is a diagram showing a cross-sectional configuration of a phase difference detection pixel 52A. FIG. [Figure 6] 1 shows a configuration in which all pixels 51 included in an imaging element 5 are used as imaging pixels, and each pixel 51 is divided into two. [Figure 7] 2 is a diagram showing an example of the timing of each operation of the digital camera 100. FIG. [Figure 8] 8A to 8C are diagrams illustrating an example of a focus operation of the digital camera 100 corresponding to FIG. 7. [Figure 9] 11 is a diagram showing another example of the timing of each operation of the digital camera 100. FIG. [Figure 10] 10A and 10B are diagrams illustrating an example of a focus operation of the digital camera 100 corresponding to FIG. 9. [Figure 11] 10 is a flowchart showing an example of a process for driving a lens by the digital camera 100. [Figure 12] 12 is a flowchart showing an example of subject movement determination in step S115 of FIG. [Figure 13] 13A and 13B are diagrams illustrating still another example of the focus operation of the digital camera 100. [Figure 14] 13 is a diagram showing an example of an overlapping portion between the second area e2 and the first area e1. FIG. [Figure 15] FIG. 13 is a diagram showing an example of AF prediction based on phase difference information in the overlap area e3. [Figure 16] 2 shows the external appearance of a smartphone 200 which is another embodiment of the imaging device of the present invention. [Figure 17] 17 is a block diagram showing a configuration of the smartphone 200 shown in FIG. 16. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] FIG. 1 is a diagram showing a schematic configuration of a digital camera 100 which is an embodiment of an imaging apparatus of the present invention.

[0014] The digital camera 100 shown in FIG. 1 includes a lens device 40 having an imaging lens 1, an aperture 2, a lens control unit 4, a lens driving unit 8, and an aperture driving unit 9.

[0015] In this embodiment, the lens device 40 may be detachable from the body of the digital camera 100, or may be fixed to the body of the digital camera 100.

[0016] The imaging lens 1 and the aperture 2 constitute an imaging optical system, which includes at least a focus lens. This focus lens is a lens for adjusting the focus position of the imaging optical system, and is composed of a single lens or multiple lenses. The focus position is adjusted by moving the focus lens in the optical axis direction of the imaging optical system.

[0017] The focus lens may be a liquid lens whose curved surface can be variably controlled to change the focus position.

[0018] The lens control unit 4 of the lens device 40 is configured to be able to communicate with the system control unit 11 of the body of the digital camera 100 via wired or wireless communication.

[0019] The lens control unit 4 drives the focus lens included in the imaging lens 1 via the lens driving unit 8 and drives the diaphragm 2 via the diaphragm driving unit 9 in accordance with commands from the system control unit 11 .

[0020] The main body of the digital camera 100 comprises an image sensor 5 such as a CCD image sensor or a CMOS image sensor that captures an image of a subject through an imaging optical system, an analog signal processing unit 6 connected to the output of the image sensor 5 and performing analog signal processing such as correlated double sampling, an analog-to-digital conversion circuit 7 that converts the analog signal output from the analog signal processing unit 6 into a digital signal, an image sensor driving unit 10, a system control unit 11 that provides overall control of the entire system, and an operation unit 14.

[0021] The analog signal processing unit 6, the analog-to-digital conversion circuit 7, and the image sensor driving unit 10 are controlled by a system control unit 11.

[0022] The system control unit 11 drives the image sensor 5 via the image sensor driving unit 10 to output a subject image captured through an imaging optical system as a captured image signal. An instruction signal from a user is input to the system control unit 11 via an operation unit 14.

[0023] The system control unit 11 includes a processor and memories such as a RAM (Random Access Memory) and a flash memory.

[0024] The system control unit 11 executes programs, including a focus control program, stored in an internal flash memory, to realize various functions, which will be described later.

[0025] Further, the electric control system of the digital camera 100 includes a main memory 16, a memory control unit 15 connected to the main memory 16, a digital signal processing unit 17 that performs signal processing on the captured image signal output from the analog-digital conversion circuit 7 to generate captured image data, a contrast AF processing unit 18 that determines the focus position by a contrast AF method, a phase difference AF processing unit 19 that calculates the defocus amount by a phase difference AF method, an external memory control unit 20 to which a removable recording medium 21 is connected, and a display control unit 22 to which a display unit 23 mounted on the back surface of the camera is connected. The defocus amount is an amount that indicates how far the current focus is off from the subject, and is an example of the focus position. Note that the focus position is not limited to the defocus amount, and may be a subject distance that is the distance between the digital camera 100 and the subject, or a driving parameter of the focus lens of the imaging lens 1 for focusing on the subject.

[0026] The memory control unit 15, the digital signal processing unit 17, the contrast AF processing unit 18, the phase difference AF processing unit 19, the external memory control unit 20, and the display control unit 22 are interconnected by a control bus 24 and a data bus 25, and are controlled by commands from the system control unit 11.

[0027] FIG. 2 is a schematic plan view showing an example of the configuration of the image sensor 5 mounted on the digital camera 100. As shown in FIG.

[0028] The image sensor 5 has a light receiving surface 50 on which a large number of pixels are arranged two-dimensionally in a row direction X and a column direction Y perpendicular to the row direction X.

[0029] In the example of FIG. 2, 63 focus detection areas (hereinafter referred to as AF areas) 53 are provided on the light receiving surface 50, which are areas to be focused on (areas on which a subject image to be focused is formed).

[0030] In the digital camera 100, one or a plurality of consecutively arranged AF areas 53 are selected from the 63 AF areas 53 shown in FIG. 2, and focus control is performed to focus on the subject being imaged using the selected AF area 53.

[0031] The AF area 53 is an area including imaging pixels and phase difference detection pixels as pixels. In the portion of the light receiving surface 50 other than the AF area 53, only the imaging pixels are arranged.

[0032] FIG. 3 is a partial enlarged view of one AF area 53 shown in FIG.

[0033] Pixels 51 (square blocks in the figure) are arranged two-dimensionally in the AF area 53. Each pixel 51 includes a photoelectric conversion unit such as a photodiode and a color filter formed above the photoelectric conversion unit. Note that each pixel 51 may be configured to perform splitting by the structure of a photodiode without using a color filter.

[0034] In FIG. 3, the pixel 51 (R pixel 51) including a color filter (R filter) that transmits red light is marked with the letter "R."

[0035] In FIG. 3, the pixel 51 (G pixel 51) including a color filter (G filter) that transmits green light is marked with the letter "G."

[0036] 3, pixels 51 (B pixels 51) including a color filter (B filter) that transmits blue light are marked with the letter “B.” The color filters are arranged in a Bayer array over the entire light receiving surface 50.

[0037] In the AF area 53, some of the G pixels 51 (pixels shaded in FIG. 3) serve as phase difference detection pixels 52A and 52B. In the example of FIG. 3, each G pixel 51 in an arbitrary pixel row among pixel rows including an R pixel 51 and a G pixel 51 serves as a phase difference detection pixel 52A, and a G pixel 51 of the same color that is closest to each G pixel 51 in the column direction Y serves as a phase difference detection pixel 52B.

[0038] A phase difference detection pixel 52A and a phase difference detection pixel 52B of the same color that is closest to the phase difference detection pixel 52A in the column direction Y form a pair. However, the arrangement of these phase difference detection pixels is only an example, and other arrangements may be used. For example, some of the G pixels 51 may be phase difference detection pixels, or may be arranged among the R pixels 51 and the B pixels 51.

[0039] The phase difference detection pixel 52A in the third pixel row from the top in FIG. 3 and the phase difference detection pixel 52B in the fifth pixel row from the top in FIG. 3 form a pair row PL1 composed of a plurality of pairs arranged in the row direction X.

[0040] The phase difference detection pixel 52A in the seventh pixel row from the top in FIG. 3 and the phase difference detection pixel 52B in the ninth pixel row from the top in FIG. 3 form a pair row PL2 consisting of a plurality of pairs arranged in the row direction X.

[0041] The phase difference detection pixel 52A in the 11th pixel row from the top in FIG. 3 and the phase difference detection pixel 52B in the 13th pixel row from the top in FIG. 3 form a pair row PL3 consisting of a plurality of pairs arranged in the row direction X.

[0042] In this manner, in the AF area 53, a plurality of paired rows are arranged in the column direction Y.

[0043] FIG. 4 is a diagram showing phase difference detection pixels constituting an arbitrary pair of rows shown in FIG.

[0044] The phase difference detection pixel 52A is a first signal detection unit that receives a light beam that has passed through one of the divided regions of the pupil region of the imaging lens 1 that is divided into two in the row direction X, and detects a signal according to the amount of received light.

[0045] The phase difference detection pixel 52B is a second signal detection unit that receives a light beam that has passed through the other divided region of the above-mentioned pupil region and detects a signal according to the amount of received light.

[0046] In the AF area 53, the pixels 51 other than the phase difference detection pixels 52A and 52B are imaging pixels. The imaging pixels receive light beams that have passed through both of the two divided regions of the pupil region of the imaging lens 1 and detect signals according to the amount of received light.

[0047] A light-shielding film is provided above the photoelectric conversion portion of each pixel 51, and an opening that defines the light-receiving area of ​​the photoelectric conversion portion is formed in the light-shielding film.

[0048] The center of the aperture of the imaging pixel coincides with the center of the photoelectric conversion unit of the imaging pixel. In contrast, the center of the aperture of the phase difference detection pixel 52A (the white portion in FIG. 4) is offset to one side (the right side) with respect to the center of the photoelectric conversion unit of the phase difference detection pixel 52A.

[0049] Moreover, the center of the aperture (the white portion in FIG. 4) of the phase difference detection pixel 52B is decentered to the other side (left side) with respect to the center of the photoelectric conversion unit of the phase difference detection pixel 52B.

[0050] Fig. 5 is a diagram showing a cross-sectional configuration of the phase difference detection pixel 52 A. As shown in Fig. 5, in the phase difference detection pixel 52 A, the aperture c is decentered to one side (right) with respect to the photoelectric conversion unit PD.

[0051] As shown in FIG. 5, by covering one side of the photoelectric conversion unit PD with a light-shielding film, it is possible to selectively block light incident from the direction opposite to the direction covered by the light-shielding film.

[0052] With this configuration, a pixel group made up of the phase difference detection pixels 52A constituting an arbitrary pair row and a pixel group made up of the phase difference detection pixels 52B constituting this pair row can detect a phase difference in the row direction X in an image captured by each of these two pixel groups.

[0053] It should be noted that the pixel configuration of the imaging element 5 is not limited to the configurations shown in FIGS.

[0054] For example, all pixels included in the imaging element 5 may be used as imaging pixels, and each imaging pixel may be divided into two in the row direction X, with one divided portion being the phase difference detection pixel 52A and the other divided portion being the phase difference detection pixel 52B.

[0055] FIG. 6 is a diagram showing a configuration in which all pixels 51 included in the imaging element 5 are used as imaging pixels, and each pixel 51 is divided into two.

[0056] In the configuration of FIG. 6, a pixel 51 marked with R in the image sensor 5 is divided into two, and the two divided pixels are designated as a phase difference detection pixel r1 and a phase difference detection pixel r2, respectively.

[0057] In addition, in the imaging element 5, a pixel 51 marked with G is divided into two, and the two divided pixels are designated as a phase difference detection pixel g1 and a phase difference detection pixel g2.

[0058] Furthermore, in the imaging element 5, the pixel 51 marked with B is divided into two, and the two divided pixels are designated as a phase difference detection pixel b1 and a phase difference detection pixel b2, respectively.

[0059] In this configuration, the phase difference detection pixels r1, g1, and b1 serve as first signal detection units, and the phase difference detection pixels r2, g2, and b2 serve as second signal detection units.

[0060] In the configuration example of Fig. 6, when the signals of the first signal detection unit and the second signal detection unit included in one pixel 51 are added, a normal imaging signal without a phase difference is obtained. That is, in the configuration of Fig. 6, all pixels can be used as both phase difference detection pixels and imaging pixels. In addition, in the configuration example of Fig. 6, the degree of freedom in setting the size and shape of the AF area can be improved.

[0061] The system control unit 11 selectively performs focus control using either a phase difference AF method or a contrast method.

[0062] In response to an instruction from the system control unit 11, the phase difference AF processing unit 19 calculates a phase difference, which is the amount of relative positional shift between the two images formed by the pair of light beams, by using a group of detection signals read out from the phase difference detection pixels 52A and the phase difference detection pixels 52B in one or more AF areas 53 selected by a user operation or the like from among the 63 AF areas 53.

[0063] Then, based on this phase difference, the phase difference AF processor 19 obtains the focus adjustment state of the imaging lens 1, here defocus data which is the defocus amount and defocus direction.

[0064] The system control unit 11 drives the focus lens based on this defocus data, thereby performing focus control by a phase difference AF method that utilizes the result of the correlation calculation.

[0065] The contrast AF processing unit 18 analyzes the image captured by the imaging element 5, and determines the focus position of the imaging lens 1 by the contrast AF method.

[0066] That is, the contrast AF processing unit 18 calculates the contrast (brightness difference) of the captured image obtained for each of the moved positions (plural positions) while moving the focus lens position of the imaging lens 1 under the control of the system control unit 11. Then, the focus lens position at which the contrast is maximized is determined as the maximum contrast focus position.

[0067] The system control unit 11 drives the focus lens based on the maximum contrast focus position determined by the contrast AF processing unit 18, thereby performing focus control by a contrast AF method that utilizes the contrast of the captured image.

[0068] Digital camera 100 may also be equipped with a continuous AF mode in which focus control for adjusting the focus on a subject is performed multiple times in succession. For example, digital camera 100 performs focus control for adjusting the focus on a subject multiple times in succession while displaying a through image (live view image) that shows captured images obtained by continuous imaging by image sensor 5 in real time.

[0069] Fig. 7 is a diagram showing an example of the timing of each operation of the digital camera 100. In Fig. 7, the horizontal axis indicates time.

[0070] The imaging operation 71 indicates a periodic operation related to imaging by the imaging element 5. Specifically, the imaging operation 71 is an operation in which an exposure operation 71a, a display operation 71b, and a recording operation 71c are periodically repeated.

[0071] Exposure operation 71a (dotted portion) is an exposure operation by the imaging element 5. Display operation 71b (diagonal hatched portion) is an operation in which a process is performed to display a through image based on pixel data obtained by the imaging element 5 on the display unit 23. Recording operation 71c (no hatching) is an operation in which a process is performed to record image data based on pixel data obtained by the imaging element 5 on the recording medium 21.

[0072] Image generation 72 is an operation for generating an image for subject detection (an image for subject detection) based on pixel data obtained by exposure operation 71a. Subject detection 73 is an operation for detecting a subject area in which a subject exists from the image for subject detection generated by image generation 72. The subject may be a subject specified by the user, or may be a subject automatically detected by digital camera 100.

[0073] AF detection 74 is an operation in which the phase difference AF processing unit 19 calculates a phase difference in a portion of the pixel data obtained by the exposure operation 71a that corresponds to the subject area detected by subject detection 73, and calculates a defocus amount based on the calculated phase difference. At this time, the phase difference AF processing unit 19 calculates a phase difference based on the pixel data used in the immediately preceding subject detection 73, rather than the latest pixel data at that time, as shown in FIG.

[0074] For example, subject detection 73b is subject detection 73 based on pixel data obtained at time t2. AF detection 74b is AF detection 74 based on subject detection 73b. The latest pixel data at the time AF detection 74 is performed is pixel data obtained at time t3. At this time, phase difference AF processing unit 19 calculates the phase difference of a portion corresponding to the subject area detected by subject detection 73b in the pixel data at time t2 used for subject detection 73b in AF detection 74b, rather than the pixel data obtained at time t3.

[0075] AF prediction 75 is an operation in which phase difference AF processor 19 calculates a predicted value of the defocus amount at the next focus timing based on the result of AF detection 74. Note that in Fig. 7, the result of the immediately previous AF detection 74 is referenced in AF prediction 75, but in order to actually calculate the predicted value of the defocus amount, the results of multiple past AF detections 74 are referenced.

[0076] Focus 76 indicates the operation timing for performing focus control by driving the focus lens of the imaging lens 1 based on the predicted value of the defocus amount calculated by the AF prediction 75.

[0077] 7, the operation of performing subject detection 73 for each cycle of imaging operation 71 has been described, but the operation is not limited to this. For example, digital camera 100 may or may not perform subject detection 73 for each cycle of imaging operation 71 depending on the processing load, etc.

[0078] Fig. 8 is a diagram showing an example of the focus operation of the digital camera 100 corresponding to Fig. 7. In Fig. 8, the horizontal axis indicates time.

[0079] Pixel data D2 to D5 are each pixel data obtained consecutively by the image sensor 5. In this example, the subject (person's face) moves from the right end toward the center. For example, pixel data D2 is pixel data obtained at time t2 shown in FIG. 7, and pixel data D5 is pixel data obtained at time t3 shown in FIG. 7.

[0080] The digital camera 100 generates a subject detection image d2 based on, for example, pixel data D2, and performs subject detection to detect a subject area in which a subject exists as a second area e2 from the subject detection image d2. The digital camera 100 then performs AF detection based on the detected second area e2.

[0081] At this point, pixel data that can be the target of AF detection include, for example, pixel data D2 to D5, and the latest pixel data is pixel data D5. In this example, however, digital camera 100 performs AF detection based on pixel data D2 used for subject detection (data from which subject detection image d2 was generated) rather than pixel data D5. That is, digital camera 100 calculates the defocus amount by calculating the phase difference of a portion of pixel data D2 that corresponds to second area e2. This defocus amount is an example of a first focus position.

[0082] Fig. 9 is a diagram showing another example of the timing of each operation of digital camera 100. In Fig. 9, parts similar to those shown in Fig. 7 are given the same reference numerals and description thereof will be omitted. Subject detection 73a shown in Fig. 9 is subject detection 73 based on pixel data obtained at time t1, which is before time t2. When the deviation between the subject area detected by subject detection 73a and the subject area detected by subject detection 73b is small, digital camera 100 uses the latest pixel data in AF detection 74b based on subject detection 73b, rather than the pixel data used in subject detection 73b.

[0083] Fig. 10 is a diagram showing an example of the focus operation of digital camera 100 corresponding to Fig. 9. In Fig. 10, parts similar to those shown in Fig. 8 are given the same reference numerals and description thereof will be omitted. Subject detection image d1 shown in Fig. 10 is a subject detection image generated from pixel data (not shown in Fig. 10) obtained at time t1 shown in Fig. 9. A first area e1 is an area detected from subject detection image d1 as a subject area in which a subject exists.

[0084] In AF detection based on the second area e2, the digital camera 100 compares the first area e1 with the second area e2, and if the deviation between the two is small, performs AF detection based on the latest pixel data D5, not the pixel data D2 (data from which the subject detection image d2 is generated). That is, the digital camera 100 calculates the defocus amount by calculating the phase difference of a portion of the pixel data D5 that corresponds to the second area e2. This defocus amount is an example of the second focus position.

[0085] In this way, the digital camera 100 detects the subject area as the second area e2 from the pixel data D2 obtained at time t2 (second time). The digital camera 100 selects either a first focus position (defocus amount by AF detection in FIG. 8) based on the data of the second area e2 in the pixel data D2 obtained at time t2, or a second focus position (defocus amount by AF detection in FIG. 10) based on the data of the second area e2 in the pixel data D5 obtained at time t3 (third time) after time t2. The digital camera 100 then drives the lens by outputting a control signal generated using the selected focus position to a lens drive mechanism of the imaging lens 1 (imaging optical system). The lens drive mechanism is a drive mechanism for a focus lens included in the imaging lens 1. The focus lens may be a liquid lens.

[0086] Specifically, digital camera 100 selects either the first focus position or the second focus position based on the comparison result between a first area e1 detected as a subject area from pixel data obtained at time t1 (first time) before time t2, and a second area e2 detected as a subject area from pixel data obtained at time t2.

[0087] Furthermore, the pixel data D5 obtained at time t3 is pixel data obtained by the image sensor 5 immediately before the second area e2 is detected. This allows AF detection to be performed using the most recent pixel data.

[0088] According to digital camera 100, when the movement of the subject in the vertical and horizontal directions (two-dimensional directions of the captured image) is large, AF is performed based on the pixel data used for subject detection, and even if the subject moves vertically or horizontally while subject detection is being performed, it is possible to avoid performing AF in a state where AF area 53 is misaligned with the subject. Therefore, it is possible to suppress a decrease in focus accuracy accompanying the movement of the subject in the vertical and horizontal directions.

[0089] Furthermore, when the subject movement in the vertical and horizontal directions is small, AF is performed based on newer pixel data (e.g., the latest pixel data) than the pixel data used for subject detection, and even if the subject moves in the depth direction (direction perpendicular to the captured image) while subject detection is being performed, it is possible to focus on the subject after the movement. Therefore, it is possible to suppress the decrease in focus accuracy accompanying the subject's movement in the depth direction.

[0090] Therefore, the focus accuracy can be improved according to the digital camera 100. Here, time t1, time t2, and time t3 are times included in the time of periodic imaging by the image sensor 5. In this way, by performing AF using pixel data obtained by periodic imaging by the image sensor 5, it is possible to improve the focus accuracy while continuously performing focus control to focus on the subject during video shooting or display of a through image.

[0091] Fig. 11 is a flowchart showing an example of a process for driving a lens by the digital camera 100. The digital camera 100 repeatedly executes, for example, the process shown in Fig. 11 as the process for driving a lens. This process is executed by at least one of the system control unit 11 and the phase difference AF processing unit 19, for example.

[0092] First, the digital camera 100 stores pixel data obtained by the image sensor 5 (step S111). The pixel data is stored, for example, in a memory included in the system control unit 11 or in the main memory 16. Next, the digital camera 100 generates an image for subject detection based on the stored pixel data (step S112).

[0093] Next, digital camera 100 determines whether or not subject detection has been completed (subject detection has been completed) based on the subject detection image generated in step S112 (step S113). If subject detection has not been completed (step S113: No), digital camera 100 returns to step S111.

[0094] In step S113, if subject detection is complete (step S113: Yes), digital camera 100 detects the subject area based on the subject detection image generated in step S112 (step S114).

[0095] Next, digital camera 100 performs subject movement determination (step S115) to determine the subject movement, which is the deviation between the subject area detected in the immediately previous step S114 (second area e2 in the example of FIG. 10) and the subject area detected in the previous step S114 (first area e1 in the example of FIG. 10). The subject movement determination in step S115 will be described later with reference to FIG. 12.

[0096] Next, digital camera 100 determines whether the subject movement is large based on the subject movement determination in step S115 (step S116). If the subject movement is large (step S116: Yes), digital camera 100 calculates a defocus amount from phase difference information of the pixel data used to detect the subject area in step S114 and the subject area detected in step S114 (step S117). The pixel data used to detect the subject area is the pixel data used to generate the subject detection image used to detect the subject area.

[0097] In step S116, if the subject movement is not large (step S116: No), digital camera 100 calculates the defocus amount from the phase difference information of the latest pixel data at that time and the subject area detected in step S114 (step S118).

[0098] Next, the digital camera 100 drives the lens based on the defocus amount calculated in either step S117 or S118 (step S119), and ends the series of processes. This lens driving is performed, for example, by the system control unit 11 outputting a control signal based on the defocus amount to the imaging lens 1.

[0099] Fig. 12 is a flow chart showing an example of subject movement determination in step S115 in Fig. 11. In step S115 in Fig. 11, digital camera 100 executes, for example, the process shown in Fig. 12 as subject movement determination.

[0100] First, digital camera 100 judges whether the difference between the previous subject position and the current subject position is equal to or less than a predetermined threshold (step S121). The current subject position is the position of the subject area (first area e1 in the example of FIG. 10) detected by the most recently executed step S114. The previous subject position is the position of the subject area (second area e2 in the example of FIG. 10) detected by the step S114 executed immediately before the most recently executed step S114. The position of the subject area is, for example, the center of the subject area, but may be an edge of the subject area, etc. If the difference between the subject positions is equal to or less than the threshold (step S121: Yes), digital camera 100 judges that the subject movement is small (step S122) and ends the series of subject movement judgments.

[0101] If the difference in subject position is not equal to or less than the threshold in step S121 (step S121: No), digital camera 100 determines whether the difference between the previous subject size and the current subject size is equal to or less than a predetermined threshold (step S123). The current subject size is the size of the subject area detected in step S114 executed immediately before. The previous subject size is the size of the subject area detected in step S114 executed immediately before the previously executed step S114. The size of the subject area is, for example, the number of pixels in the subject area. If the difference in subject size is equal to or less than the threshold (step S123: Yes), digital camera 100 proceeds to step S122, determines that the subject movement is small, and ends the series of subject movement determinations.

[0102] If the difference in subject size is not equal to or less than the threshold in step S123 (step S123: No), digital camera 100 determines whether the difference between the detection time of the previous subject area and the detection time of the current subject area is equal to or less than a predetermined threshold (step S124). If the difference in the detection time of the subject area is equal to or less than the threshold (step S124: Yes), even if the subject has moved, it is only for a short period of time, so it can be assumed that the shift of the subject is small. In this case, digital camera 100 proceeds to step S122 and determines that the subject movement is small, and ends the series of subject movement determination processes.

[0103] In step S124, if the difference in the detection times of the object areas is not equal to or less than the threshold (step S124: No), it is determined whether the current object size is equal to or greater than a predetermined threshold (step S125). If the current object size is equal to or greater than the threshold (step S125: Yes), it can be assumed that even if the object has moved, the positional deviation between the detected object area and the object is small. In this case, digital camera 100 proceeds to step S122, determines that the object movement is small, and ends the series of object movement determination processes.

[0104] If the current subject size is not equal to or larger than the threshold in step S125 (step S125: No), digital camera 100 determines that the subject movement is large (step S126) and ends the series of subject movement determination processes.

[0105] In this way, the digital camera 100 selects pixel data to be used for AF detection based on the second area e2 based on the result of comparing the previously detected position of the first area e1 with the currently detected position of the second area e2. This makes it possible to select either the pixel data used for detection of the second area e2 or newer pixel data according to the amount of movement of the subject in the image and perform AF detection.

[0106] Furthermore, the digital camera 100 selects pixel data to be used for AF detection based on the second area e2 based on a comparison result between the previously detected size of the first area e1 and the currently detected size of the second area e2. This makes it possible to select either the pixel data used for detection of the second area e2 or newer pixel data according to the amount of change in the proportion of the area occupied by the subject in the image, and perform AF detection.

[0107] Furthermore, digital camera 100 selects pixel data to be used for AF detection based on second area e2 based on a comparison result between the past detection time of first area e1 and the current detection time of second area e2. As a result, if the time elapsed since the past subject detection is short, it is possible to determine that the subject movement is small and select new pixel data to perform AF detection.

[0108] Furthermore, the digital camera 100 selects pixel data to be used for AF detection based on the second area e2 based on the size of the second area e2 detected this time. As a result, when the second area e2 is large, it is possible to determine that the subject movement is small and select new pixel data to perform AF detection.

[0109] 12 is an example, and some of the processes may be omitted. For example, at least one of the decisions in steps S121 and S123 to S125 may be omitted, and the subject movement may be determined by the remaining decisions.

[0110] Fig. 13 is a diagram showing yet another example of the focus operation of the digital camera 100. In Fig. 13, the same parts as those shown in Fig. 10 are given the same reference numerals and the description thereof will be omitted.

[0111] As shown in FIG. 13, the digital camera 100 may perform AF detection using pixel data D3 obtained while performing subject detection from subject detection image d2 based on pixel data D2 obtained at time t2, using a first area e1 detected by the previous subject detection, i.e., subject detection from subject detection image d1.

[0112] Fig. 14 is a diagram showing an example of an overlapping portion between the second area e2 and the first area e1. The overlapping area e3 shown in Fig. 14 is an overlapping portion between the first area e1 detected from the subject detection image d1 and the second area e2 detected from the subject detection image d2. The digital camera 100 calculates a phase difference in a portion corresponding to the overlapping area e3 in the pixel data D3 obtained while performing subject detection from the subject detection image d2, and uses the calculated phase difference for AF prediction during a period when subject detection is not performed.

[0113] For example, the digital camera 100 calculates the phase difference for each position obtained by dividing the portion corresponding to the first area e1 in the pixel data D3 into a plurality of portions. Then, from the phase differences calculated for each position, the phase difference for the portion not included in the second area e2 detected from the subject detection image d2 is excluded, and the remaining phase difference is used for AF prediction.

[0114] Alternatively, the digital camera 100 holds the pixel data D3 until the second area e2 is detected, and when the second area e2 is detected, calculates an overlap area e3 which is the overlapping portion between the first area e1 and the second area e2, calculates the phase difference of the portion of the pixel data D3 which corresponds to the overlap area e3, and uses the calculated phase difference for AF prediction.

[0115] 13 and 14, the phase difference of the portion corresponding to the overlap area e3 in the pixel data D3 obtained between the detection of the first area e1 and the detection of the second area e2 is calculated and used for AF prediction, but the phase difference of the portion corresponding to the overlap area e3 may be calculated for a plurality of pixel data obtained between the detection of the first area e1 and the detection of the second area e2 and used for AF prediction. The phase difference of the overlap area e3 described in FIG. 13 and FIG. 14 is an example of the third focus position.

[0116] Fig. 15 is a diagram showing an example of AF prediction based on phase difference information of overlap area e3. In Fig. 15, the horizontal axis represents time, and the vertical axis represents subject distance, and indicates the result of AF detection or the result of AF prediction.

[0117] Subject distance change 150 indicates the actual change in subject distance over time. In the example shown in Fig. 15, the subject is initially moving away from digital camera 100, but the speed at which the subject is moving away from digital camera 100 gradually decreases, and finally the distance from digital camera 100 becomes constant.

[0118] Time t11 is the timing for performing AF detection based on the first area e1 detected from the subject detection image d1 based on the pixel data obtained at time t1. The AF detection result 151 is the result of the AF detection at time t11.

[0119] Time t14 is the timing for performing AF detection based on the second area e2 detected from the subject detection image d2 based on the pixel data obtained at time t2. The AF detection result 154 is the result of the AF detection at time t14.

[0120] Time t15 is the imaging timing immediately after time t14. In order to focus on the subject at time t15, digital camera 100 performs AF prediction to predict the subject distance at time t15. If the subject distance at time t15 is predicted (linear prediction) only based on AF detection results 151 and 154, it will be like AF prediction result 155a, which will be significantly different from the actual subject distance at time t15.

[0121] Times t12 and t13 are between times t11 and t14, that is, the times from when the first area e1 is detected to when the second area e2 is detected, and are the times when pixel data was obtained. AF detection results 152 and 153 are the results of AF detection performed on the pixel data obtained at times t12 and t13 based on the portions corresponding to the overlap area e3.

[0122] Digital camera 100 can obtain AF prediction result 155b that is close to the actual subject distance at time t15 by performing AF prediction using AF detection results 152 and 153 in addition to AF detection results 151 and 154. Then, at time t15, driving the lens based on AF prediction result 155b allows accurate focusing on the subject.

[0123] In other words, if AF detection is performed only using AF detection results 151, 154 at times t11, t14, when the subject detection results are updated, the movement of the subject cannot be correctly grasped if the AF detection frame rate is rough. However, as described above, AF detection results 152, 153 can be obtained at positions where the subject is likely to have been located for times t12, t13, which are between times t11, t14, so that the latest movement of the subject can be captured and focus accuracy is improved.

[0124] In this way, when performing AF using the second focus position based on the data of the second area e2 in pixel data D5 obtained at time t3 (third time) (when the subject movement is small), digital camera 100 performs AF using the third focus position based on the data of the overlapping portion between the second area e2 and the first area e1 in pixel data D3 obtained between the time when the first area e1 detected as the subject area from the pixel data obtained at time t1 (first time) was detected and the time when the second area e2 was detected. For example, digital camera 100 performs AF prediction using this third focus position for AF detection at a timing when subject detection is not performed.

[0125] FIG. 16 shows the appearance of a smartphone 200 which is another embodiment of the imaging device of the present invention.

[0126] A smartphone 200 shown in FIG. 16 has a flat housing 201, and is provided on one surface of the housing 201 with a display panel 202 as a display unit and a display input unit 204 which is an integrated unit of an operation panel 203 as an input unit.

[0127] Moreover, such a housing 201 includes a speaker 205, a microphone 206, an operation unit 207, and a camera unit 208. Note that the configuration of the housing 201 is not limited to this, and for example, a configuration in which the display unit and the input unit are independent, or a configuration having a folding structure or a sliding mechanism may be adopted.

[0128] FIG. 17 is a block diagram showing a configuration of the smartphone 200 shown in FIG.

[0129] As shown in FIG. 17, the main components of the smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a memory unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) receiving unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0130] The smartphone 200 also has, as its main function, a wireless communication function for performing mobile wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0131] The wireless communication unit 210 performs wireless communication with the base station device BS accommodated in the mobile communication network NW in accordance with instructions from the main control unit 220. Using this wireless communication, various file data such as voice data and image data, e-mail data, etc. are sent and received, and web data, streaming data, etc. are received.

[0132] The display input unit 204 is a so-called touch panel that displays images (still images and moving images) or text information, etc. under the control of the main control unit 220 to visually convey information to the user and detects user operations on the displayed information, and is equipped with a display panel 202 and an operation panel 203.

[0133] The display panel 202 uses a liquid crystal display (LCD), an organic electro-luminescence display (OLED), or the like as a display device.

[0134] The operation panel 203 is a device placed so that an image displayed on the display surface of the display panel 202 can be seen, and detects one or more coordinates operated by a user's finger or a stylus. When this device is operated by the user's finger or a stylus, a detection signal generated by the operation is output to the main control unit 220. Next, the main control unit 220 detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0135] As shown in FIG. 17, a display panel 202 and an operation panel 203 of a smartphone 200, which is exemplified as one embodiment of the imaging device of the present invention, are integrated together to form a display input unit 204, and the operation panel 203 is positioned so as to completely cover the display panel 202.

[0136] When such an arrangement is adopted, operation panel 203 may also have a function of detecting user operations in an area outside display panel 202. In other words, operation panel 203 may have a detection area for an overlapping portion that overlaps display panel 202 (hereinafter referred to as a display area), and a detection area for the other outer edge portion that does not overlap display panel 202 (hereinafter referred to as a non-display area).

[0137] The size of the display area and the size of the display panel 202 may be completely the same, but they do not necessarily have to be the same. Also, the operation panel 203 may have two sensitive areas, an outer edge part and the other inner part. Furthermore, the width of the outer edge part is designed appropriately according to the size of the housing 201, etc.

[0138] Furthermore, the position detection method used in the operation panel 203 may be a matrix switch method, a resistive film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, a capacitance method, or the like, and any method may be used.

[0139] The call unit 211 is equipped with a speaker 205 or a microphone 206, and converts the user's voice input through the microphone 206 into voice data that can be processed by the main control unit 220 and outputs it to the main control unit 220, or decodes voice data received by the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0140] Also, as shown in FIG. 16, for example, speaker 205 can be mounted on the same surface as display input unit 204, and microphone 206 can be mounted on the side surface of housing 201.

[0141] The operation unit 207 is a hardware key using a key switch or the like, and receives instructions from a user. For example, as shown in Fig. 16, the operation unit 207 is a push-button switch mounted on the side of the housing 201 of the smartphone 200, which is turned on when pressed with a finger or the like, and turned off by the restoring force of a spring or the like when the finger is released.

[0142] The storage unit 212 stores the control program and control data of the main control unit 220, application software, address data associated with names or telephone numbers of communication partners, data of sent and received e-mails, Web data downloaded by Web browsing, downloaded content data, and also temporarily stores streaming data, etc. The storage unit 212 is composed of an internal storage unit 217 built into the smartphone and an external storage unit 218 having a removable external memory slot.

[0143] Each of the internal memory unit 217 and the external memory unit 218 constituting the memory unit 212 is realized using a storage medium such as a flash memory type, a hard disk type, a multimedia card micro type, a card-type memory (e.g., MicroSD (registered trademark) memory, etc.), a RAM (Random Access Memory), a ROM (Read Only Memory), etc.

[0144] The external input / output unit 213 serves as an interface with all external devices connected to the smartphone 200, and is for directly or indirectly connecting to other external devices via communication (e.g., Universal Serial Bus (USB), IEEE1394, Bluetooth (registered trademark), RFID (Radio Frequency Identification), Infrared Data Association (IrDA) (registered trademark), UWB (Ultra Wideband) (registered trademark), ZigBee (registered trademark), etc.) or a network (e.g., Ethernet (registered trademark), wireless LAN (Local Area Network), etc.).

[0145] Examples of external devices that may be connected to the smartphone 200 include wired / wireless headsets, wired / wireless external chargers, wired / wireless data ports, memory cards connected via card sockets, SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards, external audio / video devices connected via audio / video I / O (Input / Output) terminals, wirelessly connected external audio / video devices, wired / wirelessly connected smartphones, wired / wirelessly connected personal computers, wired / wirelessly connected personal computers, earphones, etc.

[0146] The external input / output unit 213 can transmit data received from such an external device to each component inside the smartphone 200, or transmit data inside the smartphone 200 to an external device.

[0147] The GNSS receiver 214 receives GNSS signals transmitted from the GNSS satellites ST1 to STn according to instructions from the main controller 220, executes positioning calculation processing based on the multiple received GNSS signals, and detects a position consisting of the latitude, longitude, and altitude of the smartphone 200. When the GNSS receiver 214 can obtain position information from the wireless communication unit 210 or the external input / output unit 213 (for example, wireless LAN), it can also detect the position using the position information.

[0148] The motion sensor unit 215 includes, for example, a three-axis acceleration sensor and detects the physical movement of the smartphone 200 according to an instruction from the main control unit 220. By detecting the physical movement of the smartphone 200, the moving direction or acceleration of the smartphone 200 is detected. The detection result is output to the main control unit 220.

[0149] The power supply unit 216 supplies power stored in a battery (not shown) to each unit of the smartphone 200 in accordance with instructions from the main control unit 220.

[0150] The main control unit 220 includes a microprocessor, operates according to the control program and control data stored in the storage unit 212, and generally controls each unit of the smartphone 200. The main control unit 220 also includes a mobile communication control function that controls each unit of the communication system to perform voice communication or data communication via the wireless communication unit 210, and an application processing function.

[0151] The application processing function is realized by the main control unit 220 operating in accordance with the application software stored in the storage unit 212. Examples of the application processing function include an infrared communication function for controlling the external input / output unit 213 to perform data communication with a counterpart device, an email function for sending and receiving emails, and a web browsing function for browsing web pages.

[0152] The main control unit 220 also has an image processing function for displaying video on the display input unit 204 based on image data (still image or moving image data) such as received data or downloaded streaming data.

[0153] The image processing function refers to a function in which the main control unit 220 decodes the image data, performs image processing on the decoded result, and displays the image on the display input unit 204.

[0154] Furthermore, the main control unit 220 executes display control for the display panel 202 and operation detection control for detecting user operations via the operation unit 207 and the operation panel 203 .

[0155] By executing display control, main control unit 220 displays icons for starting application software or software keys such as scroll bars, or displays a window for creating an e-mail.

[0156] The scroll bar refers to a software key for receiving an instruction to move the displayed portion of an image, such as a large image that cannot fit within the display area of ​​the display panel 202.

[0157] In addition, by executing operation detection control, the main control unit 220 detects user operations through the operation unit 207, accepts operations on the above-mentioned icons and input of character strings into the input field of the above-mentioned window through the operation panel 203, or accepts a request to scroll the displayed image through a scroll bar.

[0158] Furthermore, by executing operation detection control, the main control unit 220 determines whether the operation position on the operation panel 203 is an overlapping portion (display area) that overlaps with the display panel 202, or an outer edge portion (non-display area) that does not overlap with the display panel 202, and is equipped with a touch panel control function that controls the sensitive area of ​​the operation panel 203 or the display position of the software key.

[0159] Furthermore, the main control unit 220 can also detect a gesture operation on the operation panel 203 and execute a preset function in response to the detected gesture operation.

[0160] The gesture operation is not a simple touch operation as in the past, but rather refers to an operation in which a path is drawn with a finger or the like, multiple positions are designated simultaneously, or a combination of these to draw a path for at least one of multiple positions.

[0161] 1, except for the external memory control unit 20, the recording medium 21, the display control unit 22, the display unit 23, and the operation unit 14. The captured image data generated by the camera unit 208 can be recorded in the storage unit 212, or output via the external input / output unit 213 or the wireless communication unit 210.

[0162] In the smartphone 200 shown in FIG. 16, the camera unit 208 is mounted on the same surface as the display input unit 204, but the mounting position of the camera unit 208 is not limited to this, and the camera unit 208 may be mounted on the back surface of the display input unit 204.

[0163] In addition, the camera unit 208 can be used for various functions of the smartphone 200. For example, an image acquired by the camera unit 208 can be displayed on the display panel 202, or an image from the camera unit 208 can be used as one of the operation inputs for the operation panel 203.

[0164] Furthermore, when the GNSS receiver 214 detects a position, it can also detect the position by referring to an image from the camera unit 208. Furthermore, it can also determine the optical axis direction of the camera unit 208 of the smartphone 200 or the current usage environment by referring to an image from the camera unit 208 without using a three-axis acceleration sensor or by using the same in combination with a three-axis acceleration sensor. Of course, the image from the camera unit 208 can also be used in application software.

[0165] In addition, image data of still or video images can be supplemented with position information acquired by the GNSS receiving unit 214, voice information acquired by the microphone 206 (which may be converted to text information by the main control unit, etc.), posture information acquired by the motion sensor unit 215, etc., and stored in the memory unit 212 or output via the external input / output unit 213 or wireless communication unit 210.

[0166] In the smartphone 200 having the above configuration, the system control unit 11 shown in FIG. 1 performs the above-mentioned process, thereby improving the focus accuracy.

[0167] (Variation 1) In the above embodiment, the case where the phase difference is detected in the row direction X is taken as an example, but the present invention can also be applied to the case where the phase difference is detected in the column direction Y in the same manner.

[0168] (Variation 2) In the above embodiment, the case has been described where AF detection 74 is performed by the phase difference AF processing unit 19 using the phase difference AF method, but AF detection 74 may be performed by the contrast method using the contrast AF processing unit 18. Also, AF detection 74 may be performed by combining the phase difference AF method and the contrast method.

[0169] As described above, the present specification discloses the following:

[0170] (1) an imaging element that captures an image of a subject through an imaging optical system; A processor and an imaging device comprising: The processor, detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated by selectively using a first focus position based on data of the second area in pixel data obtained at the second time and a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; Imaging device.

[0171] (2) The imaging device according to (1), The processor, outputting a control signal to the lens drive mechanism, the control signal being generated by selectively using the first focus position and the second focus position based on a comparison result between a first area detected as a subject area from pixel data obtained by the image sensor at a first time prior to the second time and the second area; Imaging device.

[0172] (3) The imaging device according to (2), The imaging element periodically captures images, the first time, the second time, and the third time are each included in a time of periodic imaging by the image sensor; Imaging device.

[0173] (4) The imaging device according to (2) or (3), The processor, outputting a control signal to the lens driving mechanism, the control signal being generated by selectively using the first focus position and the second focus position based on a comparison result between the position of the first area and the position of the second area; Imaging device.

[0174] (5) The imaging device according to any one of (2) to (4), The processor, outputting a control signal to the lens driving mechanism, the control signal being generated by selectively using the first focus position and the second focus position based on a comparison result between a size of the first area and a size of the second area; Imaging device.

[0175] (6) The imaging device according to any one of (2) to (5), The processor, outputting a control signal to the lens driving mechanism, the control signal being generated by selectively using the first focus position and the second focus position based on a comparison result between the detection time of the first area and the detection time of the second area; Imaging device.

[0176] (7) An imaging device according to any one of (1) to (6), The processor, outputting a control signal generated by selectively using the first focus position and the second focus position based on a size of the second area to the lens driving mechanism; Imaging device.

[0177] (8) An imaging device according to any one of (1) to (7), The processor, When outputting the control signal generated using the second focus position to the lens driving mechanism, a control signal to be output to the lens driving mechanism after detecting the second area is generated using a third focus position based on data of an overlapping portion of the second area and the first area in pixel data obtained by the image sensor between a time when the first area detected as a subject area is detected from pixel data obtained by the image sensor at a first time prior to the second time and a time when the second area is detected. Imaging device.

[0178] (9) An imaging device according to any one of (1) to (8), the pixel data obtained at the third time is pixel data obtained by the imaging element immediately before the time when the second area is detected; Imaging device.

[0179] (10) A focus control method for an imaging apparatus including an imaging element that captures an image of a subject through an imaging optical system and a processor, comprising: The processor, detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated by selectively using a first focus position based on data of the second area in pixel data obtained at the second time and a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; Focus control method.

[0180] (11) A focus control method according to (10), The processor, outputting a control signal to the lens drive mechanism, the control signal being generated by selectively using the first focus position and the second focus position based on a comparison result between a first area detected as a subject area from pixel data obtained by the image sensor at a first time prior to the second time and the second area; Focus control method.

[0181] (12) A focus control method according to (11), The imaging element periodically captures images, the first time, the second time, and the third time are each included in a time of periodic imaging by the image sensor; Focus control method.

[0182] (13) The focus control method according to (11) or (12), The processor, outputting a control signal to the lens driving mechanism, the control signal being generated by selectively using the first focus position and the second focus position based on a comparison result between the position of the first area and the position of the second area; Focus control method.

[0183] (14) A focus control method according to any one of (11) to (13), The processor, outputting a control signal to the lens driving mechanism, the control signal being generated by selectively using the first focus position and the second focus position based on a comparison result between a size of the first area and a size of the second area; Focus control method.

[0184] (15) A focus control method according to any one of (11) to (14), The processor, outputting a control signal to the lens driving mechanism, the control signal being generated by selectively using the first focus position and the second focus position based on a comparison result between the detection time of the first area and the detection time of the second area; Focus control method.

[0185] (16) A focus control method according to any one of (10) to (15), The processor, outputting a control signal generated by selectively using the first focus position and the second focus position based on a size of the second area to the lens driving mechanism; Focus control method.

[0186] (17) A focus control method according to any one of (10) to (16), The processor, When outputting the control signal generated using the second focus position to the lens driving mechanism, a control signal to be output to the lens driving mechanism after detecting the second area is generated using a third focus position based on data of an overlapping portion of the second area and the first area in pixel data obtained by the image sensor between a time when the first area detected as a subject area is detected from pixel data obtained by the image sensor at a first time prior to the second time and a time when the second area is detected. Focus control method.

[0187] (18) A focus control method according to any one of (10) to (17), the pixel data obtained at the third time is pixel data obtained by the imaging element immediately before the time when the second area is detected; Focus control method.

[0188] (19) A focus control program for an imaging device including an imaging element that captures an image of a subject through an imaging optical system and a processor, The processor includes: detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated by selectively using a first focus position based on data of the second area in pixel data obtained at the second time and a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; A focus control program for executing processing.

[0189] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or revised examples within the scope of the claims, and it is understood that these also naturally belong to the technical scope of the present invention. Furthermore, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention.

[0190] This application is based on a Japanese patent application (Patent Application No. 2020-219588) filed on December 28, 2020, the contents of which are incorporated by reference into this application. [Industrial Applicability]

[0191] The present invention is highly convenient and effective when applied to digital cameras and the like. [Explanation of symbols]

[0192] 1 Imaging lens 4 Lens control section 5. Image sensor 6 Analog signal processing section 7 Analog-to-digital conversion circuit 8 Lens drive unit 9 Drive unit 10. Image sensor driver 11 System control section 14,207 Operation section 15 Memory control section 16 Main memory 17 Digital signal processing section 18 Contrast AF processing section 19 Phase-difference AF processing section 20 External memory control unit 21 Recording media 22 Display control unit 23 Display section 25 Data Bus 40 Lens device 50 Photosensitive area 51 pixels 53 AF Area 71 Imaging operation 71a Exposure operation 71b Display operation 71c Recording operation 72 Image Generation 73, 73a, 73b Object detection 74,74b AF detection 75 AF Prediction 76 Focus 100 Digital Camera 150 Change in subject distance 151~154 AF detection result 155a, 155b AF prediction results 200 Smartphones 201 Case 202 Display Panel 203 Operation Panel 204 Display and input section 205 Speaker 206 Microphone 208 Camera Club 210 Wireless Communication Department 211 Telephone section 212 Storage section 213 External input / output section 214 GNSS receiver 215 Motion Sensor Part 216 Power supply section 217 Internal storage 218 External memory unit 220 Main control unit e1 1st Area e2 Area 2 e3 overlapping area r1, r2, g1, g2, b1, b2, 52A, 52B Phase difference detection pixels t1,t2,t3,t11,t12,t13,t14,t15 Time D2~D5 pixel data d1,d2 Images for subject detection ST1~STn GNSS satellite

Claims

1. an imaging element that captures an image of a subject through an imaging optical system; A processor and an imaging device comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a position of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a position of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. Imaging device.

2. The imaging device according to claim 1, The imaging element periodically captures images, the first time, the second time, and the third time are each included in a time of periodic imaging by the imaging element; Imaging device.

3. The imaging device according to claim 1 or 2, the pixel data obtained at the third time is pixel data obtained by the imaging element immediately before the time when the second area is detected; Imaging device.

4. An image sensor for capturing an image of a subject through an image capturing optical system; A processor and an imaging device comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated by selectively using a first focus position based on data of the second area in pixel data obtained at the second time and a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; When outputting a control signal generated using the second focus position to the lens driving mechanism, a control signal to be output to the lens driving mechanism after detecting the second area is generated using a third focus position based on data of an overlapping portion of the second area and the first area in pixel data obtained by the image sensor between a time when a first area detected as a subject area is detected from pixel data obtained by the image sensor at a first time before the second time and a time when the second area is detected. Imaging device.

5. An image sensor for capturing an image of a subject through an image capturing optical system; A processor and an imaging device comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a size of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a size of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. Imaging device.

6. An image sensor for capturing an image of a subject through an image capturing optical system; A processor and an imaging device comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a detection time of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a detection time of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. Imaging device.

7. An image sensor for capturing an image of a subject through an image capturing optical system; A processor and an imaging device comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a size of the second area is equal to or larger than a predetermined threshold value, outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated using a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; if the size is less than the threshold value, outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated using a first focus position based on data of the second area in the pixel data obtained at the second time; Imaging device.

8. A focus control method for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a position of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a position of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. Focus control method.

9. The focus control method according to claim 8, The imaging element periodically captures images, the first time, the second time, and the third time are each included in a time of periodic imaging by the imaging element; Focus control method.

10. The focus control method according to claim 8, further comprising the steps of: the pixel data obtained at the third time is pixel data obtained by the imaging element immediately before the time when the second area is detected; Focus control method.

11. A focus control method for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated by selectively using a first focus position based on data of the second area in pixel data obtained at the second time and a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; When outputting a control signal generated using the second focus position to the lens driving mechanism, a control signal to be output to the lens driving mechanism after detecting the second area is generated using a third focus position based on data of an overlapping portion of the second area and the first area in pixel data obtained by the image sensor between a time when a first area detected as a subject area is detected from pixel data obtained by the image sensor at a first time before the second time and a time when the second area is detected. Focus control method.

12. A focus control method for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a size of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a size of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. Focus control method.

13. A focus control method for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a detection time of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a detection time of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. Focus control method.

14. A focus control method for an imaging apparatus including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a size of the second area is equal to or larger than a predetermined threshold value, outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated using a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; if the size is less than the threshold value, outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated using a first focus position based on data of the second area in the pixel data obtained at the second time; Focus control method.

15. A focus control program for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a position of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a position of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. A focus control program for executing processing.

16. A focus control program for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated by selectively using a first focus position based on data of the second area in pixel data obtained at the second time and a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; When outputting a control signal generated using the second focus position to the lens driving mechanism, a control signal to be output to the lens driving mechanism after detecting the second area is generated using a third focus position based on data of an overlapping portion of the second area and the first area in pixel data obtained by the image sensor between a time when a first area detected as a subject area is detected from pixel data obtained by the image sensor at a first time before the second time and a time when the second area is detected. A focus control program for executing processing.

17. A focus control program for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a size of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a size of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. A focus control program for executing processing.

18. A focus control program for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a difference between a detection time of a first area detected as a subject area from pixel data obtained by the image sensor at a first time before the second time and a detection time of the second area is equal to or smaller than a predetermined threshold value, a control signal is generated using a second focus position based on data of the second area in pixel data obtained by the image sensor at a third time after the second time, and is output to a lens drive mechanism of the image sensor; if the difference exceeds the threshold value, a control signal generated using a first focus position based on data of the second area in the pixel data obtained at the second time is output to a lens drive mechanism of the imaging optical system. A focus control program for executing processing.

19. A focus control program for an imaging device including an image sensor that captures an image of a subject through an imaging optical system, and a processor, comprising: The processor, Detecting a subject area as a second area from pixel data obtained by the imaging element at a second time; when a size of the second area is equal to or larger than a predetermined threshold value, outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated using a second focus position based on data of the second area in pixel data obtained by the imaging element at a third time after the second time; if the size is less than the threshold value, outputting a control signal to a lens drive mechanism of the imaging optical system, the control signal being generated using a first focus position based on data of the second area in the pixel data obtained at the second time; A focus control program for executing processing.

Images