Imaging apparatus, method for controlling the same, and program

Abstract

To perform AF photographing by detecting a specific subject from among many subjects, without hindering others around from viewing the subject.SOLUTION: An imaging apparatus 100 includes an imaging element 102 for outputting each signal of a visible light image and an invisible light image, a code detection part 105 for detecting a code pattern and the code value of the code pattern from the invisible light image, a code collation part 106 for collating the code value of the detected code pattern with a pre-registered personal code value, and a tracking area decision part 109 for deciding a tracking area for performing auto-focusing by an optical system 101 for forming an optical image of the subject on the imaging element 102. The tracking area decision part 109 decides an area in the visible light image of the code pattern determined by the code collation part 106 to coincide with the pre-registered personal code value among one or a plurality of predetermined code patterns as the tracking area.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, and a control method and program thereof. [Background technology]

[0002] When selecting a specific subject from among many subjects to photograph using autofocus (AF) photography, a technology has been proposed in which each subject is equipped with a light-emitting means and the imaging device identifies the subject by recognizing the light emission pattern (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-075635 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in the above Patent Document 1, since the light emitting means emits blinking light to the surroundings, for example, if an athlete participating in a game or a child competing at an athletic meet wears the light emitting means, the blinking light will be a hindrance to the surrounding people (spectators) watching the game. In addition, when shooting a video, the blinking light is also captured, so when the captured video is later viewed, the blinking light will be a hindrance to viewing.

[0005] The present invention has been made in consideration of the above circumstances, and aims to provide an imaging device that makes it possible to detect a specific subject from among many subjects and perform AF shooting without impeding the visibility of the subjects of other people in the vicinity. [Means for solving the problem]

[0006] The imaging device of the present invention comprises an imaging element having a first pixel that receives visible light and a second pixel that receives invisible light, a detection means for detecting a predetermined object from an invisible light image obtained by output from the second pixel, a comparison means for comparing the predetermined object with objects pre-registered in a storage means, and a determination means for determining a tracking area for performing autofocusing by an optical system that forms an optical image of a subject on the imaging element, wherein the determination means determines as the tracking area an area in the visible light image obtained by output from the first pixel of an object that the comparison means determines to match the pre-registered object among one or more of the predetermined objects. Effect of the Invention

[0007] According to the present invention, it is possible to detect a specific subject from among many subjects and perform AF shooting without impeding the visibility of the subjects of other people in the vicinity. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram of an imaging device according to a first embodiment. [Diagram 2] 2 is a schematic diagram showing a pixel arrangement and a pixel configuration of an imaging element. FIG. [Diagram 3] 3A to 3C are diagrams illustrating examples of a visible light image and an infrared light image in the first embodiment. [Figure 4] FIG. 13 is a diagram showing an example of a code pattern read from an infrared light image. [Diagram 5] 4 is a flowchart of an imaging operation in the imaging device according to the first embodiment. [Figure 6] FIG. 1 is a schematic diagram showing a state in which one subject is selected from a plurality of subjects. [Figure 7] FIG. 13 is a schematic diagram showing a focus detection area superimposed on an infrared light image. [Figure 8] FIG. 2 is a diagram showing a visible light image of a subject and a corresponding infrared light image. [Figure 9]FIG. 11 is a block diagram showing a schematic configuration of an imaging device according to a second embodiment. [Figure 10] 13A to 13C are diagrams illustrating examples of a visible light image and an infrared light image in the second embodiment. [Figure 11] 10 is a flowchart of an imaging operation in an imaging device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram showing a schematic configuration of an image pickup device 100 according to a first embodiment. The image pickup device 100 includes an MPU 112, an optical system 101, an optical system control unit 111, an image pickup element 102, an image processing unit 103, a memory control unit 104, and a memory unit 113. The image pickup device 100 also includes a code detection unit 105, a code matching unit 106, a database 107, a focus detection unit 108, a tracking region determination unit 109, and an optical system control information generation unit 110. Although not shown, the image pickup device 100 includes a display device such as a liquid crystal panel having a touch panel for displaying live view images, captured images, and a menu screen for performing various settings, and an operation unit (buttons, switches, etc.) for the photographer to issue various instructions to the MPU 112.

[0010] In the imaging device 100, the optical system 101 and the optical system control unit 111 are assumed to be provided integrally with a main body (housing) (not shown) of the imaging device 100. However, this is not limited thereto, and the imaging device 100 may be configured such that a lens barrel (interchangeable lens) having the optical system 101 and the optical system control unit 111 is detachable from the main body of the imaging device 100.

[0011] The optical system 101 has driving mechanisms such as a focus lens, a zoom lens, an aperture, and motors for driving these, and focuses a light beam from a subject on an imaging surface of the image sensor 102. The optical system control unit 111 receives optical system control information (specifically, the defocus amount) output from the optical system control information generation unit 110, generates an optical system control signal for driving the optical system 101, and outputs the signal to the optical system 101. In this embodiment, the optical system control signal includes a drive signal for moving a focus lens constituting the optical system 101 in the optical axis direction.

[0012] The MPU 112 is a semiconductor device that comprehensively controls the operation of each component of the imaging apparatus 100. The imaging element 102 photoelectrically converts the subject optical image formed on the imaging plane by the optical system 101 to generate an analog electrical signal. In this embodiment, the imaging element 102 includes an A / D converter that converts the generated analog electrical signal into an image signal consisting of a digital signal. Therefore, the imaging element 102 outputs an image signal consisting of a digital signal (hereinafter simply referred to as an "image signal").

[0013] Although details will be described later, the image sensor 102 has pixels sensitive to visible light and pixels sensitive to non-visible light (specifically, infrared light), and each pixel has two photoelectric conversion units. With this configuration, it is possible to obtain from the image sensor 102 a visible light image signal and a visible light parallax image signal obtained by A / D converting output signals from pixels sensitive to visible light, and a non-visible light image signal obtained by A / D converting output signals from pixels sensitive to non-visible light. The MPU 112 outputs the visible light image signal to the image processing unit 103, outputs the non-visible light image signal to the code detection unit 105, and outputs the visible light parallax image signal to the focus detection unit 108.

[0014] The image processing unit 103 acquires a visible light image signal, performs image processing such as gamma correction, white balance processing, and development processing on the visible light image, and outputs the developed visible light image data (specifically, captured image data or video data) to the storage control unit 104. The image processing unit 103 can also detect a human face from the visible light image, and output the detection result (coordinate information of the detected face) to the MPU 112.

[0015] The storage control unit 104 stores the visible light image data output from the image processing unit 103 in the storage unit 113, and also reads out the visible light image data stored in the storage unit 113. The storage unit 113 is a memory card that can be inserted into and removed from the imaging device 100, but is not limited to this.

[0016] The code detection unit 105 acquires the non-visible light image signal and detects a predetermined object. In this embodiment, a one-dimensional code pattern (hereinafter referred to as a "code pattern") each including a code value as predetermined ID information is used as the object. The code detection unit 105 detects the code pattern from the non-visible light image signal and reads the code value. Details of the code pattern will be described later.

[0017] Furthermore, the code detection unit 105 outputs coordinate information of the area of ​​the detected code pattern on the invisible light image and the read code value to the code matching unit 106. When the code detection unit 105 detects multiple code patterns, it outputs the coordinate information and the code value for each detected code pattern to the code matching unit 106.

[0018] The database 107 is a rewritable storage device such as an EEPROM, and stores the names of a plurality of subjects in association with personal code values ​​assigned to each of the plurality of subjects. The imaging device 100 includes a user interface (see FIG. 6) for selecting one subject to be photographed from among the plurality of subjects stored in the database 107. Note that a memory included in the storage unit 113 or the MPU 112 may be used as the database 107.

[0019] The code matching unit 106 judges whether or not the code value acquired from the code detection unit 105 matches a personal code value associated with one person selected by the photographer from among multiple subjects stored in the database 107. The code matching unit 106 outputs coordinate information of the code pattern with the matching code value to the tracking region determination unit 109. Note that a method of selecting one subject from multiple subjects and acquiring the personal code value will be described later.

[0020] The tracking area determination unit 109 determines a predetermined area on the invisible light image of the code pattern acquired from the code matching unit 106 as a tracking area, and outputs coordinate information of the tracking area to the optical system control information generation unit 110. The tracking area determination unit 109 also displays a tracking marker in an area on the live view image corresponding to the tracking area. For example, a circular ring or a rectangular frame can be used as the tracking marker, and the tracking marker is displayed in, for example, orange so as to stand out on the live view image. An example of the tracking area will be described later, and the coordinate information of the tracking area can be treated as information common to the invisible light image, the visible light image, and the visible light parallax image.

[0021] The focus detection unit 108 performs focus detection by what is called an imaging surface phase difference method for each predetermined region using the visible light parallax image, and outputs the obtained focus detection information (amount of image shift) to the optical system control information generation unit 110 via the tracking region determination unit 109. Note that the focus detection information (amount of image shift) may be output directly from the focus detection unit 108 to the optical system control information generation unit 110.

[0022] The optical system control information generation unit 110 compares the coordinate information of the tracking region acquired from the tracking region determination unit 109 with the focus detection information acquired from the focus detection unit 108. Then, the optical system control information generation unit 110 calculates optical system control information (defocus amount) for driving the optical system 101 from the focus detection value (image shift amount) in the tracking region, and outputs the optical system control information to the optical system control unit 111.

[0023] The memory control unit 104, the code detection unit 105, the code matching unit 106, the focus detection unit 108, the tracking region determination unit 109, the optical system control information generation unit 110, and the optical system control unit 111 may be configured as physically separate control devices. However, without being limited to this, each of these units may realize each function by one or more CPUs executing a predetermined program.

[0024] Next, the configuration of the image sensor 102 capable of acquiring a visible light image and a non-visible light image will be described. FIG. 2(a) is a schematic diagram showing a pixel array of the image sensor 102. On the image sensor 201, photoelectric conversion elements (not shown) such as photodiodes are regularly arranged vertically and horizontally at predetermined intervals (in a grid pattern) on a plane. In addition, a color filter is provided above each photoelectric conversion element for transmitting light of a specific wavelength band from the light beam from the subject. The color filter is composed of an R filter 202 that transmits red light, an IR filter 203 that transmits infrared light, a G filter that transmits green light, and a B filter that transmits blue light, which are regularly arranged as shown in the figure. The light beam transmitted through each color filter is photoelectrically converted into an analog electric signal by a photoelectric conversion unit arranged directly below the filter of each color. The analog electric signal is further converted into an image signal consisting of a digital signal by an A / D converter (not shown) and output from the image sensor 102.

[0025] 2B is a diagram showing the configurations of the R pixel 210 and the IR pixel 220 of the image sensor 102. The R pixel 210 includes two photoelectric conversion units 211, 212, one microlens 213 arranged relative to the photoelectric conversion units 211, 212, and an R filter 202 (not shown in FIG. 2B) arranged above the microlens 213. Similarly, the IR pixel 220 includes two photoelectric conversion units 221, 222, one microlens 223 arranged relative to the photoelectric conversion units 221, 222, and an IR filter 203 (not shown in FIG. 2B) arranged above the microlens 223. The configurations of the G pixel and the B pixel are similar to that of the R pixel.

[0026] From the image sensor 102, for example, by extracting signals only from the photoelectric conversion units on the left side of the RGB pixels and extracting signals only from the photoelectric conversion units on the right side of the RGB pixels, a visible light parallax image formed by light beams passing through different regions of the exit pupil can be obtained. Also, by simultaneously extracting signals from a pair of photoelectric conversion units of the RGB pixels, a visible light image for recording and display can be obtained. And, by simultaneously extracting signals from a pair of photoelectric conversion units of the IR pixel 220, a non-visible light image can be obtained. In this embodiment, the non-visible light image is specifically an infrared light image, so in the following description, the term "infrared light image" will be used instead of "non-visible light image".

[0027] Next, a visible light image and an infrared light image will be described. Fig. 3(a) is a diagram showing an example of a visible light image obtained by the image sensor 102. A visible light image 300 in Fig. 3(a) is one frame of a live view image, and three children are captured as subjects in the visible light image 300, and the three children are wearing hats 301, 303, and 305, respectively. Fig. 3(b) is a diagram showing an infrared light image obtained simultaneously with the visible light image 300, and an infrared light image 310 shows a schematic diagram of the children in the visible light image 300 captured as infrared light images.

[0028] Here, hats 301, 303, and 305 worn by three children have predetermined code patterns 311, 313, and 315 drawn on them using a material that absorbs infrared light. The material that absorbs infrared light is, for example, a colorless infrared absorbing pigment or a colored infrared absorbing dye that absorbs near-infrared light in the atmosphere. The code patterns 311, 313, and 315 drawn on them using a material that absorbs infrared light are difficult to see under a visible light source, but when an image is captured using the image capture element 102 equipped with the IR pixels 220, a difference in brightness appears between a portion where infrared light in the atmosphere is absorbed and a portion where infrared light is reflected. This makes it possible to detect the code patterns 311, 313, and 315. When the code patterns are drawn using an infrared absorbing pigment or an infrared absorbing dye, it is desirable to use a material with high infrared reflectivity for the material of the hat in order to increase the difference in brightness.

[0029] Conversely to the above configuration, the code patterns 311, 313, 315 may be drawn using a material that reflects infrared light. In this case, in order to increase the contrast between light and dark, it is desirable for the material of the hat to be one that has high infrared absorption properties.

[0030] FIG. 4 is a diagram showing an example of a code pattern that can be read from an infrared light image. A 'start code' and an 'end code' are provided in the code pattern 400 so that the code pattern can be distinguished from patterns other than the code pattern 400, and an 'ID code' is provided in the area between them. The 'start code' and the 'end code' are given in common to the code patterns of all the subjects (the three children's hats 301, 303, 305) and have a predetermined line pattern. On the other hand, the 'ID code' has a different line pattern for each of the code patterns of the hats 301, 303, 305. That is, the code pattern 400 is expressed by light and dark with a prescribed line width, and is drawn as 32 bits of digital information in total, with light areas being '1' and dark areas being '0' when viewed as an infrared light image.

[0031] A known technique can be used to detect the code pattern from the infrared image and read the code value (the value indicated by the ID code). For example, a high-cut filter is applied in the pattern direction of the code pattern to generate projection data, and the signal level of the acquired projection data (data in which the light and dark of the code pattern is extracted) is compared with a predetermined threshold to detect the code value.

[0032] Next, a description will be given of the imaging operation by the imaging device 100. Fig. 5 is a flowchart of the imaging operation by the imaging device 100. Each process (step) indicated by an S number in this flowchart is realized by the MPU 112 executing a predetermined program to comprehensively control each unit constituting the imaging device 100, and each unit cooperating to execute a predetermined operation and process.

[0033] It should be noted that the photographing operation in this flowchart is assumed to be performed by a photographer holding the imaging device 100. Each step will be described below by taking the photographing scene in FIG.

[0034] Before starting shooting, the photographer selects one subject to be photographed from among a plurality of subjects stored in database 107. In the case of the shooting scene in Fig. 3, one of the three children is selected.

[0035] 6 is a schematic diagram showing the state of selecting one subject from a plurality of subjects registered in database 107. The photographer operates the operating means to cause the display device to display a list 600 including the names of the subjects stored in database 107 and the personal code values ​​linked to each subject. The photographer can select the name of one subject to be photographed from the names of the subjects displayed on the display device by a touch operation (selection operation). When this operation is performed, the personal code value linked to the selected one subject is held in code comparison unit 106.

[0036] When the MPU 112 receives an instruction to start shooting via an operation unit (not shown), it starts the operation of acquiring a live view image by the image sensor 102, thereby starting the process of S501.

[0037] In S501, the focus detection unit 108 performs focus detection using the visible light parallax image. As described above, the image sensor 102 can obtain a visible light image, an infrared light image, and a visible light parallax image. The focus detection unit 108 obtains a first parallax image based on the output from the photoelectric conversion unit 211 arranged on the left side in FIG. 2B and a second parallax image based on the output from the photoelectric conversion unit 212 arranged on the right side in each pixel of RGB. Then, the focus detection unit 108 performs correlation calculation processing, such as Sum of Absolute Difference (SAD) calculation, on the first parallax image and the second parallax image, and obtains an image shift amount between the two images based on the degree of correlation. Here, the image shift amount is obtained in area units defined by a predetermined number of pixels, but may be obtained in units of one pixel. The focus detection unit 108 transmits the image shift amount in area units and the center coordinates of each area to the optical system control information generation unit 110 as focus detection information.

[0038] As described above, the image sensor 102 includes the IR pixels 220, and is therefore capable of outputting an infrared image signal. In S502, the code detection unit 105 detects a code pattern from the infrared image, reads a code value of the detected code pattern, and outputs the read code value to the code matching unit 106. At this time, in order to improve the focusing accuracy for the code pattern and accurately detect the code pattern, the optical system 101 may be driven to increase the aperture value within a range that does not impede detection of the code pattern. In addition, when multiple code patterns are detected and the code patterns cannot be accurately detected by driving the aperture, the focus lens may be driven to focus on each code pattern in sequence.

[0039] In S503, the code matching unit 106 determines whether or not any of the code values ​​transmitted from the code detection unit 105 in S502 matches the personal code value previously selected by the photographer. If the code matching unit 106 determines that a matching code value exists (YES in S503), it executes the process of S504.

[0040] On the other hand, if there is no matching code value in S503, it means that there is no subject selected as the subject to be photographed in the current shooting range (object scene). As described later, a tracking area is determined in S505, and a tracking marker is displayed on the live view image for the tracking area. Since the photographer can recognize that the tracking marker is not displayed as a state in which the object to be photographed is not captured within the object scene, it is expected that the photographer will search for the object to be photographed by changing the shooting direction or the shooting angle of view, etc. Therefore, when the code collation unit 106 determines that there is no matching code value (NO in S503), the process returns to S501.

[0041] In S504, the code matching unit 106 outputs coordinate information of the code pattern indicating the matched code value to the tracking area determination unit 109. The coordinate information at this time may be, for example, coordinate information of the smallest rectangular area that includes the code pattern, or the central coordinate of that rectangular area. It is not limited to these, and may be coordinate information of one or more areas that are areas where the focus detection unit 108 is performing focus detection and that at least partially overlap with the code pattern, as will be described later. Here, it is assumed that information (coordinate values) of the central coordinate of the smallest rectangular area that includes the code pattern is output to the tracking area determination unit 109.

[0042] In S505, the tracking area determination unit 109 determines a tracking area based on the coordinate information of the code pattern acquired in S504, and outputs the coordinate information of the determined tracking area to the optical system control information generation unit 110. In addition, the tracking area determination unit 109 superimposes a tracking marker indicating the determined tracking area on a live view image of the display device.

[0043] Here, a method for determining the tracking area in S505 will be described. Fig. 7 is a diagram showing areas in which the focus detection unit 108 performs focus detection, superimposed on an infrared light image 310. Here, the focus detection unit 108 performs focus detection for each of 30 areas that are configured of 6 columns x 5 rows on the imaging surface.

[0044] In the infrared light image 310, a code pattern exists in each of the regions 701, 703, and 705. Here, it is assumed that in S504, the code collation unit 106 has determined that the code pattern in the region 701 has a code value that matches the personal code value previously selected by the photographer. In this case, the tracking region determination unit 109 determines the region 701, which has the center coordinates of the code pattern, and a region 710 including the neighboring regions, as the tracking region, and outputs the coordinate information of the region 710 to the optical system control information generation unit 110. The tracking marker superimposed on the live view image is displayed to indicate the region 710.

[0045] In addition, in Fig. 7, the area 710 is four areas in the upper left of the imaging surface, but for example, if the photographer changes the shooting direction so that the tracking marker is located in the center of the shooting angle of view, the four areas in the center of the shooting angle of view indicated by the area tracking marker become the tracking area. Also, here, an example is shown in which four areas are determined as the tracking area, but the method of determining the tracking area is not limited to this. For example, if the code pattern is contained in only one area, only that one area may be determined as the tracking area, and if the code pattern spans multiple areas, only the multiple areas that span the area may be determined as the tracking area.

[0046] In S506, the optical system control information generation unit 110 extracts the focus detection value in the tracking region from the focus detection information acquired in S501, calculates the defocus amount, and outputs it to the optical system control unit 111.

[0047] In S507, the optical system control unit 111 calculates the driving amount of the focus lens based on the defocus amount acquired from the optical system control information generation unit 110, and outputs the calculated driving amount to the optical system 101, thereby focusing on the tracking region.

[0048] In S508, the MPU 112 determines whether or not a main shooting instruction has been received. If the MPU 112 determines that a main shooting instruction has not been received (NO in S508), it executes the process of S506 to continue focusing on the tracking area, and if the MPU 112 determines that a main shooting instruction has been received (YES in S508), it executes the process of S509.

[0049] In S509, the MPU 112 performs a main image capture process. The main image capture process may be a still image capture process or a moving image capture process.

[0050] In S510, the MPU 112 determines whether or not an instruction to end shooting has been received. If the MPU 112 determines that an instruction to end shooting has been received (YES in S510), it ends this process, and if it determines that an instruction to end shooting has not been received (NO in S510), it executes the process of S506. As a result, the processes of S506 to S507 are continuously performed until shooting ends. Note that in the case of video shooting, the determination in S508 from the second time onwards will automatically be 'YES'.

[0051] As described above, in the first embodiment, first, a code pattern that appears due to reflection and absorption of infrared light is provided for each of a plurality of subjects. The imaging device 100 detects the code pattern from the infrared light image and reads the code value. The imaging device 100 then compares the read code value with a personal code value associated with the subject set in advance, and determines an area in which a code pattern showing a matching code value exists as a tracking area. This makes it possible to identify the subject from among a plurality of subjects without interfering with the visual recognition of the subjects by the naked eye under a visible light source, and thereafter, by tracking the subject and performing AF processing, it becomes possible to obtain an image or video in which the subject is focused.

[0052] The settings in the above embodiment are merely examples, and may be changed as follows. For example, in the above embodiment, a personal code value is assigned to each of a plurality of subjects and stored in the database 107, and the personal code value associated with one subject to be photographed is read out and compared with the code value indicated by the code pattern. In addition to this method, in the case of the photographing scene in FIG. 3, a code pattern drawn on a hat worn by a child to be photographed is photographed in advance and stored in the memory of the MPU 112 or the like. Then, when live view display is started for actual photographing, the same code pattern as the code pattern stored in the memory of the MPU 112 or the like may be extracted from the infrared image to determine the tracking area. When using such a method, an object such as a predetermined pattern may be used instead of the code pattern.

[0053] In the above embodiment, the tracking area to be focused on is set to the area where the code pattern exists. This is because, when the code pattern is drawn on the hat as in the shooting scene of Fig. 3, the face of the subject is close to the hat, so it is considered that the face of the subject can be focused on by setting the area where the code pattern is detected as the tracking area.

[0054] On the other hand, it is presumed that there is a demand to reliably focus on the face of the subject. In that case, for example, when a code pattern is drawn on a hat, the subject's face is located immediately below the hat, so an area adjacent to and below the area where the code pattern is detected may be selected as the tracking area. For example, in the explanation of FIG. 7, only an area 701A immediately below an area 701 where a code pattern is present may be determined as the tracking area. Alternatively, face detection may be performed by the image processing unit 103, and an area of ​​a face adjacent to the code pattern showing a code value that matches the personal code value of the subject may be determined as the tracking area. Information regarding the face area can be acquired by the tracking area determination unit 109 from the image processing unit 103 or the MPU 112.

[0055] The part (position) on which the code pattern for identifying the subject to be photographed from among a plurality of subjects is drawn is not limited to the hat. FIG. 8(a) is a diagram showing a visible light image of the subject. FIG. 8(b) is a schematic diagram showing an infrared light image corresponding to the visible light image of FIG. 8(a). A code pattern 810 is drawn on the subject's jacket 801, but the code pattern 810 does not appear in the visible light image as shown in FIG. 8(a), and the code pattern 810 can be detected in the infrared light image as shown in FIG. 8(b). By drawing the code pattern 810 so as to go around the jacket 801, the code pattern (code value) can be detected no matter which direction the subject faces. In the above embodiment, a configuration using a one-dimensional code pattern as shown in FIG. 4 and FIG. 8 has been described, but a two-dimensional code pattern may be adopted depending on the photographing scene, etc.

[0056] The image sensor 102 has RGB pixels that receive visible light and IR pixels that receive infrared light, and correspondingly, a code pattern that can be read by infrared light is drawn on multiple subjects. This configuration is not limited to this, and the image sensor may have UV pixels that receive ultraviolet light instead of IR pixels, and the code pattern for detecting one subject to be photographed from multiple subjects may be read by ultraviolet light. With this configuration, the same effect as the above embodiment can be obtained.

[0057] In addition, in the first embodiment, a configuration has been described in which focus detection is performed using an imaging surface phase difference method, but focus detection may be performed using a general phase difference method with each of the RGB pixels of the image sensor 102 having one photoelectric conversion unit. The IR pixel 220 has two photoelectric conversion units like the R pixel 210, but since focus detection is not performed using an output signal from the IR pixel 200, the IR pixel may have one photoelectric conversion unit.

[0058] Incidentally, in the above embodiment, the driving of the aperture constituting the optical system 101 and the change of the ISO sensitivity for setting the luminance of the tracking target to an appropriate value in association with the driving of the aperture are not mentioned, but the adjustment of the aperture and the ISO sensitivity are performed, for example, as follows. For example, the MPU 112 always controls the aperture and the ISO sensitivity so that the luminance is balanced over the entire shooting angle of view. Therefore, after the determination of the tracking area, the MPU 112 may acquire luminance information in the tracking area in the visible light image to set an appropriate ISO sensitivity and aperture value, and when the aperture value is changed, the changed aperture value may be output to the optical system control information generation unit 110. In this case, the optical system control information generation unit 110 calculates the aperture drive amount in addition to the defocus amount, and outputs it to the optical system control unit 111. The optical system control unit 111 calculates the control amount of the motor driving the focus lens and the aperture, and controls the optical system 101.

[0059] Next, an imaging device according to a second embodiment will be described. In the second embodiment, the number of code patterns to be matched with a pre-set personal code value is narrowed down from among a plurality of code patterns in an infrared light image using the analysis result of a visible light image.

[0060] 9 is a block diagram showing a schematic configuration of an image capturing device 100A according to the second embodiment. The configuration of the image capturing device 100A differs from the configuration of the image capturing device 100 according to the first embodiment only in that the image capturing device 100A includes an image analysis unit 901. Therefore, the same components of the image capturing device 100A as those of the image capturing device 100 are denoted by the same reference numerals and will not be described.

[0061] The image analysis unit 901 detects features of a subject in the image from the visible light image developed by the image processing unit 103, and outputs the detection result to the code matching unit 106 that outputs the detection result. The features of the subject can be a predetermined color or a body part such as the head, face, torso, hands, and feet when the subject is a human. A method for detecting a region of a predetermined color from a visible light image can use a known technique such as threshold determination based on a color difference signal constituting the visible light image. Also, a method for detecting a body part such as the head or torso can use a known technique such as template matching. In the imaging device 100A, the image analysis unit 901 is provided as a functional block different from the image processing unit 103, but the image processing unit 103 may have the function of the image analysis unit 901.

[0062] Fig. 10(a) is a diagram showing an example of a visible light image obtained by the image sensor 102. The visible light image 1000 in Fig. 3(a) is one frame of a live view image, and three children are captured as subjects in the visible light image 1000. Fig. 10(b) is a diagram showing an infrared light image obtained simultaneously with the visible light image 1000, and an infrared light image 1010 diagrammatically shows how the children in the visible light image 1000 are captured as infrared light images.

[0063] The three children are wearing hats 1001, 1003, and 1005, respectively, and code patterns 1011, 1013, and 1015 are drawn on the hats 1001, 1003, and 1005, respectively. The three children are also wearing bibs (numbers) 1002, 1004, and 1006, respectively, and code patterns 1012, 1014, and 1016 are drawn on the bibs 1002, 1004, and 1006, respectively. The drawing of the code patterns in the hats 1001, 1003, and 1005 and the bibs 1002, 1004, and 1006 is similar to the drawing of the code patterns 311, 313, and 315 in the first embodiment, so a description thereof will be omitted here.

[0064] 10(b), when many code patterns exist in an infrared light image, it takes a long time to determine a tracking area if code values ​​are detected from all the code patterns and compared with a predetermined personal code value by the code comparison unit 106. Therefore, the image analysis unit 901 detects the features of the subject from a visible light image developed in a format such as YUV (luminance, color difference), and outputs coordinate information of the detected feature area to the code detection unit 105. Based on the coordinate information of the feature area acquired from the image analysis unit 901, the code detection unit 105 detects only the code patterns in the feature area and reads the code value.

[0065] In order to realize such a process, the imaging device 100A has a user interface (not shown) for designating a feature to be detected from a menu screen displayed on a display device. For example, in the shooting scene of FIG. 10, the photographer can designate the color of the hat 1001. Also, if the photographer wants to detect the code patterns of the hats 1001, 1003, and 1005 and not to detect the code patterns of the bibs 1002, 1004, and 1006, the photographer can designate the head of the subject (child). When the color of the hat 1001 is designated, the code detection unit 105 detects only the code pattern 1011, reads the code value, and outputs it to the code matching unit 106. When the head of the subject (child) is designated, the code detection unit 105 detects the code patterns 1011, 1013, and 1015, reads the code values, and outputs them to the code matching unit 106.

[0066] Fig. 11 is a flowchart of the imaging operation by the imaging device 100A. Each process (step) indicated by an S number in this flowchart is realized by the MPU 112 executing a predetermined program to comprehensively control each unit constituting the imaging device 100A, and each unit cooperating to execute a predetermined operation and process. Note that advance preparations (setting of a personal code value, etc.) conform to those in the first embodiment. Also, the shooting operation in this flowchart assumes a scene in which a photographer holds the imaging device 100A and shoots. Also, the shooting scene in Fig. 10 will be appropriately taken up and described.

[0067] The content of the process in S1101 is the same as the process in S501 in the flowchart of the first embodiment (FIG. 5), so a description thereof will be omitted.

[0068] In S1102, the image analysis unit 901 analyzes the visible light image developed by the image processing unit 103, and detects features designated in advance by the photographer. For example, the head of the human subject (the three children in the case of the shooting scene in FIG. 10) is detected as a feature region. The image analysis unit 901 outputs coordinate information of the detected feature region to the code detection unit 105.

[0069] In S1103, the code detection unit 105 detects a code pattern from an area on the infrared light image corresponding to the characteristic area detected from the visible light image, and reads the code value of the extracted code pattern. For example, in the case of the photographed scene of Fig. 10, if the heads of three children are detected as characteristic areas in S1102, code patterns 1011, 1013, and 1015 are detected from hats 1001, 1003, and 1005 in S1103, and their code values ​​are read. Note that code patterns 1012, 1014, and 1016 are drawn on bibs 1002, 1004, and 1006 of the three children, respectively, but the code patterns 1012, 1014, and 1016 are excluded from detection targets by the process of S1102.

[0070] The processing in S1104 to S1111 is the same as the processing in S503 to S510 in the first embodiment (FIG. 5), and therefore a description thereof will be omitted.

[0071] As described above, in the second embodiment, even in a shooting scene where many code patterns are detected from an infrared light image, the number of code patterns to be detected can be limited based on the feature area of ​​the shooting target detected from a visible light image. This makes it possible to quickly determine the shooting target and the tracking area, and the more code patterns that can be detected from an infrared light image, the more noticeable this effect becomes.

[0072] In the second embodiment, the image analysis unit 901 first detects the features, thereby limiting the code patterns to be detected by the code detection unit 105. However, the present invention is not limited to this, and the code detection unit 105 may first read the code values ​​of a plurality of code patterns, and then the image analysis unit 901 may detect a characteristic region, and the code matching unit 106 may narrow down the plurality of code values ​​based on the characteristic region and compare them with the personal code value of the subject to be photographed. Even in this case, it is possible to determine the subject to be photographed (tracking region) more quickly than when the processing by the image analysis unit 901 is not incorporated.

[0073] In the second embodiment, the image analysis unit 901 detects the features of a subject that may be a subject of photography from a visible light image, but the features may be detected from an infrared light image. In this case, since the infrared light image does not have color information, it is not possible to specify a color and detect an area of ​​that color, but it is possible to detect body parts such as the head and arms as silhouettes.

[0074] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) An imaging device comprising an imaging element having a first pixel that receives visible light and a second pixel that receives invisible light, a detection means that detects a predetermined object from an invisible light image obtained by output from the second pixel, a comparison means that compares the predetermined object with objects pre-registered in a storage means, and a determination means that determines a tracking area for performing autofocus using an optical system that forms an optical image of a subject on the imaging element, wherein the determination means determines as the tracking area an area in the visible light image obtained by output from the first pixel of an object that the comparison means determines to match the pre-registered object among one or more of the predetermined objects. (Configuration 2) An imaging device as described in configuration 1, comprising a control means for controlling the optical system and a focus detection means for performing focus detection for each predetermined area based on the visible light image, wherein the optical system has a focus lens, and the control means controls the driving of the focus lens so that the focus is achieved in the tracking area. (Configuration 3) The imaging device described in Configuration 2, wherein the first pixel has a plurality of photoelectric conversion units that receive light beams passing through different regions of an exit pupil of the optical system, and the focus detection means performs focus detection using an image plane phase difference method based on signals output from the plurality of photoelectric conversion units. (Configuration 4) The imaging device according to any one of configurations 1 to 3, wherein the non-visible light is infrared light or ultraviolet light. (Claim 5) The imaging device according to any one of configurations 1 to 4, wherein the object is a one-dimensional code or a two-dimensional code having predetermined ID information. (Configuration 6) The imaging device according to any one of configurations 1 to 5, wherein the object is made of a material that absorbs the invisible light or a material that reflects the invisible light. (Configuration 7) The imaging device according to any one of configurations 1 to 6, wherein the object is provided so as to surround each of a plurality of subjects. (Configuration 8) An imaging device as described in any one of configurations 1 to 7, comprising a designation means for designating a feature of a subject, and an extraction means for extracting a feature region having the feature from the visible light image, wherein the detection means detects the specified object from a region in the non-visible light image corresponding to the feature region. (Configuration 9) An imaging device described in any one of configurations 1 to 7, comprising a designation means for designating characteristics of a subject, and an extraction means for extracting a characteristic area having the characteristics from the visible light image, wherein the matching means matches an object that is in the characteristic area among the specified objects detected by the detection means with the pre-registered object. (Configuration 10) The imaging device according to configuration 8 or 9, wherein the feature is a color or a body part of the subject. (Configuration 11) An imaging device described in any one of configurations 1 to 7, comprising a designation means for designating a body part of a subject, and an extraction means for extracting the body part from the non-visible light image, wherein the detection means detects the specified object from the area of ​​the body part in the non-visible light image. (Configuration 12) An imaging device as described in any one of configurations 1 to 7, comprising a designation means for designating a body part of a subject, and an extraction means for extracting the body part from the non-visible light image, wherein the matching means matches an object that is in the area from which the body part is extracted among the specified objects detected by the detection means with the pre-registered object. (Configuration 13) A program for causing a computer to function as each of the means of the imaging device according to any one of configurations 1 to 12. (Method 1) A method for controlling an imaging device, comprising the steps of: detecting a specified object from an invisible light image obtained by output from a second pixel of an imaging element having a first pixel that receives visible light and a second pixel that receives invisible light; comparing the specified object with pre-registered objects; and determining an area in a visible light image obtained by output from the first pixel of an object determined to match the pre-registered object among one or more of the specified objects as a tracking area for performing autofocus.

[0075] Although the present invention has been described in detail above based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-mentioned embodiments merely shows one embodiment of the present invention, and each embodiment can be appropriately combined.

[0076] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0077] 100,100A Imaging Device 101 Optical system 102 Image sensor 103 Image processing section 105 Code detector 106 Code Verification Unit 108 Focus detection section 109 Tracking area determination unit 110 Optical system control information generator 111 Optical system control unit 112 MPU

Claims

1. an imaging element having a first pixel that receives visible light and a second pixel that receives non-visible light; a detection means for detecting a predetermined object from a non-visible light image obtained by output from the second pixel; a matching means for matching the predetermined object with an object previously registered in a storage means; a determination unit for determining a tracking area for performing autofocus by an optical system that forms an optical image of a subject on the image sensor, The imaging device is characterized in that the determination means determines, as the tracking area, an area in a visible light image obtained by output from the first pixel of an object that the comparison means determines to match the pre-registered object among one or more of the specified objects.

2. A control means for controlling the optical system; a focus detection unit that performs focus detection for each predetermined area based on the visible light image, The optical system includes a focus lens.

2. The imaging apparatus according to claim 1, wherein the control means controls the driving of the focus lens so that the focus is adjusted in the tracking area.

3. the first pixel has a plurality of photoelectric conversion units that receive light beams passing through different regions of an exit pupil of the optical system, 3. The image pickup apparatus according to claim 2, wherein the focus detection means performs focus detection by an image pickup surface phase difference method based on signals output from the plurality of photoelectric conversion units.

4. 3. The imaging device according to claim 1, wherein the invisible light is infrared light or ultraviolet light.

5. 3. The imaging device according to claim 1, wherein the object is a one-dimensional code or a two-dimensional code having predetermined ID information.

6. 3. The imaging device according to claim 1, wherein the object is made of a material that absorbs the invisible light or a material that reflects the invisible light.

7. 3. The imaging apparatus according to claim 1, wherein the object is provided around each of a plurality of subjects so as to surround the subject.

8. A designation means for designating a feature of a subject; an extraction means for extracting a feature region having the feature from the visible light image, The imaging device according to claim 1 , wherein the detection means detects the predetermined object from a region in the non-visible light image that corresponds to the characteristic region.

9. A designation means for designating a feature of a subject; an extraction means for extracting a feature region having the feature from the visible light image, 2. The imaging apparatus according to claim 1, wherein the comparison means compares an object that is present in the characteristic region among the predetermined objects detected by the detection means with the preregistered object.

10. 10. The imaging apparatus according to claim 8, wherein the feature is a color or a body part of the subject.

11. A designation means for designating a body part of a subject; an extraction means for extracting the body part from the invisible light image, 3. The imaging device according to claim 1, wherein the detection means detects the predetermined object from a region of the body part in the non-visible light image.

12. A designation means for designating a body part of a subject; an extraction means for extracting the body part from the invisible light image, 3. The imaging device according to claim 1, wherein the matching means matches an object detected by the detection means that is in a region from which the body part is extracted, with the preregistered object.

13. A control method for an imaging device, comprising: detecting a predetermined object from a non-visible light image obtained by an output from a second pixel of an image sensor having a first pixel that receives visible light and a second pixel that receives non-visible light; A step of matching the predetermined object with a pre-registered object; determining, as a tracking area for performing autofocus, an area in a visible light image obtained by outputting from the first pixel of an object among one or more of the specified objects that is determined to match the pre-registered object, as a tracking area for performing autofocus.

14. A program for causing a computer to function as each of the means of the imaging apparatus according to claim 1 or 2.

Images