Control device, control method, and program

Abstract

To prevent a subject unsuitable for authentication from being detected.SOLUTION: A control device includes: determination means for determining a focus degree of two or more partial regions in a photographed image obtained by photographic means; and determination means for determining a detection area of a subject in a photographed image obtained by the photographic means, on the basis of the focus degree of each of the two or more partial regions determined by the determination means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a subject detection method. [Background technology]

[0002] There are systems that identify objects from captured images. One example is a face recognition system that detects a person from a captured image and authenticates whether the person is already registered. In such authentication systems, it takes time to recognize people if they are processed one by one, so it is required to authenticate multiple people simultaneously. In addition, since the detection accuracy required for detecting an object is generally lower than the recognition accuracy required for authenticating the object, for example, even if a face is detected, it may not be possible to authenticate the face. In addition, when there is an upper limit on the number of subjects that can be detected, if the detected subjects are dominated by subjects that do not reach the accuracy required for authentication, there is a problem that other subjects that would otherwise be successfully authenticated cannot be detected.

[0003] To address such problems, Patent Document 1 proposes a technology for narrowing down the target to be prioritized for detection in a system for detecting faces from captured images by determining the detection target based on the size of the face. Patent Document 2 also proposes a technology for narrowing down the target area for face detection in a mobile terminal equipped with an imaging function by excluding areas where focusing is impossible due to the hardware structure from the target of face detection. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-25719 A [Patent Document 2] JP 2010-183200 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, there is still a risk that a subject unsuitable for authentication processing may be detected. For example, in the technology disclosed in Patent Document 1, the detection target is determined by the size of the face, so that a subject with a large face size but insufficient image quality may be detected as a target for authentication processing. Also, in the technology disclosed in Patent Document 2, if the image quality of the detection area set based on the hardware constraints is insufficient, a subject unsuitable for authentication processing may continue to be detected as a target for authentication processing. The present invention has been made in consideration of the above circumstances, and has an object to make it easier to detect a subject that is more suitable for authentication processing. [Means for solving the problem]

[0006] The control device of the present invention is characterized in having a judgment means for judging the degree of focus of two or more partial regions in a captured image obtained by a photographing means, and a determination means for determining a detection area of ​​a subject in the captured image obtained by the photographing means based on the degree of focus of each of the two or more partial regions judged by the judgment means. Effect of the Invention

[0007] According to the present invention, a subject more suitable for authentication processing can be more easily detected. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating an example of a hardware configuration of an imaging apparatus. [Diagram 2] FIG. 2 is a diagram illustrating an example of the functional configuration of an imaging apparatus according to the first embodiment. [Diagram 3] 5 is a flowchart showing an example of processing performed by the imaging apparatus in the first embodiment. [Figure 4] 10A and 10B are diagrams for explaining determination of the focus level of a captured image, etc. FIG. [Diagram 5] FIG. 13 is a diagram illustrating detection priority. [Figure 6] FIG. 11 is a diagram illustrating an example of the functional configuration of an imaging apparatus according to a second embodiment. [Figure 7] 10 is a flowchart showing an example of processing performed by an imaging apparatus according to a second embodiment. [Figure 8] FIG. 13 is a diagram illustrating priority setting for a captured image. [Figure 9] FIG. 13 is a diagram illustrating a process in a third embodiment. [Figure 10] FIG. 13 is a diagram illustrating an example of the functional configuration of an imaging device according to a third embodiment. [Figure 11] 13 is a flowchart showing an example of processing performed by an imaging apparatus according to a third embodiment. [Figure 12] FIG. 13 is a diagram illustrating an example of the functional configuration of an imaging apparatus according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] (First embodiment) 1 is a block diagram showing an example of the hardware configuration of an imaging device to which a control device according to this embodiment is applied. The imaging device 100 has a CPU 101, a memory 102, a storage unit 103, an input unit 104, a display unit 105, an imaging unit 106, a communication unit 107, and a bus 108. The CPU 101, the memory 102, the storage unit 103, the input unit 104, the display unit 105, the imaging unit 106, and the communication unit 107 are connected to each other via the bus 108 so as to be able to communicate with each other. Note that the imaging device 100 may further have other configurations.

[0011] A CPU (Central Processing Unit) 101 is responsible for overall control of the imaging device 100. The CPU 101 controls the operation of each functional unit connected via, for example, a bus 108. The memory 102 stores programs and various data used by the CPU 101 for processing. The memory 102 also functions as a main memory, a work area, and the like for the CPU 101. The CPU 101 executes processing based on the programs stored in the memory 102 to realize functions and processing of the imaging device, which will be described later. Note that the functions and processing of the imaging device, which will be described later, may be realized using a processor other than the CPU. For example, a GPU (Graphics Processing Unit) may be used instead of the CPU.

[0012] The storage unit 103 stores, for example, various data required when the CPU 101 performs processing related to a program. The storage unit 103 also stores, for example, various data obtained by the CPU 101 performing processing related to a program. The storage unit 103 may store the programs and various data used by the CPU 101 for processing. The input unit 104 has an operation member such as a button or a switch, and inputs a user's operation to the imaging device 100. The display unit 105 has a display member such as a liquid crystal display, and displays the results of processing by the CPU 101, captured images, etc. The imaging unit 106 captures an image of a subject, etc., and generates an image. The communication unit 107 connects the imaging device 100 to a network and controls communication with other devices, etc.

[0013] 2 is a block diagram showing an example of the functional configuration of an image capturing device to which the control device of this embodiment is applied. The image capturing device 200 has a lens unit 201, an image capturing sensor 202, a signal processing unit 203, a subject detection unit 204, a focus level evaluation unit 205, a sensor control unit 206, a lens control unit 207, and a detection area control unit 208. Note that the image capturing device 100 and the image capturing device 200 in this embodiment are the same device.

[0014] The lens unit 201 includes a lens group including a focus lens and a zoom lens, an aperture mechanism, etc. The lens unit 201 collects light incident from a subject, etc., and forms an optical image on the imaging surface of the imaging sensor 202. The imaging sensor 202 is an imaging element such as a CMOS sensor or a CCD sensor, and converts light into an electric signal to generate an imaging signal. The imaging signal generated by the imaging sensor 202 is output to a signal processing unit 203. The signal processing unit 203 generates an image (captured image) by performing various signal processing and image processing on the imaging signal output from the imaging sensor 202. Note that the signal processing unit 203 may be configured with a dedicated circuit block (such as a video engine) for performing predetermined image processing, etc.

[0015] The subject detection unit 204 performs a detection process for a subject to be detected in the captured image generated by the signal processing unit 203, and outputs the detection result to the detection area control unit 208. The detection result output by the subject detection unit 204 includes, for example, information indicating the area of ​​the detected subject (hereinafter also referred to as the "subject area") (for example, coordinate information for identifying the subject area). The subject can be detected using an existing known method, such as a template matching method or a method of detecting a subject by deep learning. In this embodiment, as an example, the subject to be detected is a face, and the subject detection unit 204 performs face detection.

[0016] The focus evaluation unit 205 evaluates the focus degree, which is the degree of focus for the captured image generated by the signal processing unit 203, and outputs evaluation information regarding the focus degree of the captured image to the detection area control unit 208. The focus evaluation unit 205 is an example of a determination means. In the following, the focus degree is expressed as a numerical value as an example, and a larger numerical value indicates a more focused state. The focus evaluation unit 205 also performs various evaluations regarding camera control such as brightness and focus degree in the captured image, and based on the evaluation results, the sensor control unit 206 controls the imaging sensor 202 and the lens control unit 207 controls the lens unit 201.

[0017] The detection area control unit 208 determines a detection area of ​​the subject in the captured image based on the evaluation information on the focus level of the captured image from the focus level evaluation unit 205. The detection area control unit 208 outputs detection area information indicating the determined detection area of ​​the subject together with the image. The detection area control unit 208 is an example of a determination means. The detection area control unit 208 determines the detection area of ​​the subject from the subject area indicated by the subject detection result based on the focus level of a partial area corresponding to the subject area indicated by the evaluation information on the focus level, and outputs the detection area information.

[0018] The detection area control unit 208, for example, sets, as the detection area of ​​the subject, a subject area whose corresponding part has a focus degree greater than a predetermined threshold among the subject areas detected by the subject detection unit 204. Note that, when the number of subject areas detected by the subject detection unit 204 does not exceed a predetermined limit value (for example, the upper limit value of the number of detected subjects), all of the detected subject areas may be set as the detection area of ​​the subject regardless of the focus degree. Also, when the number of subject areas whose corresponding part in the captured image has a focus degree greater than a predetermined threshold exceeds a predetermined limit value (for example, the upper limit value of the number of detected subjects), a predetermined number of subject areas may be set as the detection area of ​​the subject in order from the side with the highest focus degree. Also, when the number of subject areas whose corresponding part in the captured image has a focus degree greater than a predetermined threshold exceeds a predetermined limit value (for example, the upper limit value of the number of detected subjects), a predetermined number of subject areas may be set as the detection area of ​​the subject according to the position in the captured image.

[0019] FIG. 3 is a flowchart showing an example of a process related to subject detection by the imaging device in the first embodiment. In step S301, the imaging device 100 captures video. In the video capture in step S301, an optical image formed by the lens unit 201 is photoelectrically converted by the imaging sensor 202 to generate an imaging signal, and the generated imaging signal is subjected to image processing and the like by the signal processing unit 203 to generate a captured image.

[0020] In step S302, the focus degree evaluation unit 205 determines the focus degree of the captured image generated in the video shooting in step S301. The focus degree evaluation unit 205 obtains the focus degree for each region (two or more partial regions) in the captured image and outputs it to the detection region control unit 208.

[0021] In step S303, the subject detection unit 204 performs subject detection processing on the captured image generated by the video shooting in step S301 to detect a subject area in the captured image. The subject detection unit 204 outputs a detection result indicating the detected subject area in the captured image to the detection area control unit 208.

[0022] In step S304, detection area control unit 208 determines a detection area of ​​the subject in the captured image based on the focus degree in each area in the captured image obtained in step S302 and the subject area in the captured image detected in step S303. Detection area control unit 208 generates detection area information indicating the detection area of ​​the subject based on the detected subject area in the captured image and the focus degree of the subject area, and outputs it together with the image.

[0023] 3, the processes of steps S302 and S303 may be performed in any order as long as they are completed before the process of step S304 is started. For example, the process of step S302 may be performed after the process of step S303, or the processes of steps S302 and S303 may be performed in parallel.

[0024] With reference to Fig. 4, the determination of the focus degree of a captured image will be described. Fig. 4(a) shows a captured image 410. Fig. 4(b) shows the focus degree in each area (each of two or more partial areas) in the captured area 420. The focus degree here indicates the reliability of the image in that area, and can be calculated based on, for example, a contrast value, distance information, a pattern matching degree, attribute information of the detected object, or a combination of these. The attribute information of the detected object includes, for example, information obtained based on at least one of the size, sex, age, color, shape, movement direction, and brightness of the detected object.

[0025] Fig. 4(b) shows an example of a method of dividing the shooting area into a predetermined size as a method of setting each area for determining the focus degree, and divides the shooting area 420 into a plurality of areas of a predetermined size to obtain the focus degree for each area. In Fig. 4(b), for convenience, the focus degree of each area is represented by a shade of color, and the darker the color, the lower the focus degree. When the focus degree of each area in the shooting area 420 is calculated as shown in Fig. 4(b), the detection area control unit 208 determines the detection area of ​​the subject based on the focus degree of the area in the captured image corresponding to the subject area detected by the subject detection unit 204.

[0026] As another example of a method for setting each region for determining the degree of focus, the degree of focus may be calculated for the subject region detected by the subject detection unit 204, as shown in FIG. 4(c). In the example shown in FIG. 4(c), the degree of focus in two subject regions 431 and 432 detected in the captured image 430 is calculated, and the degree of focus in the subject region 431 is displayed as 0.94, and the degree of focus in the subject region 432 is displayed as 0.63. In the example shown in FIG. 4(c), the value of the degree of focus is normalized, and the closer the value is to 1, the better the focus state is. For example, if a region with a degree of focus of 0.7 or less is not suitable for subsequent processing (for example, recognition processing), the threshold value of the degree of focus for determining the detection region of the subject is set to 0.7, and the detection region control unit 208 determines only the subject region 431 exceeding 0.7 as the detection region of the subject.

[0027] The imaging device in the first embodiment judges the image quality of the subject area by taking into account the focus level in each area in the captured image to the detection result of the subject to be detected in the captured image, and determines the subject area more suitable for the subsequent processing (for example, recognition processing) as the detection area of ​​the subject. In this way, instead of simply outputting information on the subject area where the subject is detected, it outputs highly accurate detection area information indicating the area of ​​the subject having appropriate image quality in the captured image, making it easier to detect a subject that is more suitable for authentication processing.

[0028] In the above-described configuration example, imaging device 200 has subject detection unit 204 and detection area control unit 208, but the present invention is not limited to this. For example, subject detection unit 204 and detection area control unit 208 may be provided outside imaging device 200 as subject detection device 210 capable of communicating with imaging device 200. Furthermore, the subject detection device may be provided with a functional unit that evaluates the focus level of the captured image in the same manner as focus level evaluation unit 205, and the subject detection device may evaluate the focus level related to the determination of the subject detection area.

[0029] Second embodiment In the above-described first embodiment, an example has been described in which the image quality in the subject area is determined taking into account the degree of focus in the captured image, and a subject area more suitable for subsequent processing (e.g., recognition processing) is selected as the subject detection area. When determining the subject detection area from within the subject area, it is assumed that there may be cases where it is useful to give priority to a certain subject area as a detection target.

[0030] For example, if there is an upper limit to the number of subjects that can be processed in the detection process or the subsequent recognition process, it may be necessary to give priority to a region that is less focused than other regions but is still recognizable.As shown in an example in Figure 5, if multiple people enter the shooting range moving from the right edge to the left edge, if processing is not started from the person on the left, that person may move out of the shooting range, so it is necessary to consider not only whether the image quality is good but also which region to give priority to processing.

[0031] In the second embodiment, an example will be described in which a subject detection area is determined in consideration of priority. In the second embodiment, a weighting adjustment of the priority regarding the determination of the subject detection area is performed by setting a weighting for the focus level evaluation unit. An example of a weighting factor for the priority is as follows: ·Direction of travel - Nearby positions of the focus object (surroundings) - Nearby position (distance) of the focus target ·position Possible reasons include:

[0032] In the case of the direction of travel, for example, the user sets a preferred direction of travel in advance, and among the subjects to be detected that move in that direction in the captured image, those with good image quality (those with a predetermined image quality or higher) are given priority as detection targets. For the position near the focus target, it is also possible to detect a highly focused object in the captured image and give priority to the surrounding objects as detection targets. Assumed use cases include detection and recognition of a target area in a crowd or objects around a target. The surroundings in this case can be assumed to be the same distance to the subject, or the object is close on a plane, or a concentric distance that combines both. It is also possible to weight the position in the captured image. For example, when considering the use of a conventional intrusion detection line on a captured image, it is also possible to consider a method of increasing the priority of a certain position on the captured image and decreasing the priority as the distance from that position increases. This method of changing the priority according to the distance can also be applied to the position near the focus target described above. As for the position information, it is also possible to set a priority around a position specified by the user and determine the detection area of ​​the subject.

[0033] The hardware configuration of the imaging device in the second embodiment is the same as that of the imaging device in the first embodiment, and therefore the description thereof will be omitted. Fig. 6 is a block diagram showing an example of the functional configuration of an imaging device to which a control device in the second embodiment is applied. In Fig. 6, components having the same functions as those shown in Fig. 2 are given the same reference numerals, and duplicated descriptions will be omitted. The imaging device 600 has a lens unit 201, an imaging sensor 202, a signal processing unit 203, a subject detection unit 204, a sensor control unit 206, a lens control unit 207, a detection area control unit 208, a focus evaluation unit 601, and a priority setting unit 602.

[0034] The focus level evaluation unit 601 evaluates the focus level of the captured image generated by the signal processing unit 203. Furthermore, the focus level evaluation unit 601 weights the focus level of the captured image based on priority information set by the priority setting unit 602, and outputs evaluation information related to the weighted focus level to the detection area control unit 208. The focus level evaluation unit 601 also performs various evaluations related to camera control such as brightness and focus level of the captured image, and based on the evaluation results, the sensor control unit 206 controls the imaging sensor 202 and the lens control unit 207 controls the lens unit 201.

[0035] The priority setting unit 602 sets priority information relating to the priority when determining the detection area of ​​the subject. As described above, the priority information may be, for example, the traveling direction, the position near the focus target, the position, etc.

[0036] In this embodiment, the focus level evaluation unit 601 weights the focus level of the captured image based on the set priority information, and outputs evaluation information related to the weighted focus level to the detection area control unit 208, but is not limited to this. For example, the focus level evaluation unit 601 may not weight the focus level of the captured image, and the detection area control unit 208 may receive the focus level of the captured image and information related to the weight based on the priority information from the focus level evaluation unit 601 to determine the weighted focus level.

[0037] FIG. 7 is a flowchart showing an example of a process related to subject detection by the imaging device according to the second embodiment. In step S701, the imaging device 600 captures video. In the video capture in step S701, an optical image formed by the lens unit 201 is photoelectrically converted by the imaging sensor 202 to generate an imaging signal, and the generated imaging signal is subjected to image processing and the like by the signal processing unit 203 to generate a captured image.

[0038] In step S702, the priority setting unit 602 sets priority information relating to the priority when determining the subject detection area in response to an input by the user or the like. In step S703, the focus level evaluation unit 601 weights the priority levels according to the priority information set by the priority setting unit 602 in step S702.

[0039] In step S704, the focus level evaluation unit 601 determines the focus level of the captured image generated in the video capture in step S701. In step S705, the subject detection unit 204 performs subject detection processing on the captured image generated by the video shooting in step S701, and detects a subject area in the captured image.

[0040] In step S706, the detection area control unit 208 determines the detection area of ​​the subject in the captured image based on the subject area in the captured image obtained by processing by the subject detection unit 204 and the focus degree and weighting of the subject area obtained by processing by the focus degree evaluation unit 601. The detection area control unit 208 outputs detection area information indicating the determined detection area of ​​the subject together with the image.

[0041] 7, the processes of steps S703, S704, and S705 may be completed before starting the process of step S706, and the order of execution of the processes of steps S703, S704, and S705 is arbitrary. For example, the process of step S704 may be performed after the process of step S705, or some or all of the processes of steps S703, S704, and S705 may be performed in parallel.

[0042] With reference to FIG. 8(a) and FIG. 8(b), an example of setting priority as described above will be described. FIG. 8(a) shows an example of a photographed image 810. A photographed image 810 shows a plurality of people, and when, for example, a user selects a subject 811 for this photographed image 810, a priority coefficient (weight coefficient) is set with the area as the center. FIG. 8(b) shows a schematic diagram of the state of priority for the photographed image. In FIG. 8(b), the priority is expressed by a shade of color, and the darker the color, the lower the priority, and the priority decreases as the distance from the selected subject 811 increases from the center. In this embodiment, the detection area of ​​the subject is determined by taking into consideration the priority coefficient (weight coefficient) and the degree of focus in the photographed image. Therefore, if there is a subject with a high degree of focus in a position away from the area where the selected subject 811 exists, it will be excluded if it has a low priority. In addition, it is also possible to set the subject to be excluded in the opposite way.

[0043] The imaging device in the second embodiment weights the focus level in each area in the captured image according to the priority, judges the image quality of the subject area in the captured image, and determines the subject area more suitable for the subsequent processing (e.g., recognition processing) as the subject detection area. In this way, as in the first embodiment, it becomes easier to detect a subject more suitable for the authentication processing. In addition, it becomes possible to set a priority area and determine the subject detection area, which leads to improved usability.

[0044] In the above-described configuration example, the imaging device 600 has the subject detection unit 204 and the detection area control unit 208, but the present invention is not limited to this. For example, the subject detection unit 204 and the detection area control unit 208 may be provided outside the imaging device 600 as a subject detection device 210 capable of communicating with the imaging device 600. Furthermore, the subject detection device may be provided with a function unit that evaluates the focus level for the captured image and a priority setting unit 602, and the subject detection device may perform processing related to the evaluation of the focus level for determining the detection area of ​​the subject.

[0045] Third embodiment In the first and second embodiments, the focal area is fixed, but in the third embodiment, an example will be described in which the detection process and the recognition process (authentication process) of the subject are performed while changing the focal area. In order to adjust the focal position, an imaging device (camera) has a flange back adjustment that changes the position of the focus lens of the lens unit or the imaging sensor. With such a focus adjustment mechanism, it is possible to improve the focusing conditions of the subject by adjusting the focal position for each subject at different distances. Therefore, it is possible to achieve a good focus state of the subject with respect to the detection target area.

[0046] For example, by acquiring distance information of the subject, the area (distance) to be detected can be selected and specified, and the subject can be detected in a good focusing state and a recognition process (authentication process) can be performed. The distance information of the subject can be obtained, for example, based on the position of the focus lens, or can be obtained by a phase difference sensor, a method using a phase difference in the sensor plane, or a range finder. By setting the target distance (focus position) in this way, the detection of the subject can be restricted to the detection target area, and unintended detection and recognition of the subject outside the target area can be reduced. In addition, when the focusing operation by focus adjustment can be performed at high speed or when the focus search range (shooting distance) is changed in sequence, the next focus target is determined after the recognition process at the focusing target position is completed. In this way, it is possible to make a judgment with high recognition accuracy.

[0047] 9, when subjects are lined up in the depth direction, focusing on a close subject 901 in front may result in the focus being shifted from a subject 902 in the depth direction. In such a case, by completing the recognition process on the subject in front, focusing on the next subject at a distance (here, the subject in the back) and performing recognition process on the target area of ​​that distance information, it becomes possible to make a judgment with improved recognition accuracy of the recognition target.

[0048] The hardware configuration of the imaging device in the third embodiment is the same as that of the imaging device in the first embodiment, and therefore the description thereof will be omitted. Fig. 10 is a block diagram showing an example of the functional configuration of an imaging device to which a control device in the third embodiment is applied. In Fig. 10, components having the same functions as those shown in Fig. 2 are given the same reference numerals, and duplicated descriptions will be omitted. The imaging device 1000 has a lens unit 201, an imaging sensor 202, a signal processing unit 203, a focus evaluation unit 205, a distance detection unit 1001, an imaging system driving unit 1002, and a recognition processing unit 1003.

[0049] The distance detection unit 1001 detects the distance to a subject present within the imaging range of the imaging device 1000. The distance detection unit 1001 detects the subject distance based on, for example, the focus state of the subject in the captured image and the position of the focus lens. The distance detection unit 1001 may also detect the subject distance using a phase difference sensor, a method using a phase difference within the sensor plane, a range finder, or the like.

[0050] The imaging system driving unit 1002 includes a sensor control unit 206 and a lens control unit 207, and drives an imaging system such as the lens unit 201 and the imaging sensor 202 based on the subject distance detected by the distance detection unit 1001. The imaging system driving unit 1002 is an example of an adjustment unit that adjusts the focus position when capturing an image. Based on input from the distance detection unit 1001 and the focus degree evaluation unit 205, the imaging system driving unit 1002 drives the imaging system to adjust the focus position, thereby changing the in-focus area in the captured image.

[0051] The recognition processing unit 1003 performs a detection process of a target subject in the captured image for the captured image generated by the signal processing unit 203. The recognition processing unit 1003 performs a detection process of a subject for each detection area in the captured image that is set based on the subject distance detected by the distance detection unit 1001 for the captured image generated by the signal processing unit 203. The recognition processing unit 1003 also performs a recognition process (authentication process) on the subject detected by the subject detection process.

[0052] FIG. 11 is a flowchart showing an example of processing by the imaging device in the third embodiment. In step S1101, the imaging device 1000 starts capturing a video. In the imaging device 1000, an optical image formed by the lens unit 201 is photoelectrically converted by the imaging sensor 202 to generate an imaging signal, and the generated imaging signal is subjected to image processing and the like by the signal processing unit 203 to generate a captured image.

[0053] In step S1102, the distance detection unit 1001 detects the distance (subject distance) of a subject that is present within the imaging range of the imaging device 1000 and is to be detected. In step S1103, the imaging system driver 1002 drives the imaging system to perform focus adjustment based on the subject distance detected in step S1102. The imaging system driver 1002 drives the imaging system to focus at a subject distance selected from the subject distances detected in step S1102. The adjustment amount of the focus position in this focus adjustment in step S1103 is set, for example, based on hyperfocal distance information obtained from the shooting conditions based on the detected subject distance.

[0054] In step S1104, the recognition processing unit 1003 sets a detection area for the subject in the captured image according to the selected subject distance. In step S1105, the recognition processing unit 1003 performs subject detection processing for the detection area in the captured image set in step S1104, and performs recognition processing (authentication processing) on ​​the subject detected in the detection processing.

[0055] After completing the process in step S1105, in step S1106, the imaging device 1000 determines whether there is a next object distance for which the object detection process and the object recognition process have not been performed. The imaging device 1000 determines whether there is an unselected object distance among the object distances detected in step S1102. If it is determined that there is a next object distance (YES in step S1106), the imaging device 1000 returns to step S1103, performs focus adjustment based on the next object distance, and executes the processes of steps S1104 and S1105. On the other hand, if it is determined that there is no next object distance, that is, if it is determined that the object detection process and the object recognition process have been performed for all the object distances detected in step S1102 (NO in step S1106), the imaging device 1000 ends the process shown in FIG. 11.

[0056] When there is no need to interrupt the detection and recognition processing of a subject related to a captured image, the imaging device 1000 can continue to perform detection and recognition in real time by repeatedly performing the processing of the flowchart shown in FIG.

[0057] According to the third embodiment, it is possible to detect a subject in a captured image by setting a well-focused area in the captured image as a target area for detection processing while changing the focus position based on the distance of the subject existing within the shooting range of the imaging device 1000. This makes it easier to detect a subject that is more suitable for authentication processing.

[0058] (Fourth embodiment) In the fourth embodiment, an example of determining a detection area of ​​a subject based on the camera settings of an imaging device when photographing will be described. The hardware configuration of the imaging device in the fourth embodiment is the same as that of the imaging device in the first embodiment, so the description thereof will be omitted. FIG. 12 is a block diagram showing an example of a functional configuration of an imaging device to which a control device in the fourth embodiment is applied. The imaging device 1200 has a lens unit 1201, an optical aperture 1202, an imaging sensor 1203, a gain application unit 1204, a signal processing unit 1205, and a subject detection unit 1206. The imaging device 1200 also has a focus evaluation unit 1207, a focus evaluation area setting unit 1208, a lens control unit 1209, an exposure evaluation unit 1210, an exposure evaluation area setting unit 1211, an exposure control unit 1212, and a detection area control unit 1213.

[0059] The lens unit 1201 includes a focus lens and a zoom lens, and collects light incident from a subject or the like. The optical aperture 1202 adjusts the amount of light incident on the image sensor 1203. The light incident on the image capturing device 1200 from a subject or the like passes through the lens unit 1201 and the optical aperture 1202 and forms an image on the imaging surface of the image sensor 1203.

[0060] The imaging sensor 1203 is an imaging element such as a CMOS sensor or a CCD sensor, and converts light into an electrical signal to generate an imaging signal. The gain application unit 1204 applies gains related to sensitivity and exposure to the imaging signal generated by the imaging sensor 1203. The signal processing unit 1205 performs various signal processing and image processing on the imaging signal output from the gain application unit 1204 to generate an image (captured image). Note that the signal processing unit 1205 may be configured with a dedicated circuit block (such as a video engine) for performing predetermined image processing and the like.

[0061] The subject detection unit 1206 performs a detection process of a subject to be detected in the captured image generated by the signal processing unit 1205, and outputs the detection result to the detection area control unit 1213. The detection result output by the subject detection unit 204 includes, for example, information indicating the detected subject area (for example, coordinate information for identifying the subject area). The subject detection method may be an existing publicly known method, such as a template matching method or a method of detecting a subject by deep learning. In this embodiment, as an example, the subject to be detected is a face, and the subject detection unit 1206 performs face detection.

[0062] The focus level evaluation unit 1207 evaluates the focus level of the captured image generated by the signal processing unit 1205. The focus level evaluation unit 1207 evaluates the focus level of an evaluation area (e.g., an evaluation frame) based on the setting of the evaluation area by the focus level evaluation area setting unit 1208. The focus level evaluation area setting unit 1208 sets an evaluation area for the focus level of the captured image for performing AF control (autofocus control) in the imaging device 1200. The lens control unit 1209 controls the lens unit 1201 based on the evaluation result by the focus level evaluation unit 1207.

[0063] The exposure evaluation unit 1210 evaluates the exposure in the captured image generated by the signal processing unit 1205. The exposure evaluation unit 1210 evaluates the exposure in an evaluation area (e.g., an evaluation frame) based on the setting of the evaluation area by the exposure evaluation area setting unit 1211. The exposure evaluation area setting unit 1210 sets an evaluation area for the exposure of the captured image for performing AE control (automatic exposure control) in the imaging device 1200. The exposure control unit 1212 controls the optical aperture 1202, the imaging sensor 1203, and the gain application unit 1204 based on the evaluation result by the exposure evaluation unit 1210.

[0064] The detection area control unit 1213 determines the detection area of ​​the subject based on the detection result of the subject in the captured image from the subject detection unit 1206 and the setting of the evaluation area by the focus evaluation area setting unit 1208 and the exposure evaluation area setting unit 1211. The detection area control unit 1213 outputs detection area information indicating the determined detection area of ​​the subject together with the image. For example, the detection area control unit 1213 determines the detection area of ​​the subject by preferentially selecting the subject areas indicated by the detection result from the vicinity of the set evaluation area so that the number of the selected subject areas does not exceed a predetermined number, and outputs the detection area information.

[0065] The imaging device in the fourth embodiment determines the subject detection area by preferentially selecting a detected subject area from the vicinity of the set evaluation area based on the camera settings when shooting, such as the setting of an evaluation area for AF control or the setting of an evaluation area for AE control. In this way, it is possible to preferentially determine the subject detection area from the vicinity of each evaluation area where the image quality is likely to be appropriate, and it becomes easier to detect a subject that is more suitable for authentication processing.

[0066] In the above-described configuration example, the imaging device 1200 has the subject detection unit 1206 and the detection area control unit 1213, but the present invention is not limited to this. For example, the subject detection unit 1206 and the detection area control unit 1213 may be provided outside the imaging device 1200 as a subject detection device 1220 capable of communicating with the imaging device 1200. The subject detection device may also have a configuration including a focus degree evaluation area setting unit 1208 and an exposure evaluation area setting unit 1211.

[0067] In the above-described embodiments, the subject to be detected is a person's face, but the present invention is not limited thereto. For example, the present invention can be applied to other subjects that can be detected as objects such as cars and objects. In addition, the present invention is not limited to imaging devices such as cameras having only a photographing function, but can also be applied to, for example, smartphones and tablet terminals having a photographing function. In the above-described embodiments, the present invention is mainly applied to an example in which the imaging device 100 executes the processes shown in the flowcharts shown in FIG. 7 and FIG. 11, but is not limited thereto. For example, a control device having each configuration other than the imaging unit 106 among the hardware shown in FIG. 1 may execute the processes of each flowchart. In addition, the control device may be connected to the imaging device 100 and execute lens control according to the evaluation result of the focus degree.

[0068] (Another embodiment of the present invention) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiment is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0069] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0070] The disclosure of the present embodiment includes the following configurations, methods, etc. (Configuration 1) A determination means for determining a focus degree of each of two or more partial regions in a captured image obtained by the photographing means; a determination means for determining a detection area of ​​a subject in an image captured by the imaging means based on the focus degree of each of the two or more partial areas determined by the determination means. (Configuration 2) 2. The control device according to configuration 1, wherein the determination means divides a photographing area into a plurality of partial areas of a predetermined size and determines a degree of focus of each partial area. (Configuration 3) A detection means for detecting a subject from an image, 2. The control device according to configuration 1, wherein the determining means determines a degree of focus of a partial area in the captured image that corresponds to the subject detected by the detecting means. (Configuration 4) a setting unit for setting a priority for the determination unit to determine a detection area of ​​the object; The control device according to any one of configurations 1 to 3, characterized in that the determination means determines a detection area of ​​a subject in the captured image based on the focus degree of each partial area determined by the determination means and the priority set by the setting means. (Configuration 5) The control device according to configuration 4, wherein the determination means determines a detection area of ​​the subject in the captured image based on a detection result of the subject in the captured image and a focus degree of each partial area in the captured image weighted based on the priority. (Configuration 6) The control device according to any one of configurations 1 to 5, wherein the determination means determines the degree of focus in the image based on at least one of a contrast value, distance information, a degree of pattern matching, and attribute information of the object to be detected. (Configuration 7) An adjustment means for adjusting a focus position during photographing by the photographing means, The control device according to configuration 1, characterized in that the adjustment means changes the focus position based on distance information of a subject detected from the captured image, and the determination means changes the detection area in accordance with the distance information of the subject. (Configuration 8) The control device according to configuration 7, wherein the adjustment means changes to a second focus position based on distance information of a second subject when detection processing of one or more subjects including the first subject in a first detection area according to distance information of the first subject is completed. (Configuration 9) 8. The control device according to configuration 7, wherein the amount of adjustment of the focus position by the adjustment means is set based on hyperfocal distance information obtained from shooting conditions on the basis of distance information of the subject. (Configuration 10) 2. The control device according to configuration 1, further comprising a detection area control means for determining a detection area of ​​the subject based on a camera setting when the photographing means photographs the subject. (Configuration 11) 11. The control device according to configuration 10, wherein the camera setting is at least one of a setting of an evaluation area related to autofocus control and a setting of an evaluation area related to automatic exposure control. (Configuration 12) 12. The control device according to any one of configurations 1 to 11, comprising the imaging means. (Configuration 13) 13. The control device according to any one of configurations 1 to 12, further comprising an authentication means for performing an authentication process on a subject detected from the subject detection area. (Method 1) A control method performed by a control device, determining a focus degree for each of two or more partial regions in a captured image obtained by the imaging means; A control method comprising determining a detection area of ​​a subject in a photographed image obtained by said photographing means based on the determined focus degree of each of said two or more partial areas. (Program 1) On the computer, determining a focus degree for each of two or more partial regions in a captured image obtained by the imaging means; A program for causing an object to be detected in a captured image obtained by the imaging means to be determined based on the degree of focus of each of the determined two or more partial regions. [Explanation of symbols]

[0071] 201: Lens unit 202: Imaging sensor 203: Signal processing unit 204: Subject detection unit 205, 601: Focus evaluation unit 206: Sensor control unit 207: Lens control unit 208: Detection area control unit 602: Priority setting unit 1001: Distance detection unit 1002: Imaging system drive unit 1003: Recognition processing unit

Claims

1. a determination means for determining a focus degree of each of two or more partial regions in a captured image obtained by the image capture means; a determination means for determining a detection area of ​​a subject in an image captured by the imaging means based on the focus degrees of each of the two or more partial areas determined by the determination means.

2. 2. The control device according to claim 1, wherein the determination means divides the photographing area into a plurality of partial areas of a predetermined size and determines the degree of focus of each partial area.

3. A detection means for detecting a subject from an image, 2. The control device according to claim 1, wherein the determining means determines a degree of focus of a partial area in the captured image that corresponds to the subject detected by the detecting means.

4. a setting unit for setting a priority for the determination unit to determine a detection area of ​​the object; The control device according to any one of claims 1 to 3, characterized in that the determination means determines a detection area of ​​a subject in the captured image based on the focus degree of each partial area determined by the determination means and the priority set by the setting means.

5. The control device according to claim 4, characterized in that the determination means determines the detection area of ​​the subject in the captured image based on the detection result of the subject in the captured image and the focus degree of each partial area in the captured image weighted based on the priority.

6. The control device according to any one of claims 1 to 3, characterized in that the determination means determines the degree of focus in the image based on at least one of a contrast value, distance information, a degree of pattern matching, and attribute information of the object to be detected.

7. An adjustment means for adjusting a focus position during photographing by the photographing means, 2. The control device according to claim 1, wherein the adjustment means changes the focus position based on distance information of a subject detected from the captured image, and the determination means changes the detection area in accordance with the distance information of the subject.

8. The control device according to claim 7, characterized in that the adjustment means changes to a second focus position based on distance information of a second subject when detection processing of one or more subjects including the first subject in a first detection area according to distance information of the first subject is completed.

9. 8. The control device according to claim 7, wherein the amount of adjustment of the focus position by said adjustment means is set based on hyperfocal distance information obtained from photographing conditions on the basis of distance information of the subject.

10. 2. The control device according to claim 1, further comprising a detection area control means for determining a detection area of ​​the subject based on a camera setting when the photographing means photographs the subject.

11. 11. The control device according to claim 10, wherein the camera setting is at least one of a setting of an evaluation area related to autofocus control and a setting of an evaluation area related to automatic exposure control.

12. 2. The control device according to claim 1, further comprising the photographing means.

13. 2. The control device according to claim 1, further comprising an authentication unit for performing an authentication process on the subject detected from the subject detection area.

14. A control method performed by a control device, determining a focus degree for each of two or more partial regions in a captured image obtained by the imaging means; A control method comprising determining a detection area of ​​a subject in a photographed image obtained by said photographing means based on the determined focus degrees of each of said two or more partial areas.

15. On the computer, determining a focus degree for each of two or more partial regions in a captured image obtained by the imaging means; A program for causing an object to be detected in an image captured by the imaging means to be determined based on the degree of focus of each of the determined two or more partial regions.

Images