Control device, control method, and program

Abstract

Provided is a control device capable of reducing processing time. An image acquisition unit (2) acquires an image that has been obtained by imaging the surrounding environment. A detection unit (4) performs a process for detecting a detection target from the acquired image. A calculation unit (6) calculates brightness using a portion of pixels that are in a region of the detection target detected from image and that have been specified on the basis of a prescribed condition. A determination unit (8) determines, on the basis of the calculated brightness, a control parameter to be used in an imaging device. A control unit (10) controls the imaging device to perform image capture using the determined control parameter.

Description

[Technical Field]

[0001] The present disclosure relates to a control device, a control method, and program Regarding. [Background technology]

[0002] Patent Document 1 discloses a face recognition device. The face recognition device uses one high-resolution camera and multiple low-resolution cameras to capture facial images of a person from different angles and detects a facial region from the high-resolution input image captured by the high-resolution camera. The face recognition device also controls the brightness (gain control, etc.) of the other low-resolution camera based on the pixel value distribution of the detected facial region. After brightness control, the face recognition device detects the facial region from the low-resolution input image captured by the low-resolution camera. The face recognition device also selects the image with the highest evaluation value from the facial region image obtained from the high-resolution input image and the facial region image obtained from the low-resolution input image, and performs face matching processing. In brightness control, the face recognition device also calculates the pixel value distribution of the facial region and sends a brightness control signal based on the calculated pixel value distribution to the low-resolution camera, thereby controlling the brightness of each imaging condition of the low-resolution camera, such as gain, shutter speed, and aperture. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-134593 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology disclosed in Patent Document 1, brightness is controlled using pixels in the entire face area that is the detection target. In this way, controlling brightness using pixels in the entire area of ​​the detection target (object) can result in a long processing time. In particular, the processing time increases when the size of the detection target, such as a face, in the image is large.

[0005] The object of the present disclosure is to solve such problems and to provide a control device, a control method, and a program that can reduce processing time. [Means for solving the problem]

[0006] The control device according to the present disclosure includes an image acquisition means for acquiring an image obtained by photographing the surrounding environment, a detection means for performing processing to detect a detection target from the acquired image, a calculation means for calculating brightness using a portion of pixels identified based on predetermined conditions in an area of ​​the detection target detected from the image, a determination means for determining control parameters to be used in the imaging device based on the calculated brightness, and a control means for controlling the imaging device to perform imaging using the determined control parameters.

[0007] In addition, the control method according to the present disclosure acquires an image obtained by photographing the surrounding environment, performs a process of detecting a detection target from the acquired image, calculates brightness using a portion of pixels identified based on predetermined conditions in the area of ​​the detection target detected from the image, determines control parameters to be used in an imaging device based on the calculated brightness, and controls the imaging device to perform imaging using the determined control parameters.

[0008] In addition, the program according to the present disclosure causes a computer to execute the steps of: acquiring an image obtained by photographing the surrounding environment; performing a process to detect a detection target from the acquired image; calculating brightness using a portion of pixels identified based on predetermined conditions in the area of ​​the detection target detected from the image; determining control parameters to be used in an imaging device based on the calculated brightness; and controlling the imaging device to perform imaging using the determined control parameters. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a control device, a control method, and a program that can reduce processing time. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an overview of a control device according to an embodiment of the present invention; [Figure 2] 3 is a flowchart showing an outline of a control method executed by the control device according to the present embodiment. [Figure 3] FIG. 1 is a diagram illustrating a configuration of a control system according to a first embodiment. [Figure 4] FIG. 2 is a diagram illustrating a configuration of a control device according to the first embodiment. [Figure 5] 4 is a flowchart illustrating a control method executed by the control device according to the first embodiment. [Figure 6] FIG. 4 is a diagram for explaining processing by a detection unit according to the first embodiment. [Figure 7] 10 is a flowchart showing a first modified example of the process of the brightness calculation unit according to the first embodiment. [Figure 8] 10 is a flowchart showing a second modified example of the process of the brightness calculation unit according to the first embodiment. [Figure 9] 10 is a flowchart showing a control method executed by the control device according to the second embodiment. [Figure 10] 10 is a flowchart showing a control method executed by the control device according to the third embodiment. [Figure 11] 10 is a flowchart showing a control method executed by the control device according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] (Outline of this embodiment) Prior to describing the present embodiment, an outline of the present embodiment will be described. Fig. 1 is a diagram showing an outline of a control device 1 according to the present embodiment. Fig. 2 is a flowchart showing an outline of a control method executed by the control device according to the present embodiment.

[0012] The control device 1 is, for example, a computer. The control device 1 has an image acquisition unit 2, a detection unit 4, a calculation unit 6, a determination unit 8, and a control unit 10. The image acquisition unit 2 functions as an image acquisition means. The detection unit 4 functions as a detection means. The calculation unit 6 functions as a calculation means. The determination unit 8 functions as a determination means. The control unit 10 functions as a control means.

[0013] The image acquisition unit 2 acquires an image obtained by capturing an image of the surrounding environment (step S12). The image may be captured by an imaging device described below. The image may be a moving image (video) or a still image. The image may also be a frame image that constitutes a video. In the following, the term "image" also means "image data representing an image" as a processing target in information processing.

[0014] The detection unit 4 performs a process of detecting a detection target from the acquired image (step S14). The detection target is, for example, a person's face, but is not limited to this. The detection target may be any object. The detection target may be the entire body of a person. Alternatively, the detection target may be a moving object other than a person, or may be a structure. The detection unit 4 may also detect one or more feature points that characterize the detection target in the area of ​​the detection target on the image.

[0015] The calculation unit 6 calculates brightness using a portion of pixels in the detection target area detected from the image that are identified based on a predetermined condition (step S16). Therefore, the calculation unit 6 does not calculate brightness using all of the pixels in the detection target area. An example of the "predetermined condition" will be described later.

[0016] The calculation unit 6 may calculate the brightness using pixels corresponding to one or more feature points in the detection target area that characterize the detection target. Furthermore, when a portion of the detection target is occluded, the calculation unit 6 may calculate the brightness using pixels corresponding to feature points other than those located in the occluded area. Furthermore, when a portion of the detection target is occluded, the calculation unit 6 may calculate the brightness by reducing the weight of the pixels corresponding to the feature points located in the occluded area and calculating a weighted average of the pixel values ​​of the pixels corresponding to the feature points. Furthermore, the calculation unit 6 may calculate the brightness by using pixels corresponding to feature points related to a part with a high priority that has been set in advance for the part of the detection target over pixels corresponding to feature points related to a part with a low priority. Furthermore, when the detection target is a face, for example, the parts constituting the detection target include the eyes, nose, mouth, eyebrows, etc.

[0017] The calculation unit 6 may also calculate the brightness using pixels in an unobstructed area of ​​the detection target area. The calculation unit 6 may also calculate the brightness using pixels located at predetermined intervals in the detection target area. The calculation unit 6 may also calculate the brightness using pixels in the detection target area that correspond to one or more parts that make up the detection target.

[0018] The determination unit 8 determines the control parameters to be used in the imaging device based on the calculated brightness (step S18). The control unit 10 controls the imaging device to perform imaging using the determined control parameters (step S20). The imaging device is, for example, a camera, but is not limited to this. Furthermore, the "control parameters" are, for example, exposure time and gain, but are not limited to these.

[0019] In this manner, in this embodiment, brightness is calculated using a portion of pixels identified based on predetermined conditions in the detection target area detected from the image. In other words, brightness is calculated by thinning out the pixels to be used rather than calculating brightness using all the pixels in the detection target area. Therefore, in this embodiment, it is possible to reduce processing time compared to when brightness is calculated using all the pixels in the detection target area. Note that the processing time can also be reduced by using a program that executes the above-described control method.

[0020] (Embodiment 1) Hereinafter, embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In addition, the same elements in each drawing are designated by the same reference numerals, and duplicate explanations have been omitted as necessary.

[0021] 3 is a diagram showing a configuration of a control system 20 according to the first embodiment. The control system 20 includes an imaging device 30 and a control device 100. The control device 100 is communicably connected via a wired or wireless network 22. The network 22 is, for example, a LAN (Local Area Network) or the Internet.

[0022] The imaging device 30 is, for example, a camera. The imaging device 30 captures an image of the surrounding environment. The imaging device 30 may be installed in a vehicle such as a passenger car. In this case, the imaging device 30 may capture an image of the environment outside (or inside) the vehicle.

[0023] The control device 100 corresponds to the control device 1 shown in FIG. 1. The control device 100 acquires an image by receiving the image from the imaging device 30. The control device 100 detects a detection target from the image and calculates brightness using a portion of pixels in the area of ​​the detected detection target that are identified based on predetermined conditions. The control device 100 also determines control parameters to be used by the imaging device 30 based on the calculated brightness, and controls the imaging device 30 so that an image is captured using the determined control parameters. This will be described in more detail below.

[0024] In the first embodiment, a person's face is detected as the detection target, but the detection target is not limited to a person's face. The control device 100 may perform face authentication using an image of a face detected from an image acquired from the imaging device 30. For example, the imaging device 30, which can capture images of the exterior of the vehicle, captures images of people around the vehicle. The control device 100 detects the person's face from the captured image and performs face authentication by comparing the image of the detected face with an image of the face of the owner of the vehicle (e.g., the driver, etc.). The control device 100 may unlock the vehicle if face authentication is successful.

[0025] Fig. 4 is a diagram showing the configuration of the control device 100 according to the first embodiment. As shown in Fig. 4, the control device 100 has, as its main hardware components, a control unit 102, a storage unit 104, a communication unit 106, and an interface unit 108 (IF; Interface). The control unit 102, the storage unit 104, the communication unit 106, and the interface unit 108 are connected to each other via a data bus or the like. Note that the imaging device 30 may also have the hardware configuration of the control device 100 shown in Fig. 4.

[0026] The control unit 102 is a processor such as a CPU (Central Processing Unit). The control unit 102 functions as an arithmetic device that performs control processing, arithmetic processing, etc. The control unit 102 may have multiple processors. The storage unit 104 is a storage device such as a memory or a hard disk. The storage unit 104 is, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 104 has a function for storing control programs, arithmetic programs, etc. executed by the control unit 102. In other words, the storage unit 104 (memory) stores one or more instructions. The storage unit 104 also has a function for temporarily storing processing data, etc. The storage unit 104 may include a database. The storage unit 104 may have multiple memories.

[0027] The communication unit 106 performs processing necessary for communicating with other devices, such as the imaging device 30, via a network. The communication unit 106 may include a communication port, a router, a firewall, etc. The interface unit 108 (IF; Interface) is, for example, a user interface (UI). The interface unit 108 has an input device, such as a keyboard, a touch panel, or a mouse, and an output device, such as a display or a speaker. The interface unit 108 may be configured such that the input device and the output device are integrated, such as a touch screen (touch panel). The interface unit 108 accepts data input operations by a user (operator) and outputs information to the user. For example, the interface unit 108 may output (display) determined control parameters.

[0028] The control device 100 according to the first embodiment includes, as components, an image acquisition unit 120, a detection unit 130, a brightness calculation unit 140, a control parameter determination unit 150, and a control unit 160. Note that the control device 100 does not need to be configured as a single physical device. In this case, the above-described components may be realized by a plurality of physically separate devices.

[0029] The image acquisition unit 120 corresponds to the image acquisition unit 2 shown in FIG. 1. The image acquisition unit 120 functions as an image acquisition means. The detection unit 130 corresponds to the detection unit 4 shown in FIG. 1. The detection unit 130 functions as a detection means. The brightness calculation unit 140 corresponds to the calculation unit 6 shown in FIG. 1. The brightness calculation unit 140 functions as a brightness calculation means (calculation means). The control parameter determination unit 150 corresponds to the determination unit 8 shown in FIG. 1. The control parameter determination unit 150 functions as a control parameter determination means (determination means). The control unit 160 corresponds to the control unit 160 shown in FIG. 1. The control unit 160 functions as a control means.

[0030] Each of the above-described components can be realized, for example, by executing a program under the control of the control unit 102. More specifically, each component can be realized by the control unit 102 executing a program (instructions) stored in the storage unit 104. Alternatively, each component may be realized by recording a necessary program on an arbitrary non-volatile recording medium and installing it as needed. Each component may not necessarily be realized by software programs, but may be realized by any combination of hardware, firmware, and software. Each component may also be realized using a user-programmable integrated circuit, such as an FPGA (field-programmable gate array) or a microcomputer. In this case, a program consisting of each of the above-described components may be realized using this integrated circuit. Specific functions of each component will be described later.

[0031] 5 is a flowchart showing a control method executed by the control device 100 according to the first embodiment. The image acquisition unit 120 acquires an image (step S102). Specifically, the image acquisition unit 120 acquires an image received from the imaging device 30 using the communication unit 106. The image acquisition unit 120 may acquire a frame image each time a frame image is generated by the imaging device 30. Alternatively, if the frame images from the imaging device 30 are temporarily stored in the storage unit 104 (buffer), the image acquisition unit 120 may acquire the frame images from the buffer. Note that if the imaging device 30 generates still images consecutively in time series, the image acquisition unit 120 may acquire the still images.

[0032] The detection unit 130 detects a face, which is the detection target, from the acquired image (step S104). Specifically, the detection unit 130 detects a face area (a rectangle including an image of the face) in the image. Then, the detection unit 130 detects feature points in the detected face area. At this time, the detection unit 130 may detect feature points by determining which pixels in the face area correspond to predetermined feature points. A feature point is a point that characterizes a face (detection target). For example, a feature point may be located at a position corresponding to a feature that characterizes a face. The feature points are, for example, the eyes, nose, mouth, and eyebrows. In other words, it is predefined which feature point belongs to which face part (eyes, nose, mouth, eyebrows, etc.).

[0033] FIG. 6 is a diagram illustrating the processing of the detection unit 130 according to the first embodiment. The detection unit 130 performs face detection processing on the acquired image Im to detect a face region R1, which is a rectangle including an image of a face, from the image Im. Next, the detection unit 130 performs face feature point detection processing on the face region R1. As a result, the detection unit 130 detects the positions of face feature points Pf corresponding to the eyes, nose, mouth corners, etc. As illustrated in FIG. 6, one or more feature points Pf may exist at positions corresponding to the eyes in the face region R1. Furthermore, one or more feature points Pf may exist at positions corresponding to the nose in the face region R1. Furthermore, one or more feature points Pf may exist at positions corresponding to the mouth in the face region R1. The face detection processing and face feature point detection processing may be performed using existing technology.

[0034] The detection unit 130 can detect the position of each part of the face (eyes, nose, mouth, etc.) by performing a facial feature point detection process. For example, the detection unit 130 may detect an area near a pixel corresponding to a feature point corresponding to each part as the area of ​​that part. For example, the detection unit 130 may detect an area near a pixel corresponding to a feature point corresponding to the eye as the area (position) of the eye. Furthermore, the detection unit 130 may perform a facial authentication process (matching process) using the position of each part in the face area. A feature point may correspond to one pixel. Alternatively, a feature point may correspond to multiple pixels. Alternatively, a feature point may correspond to all pixels in the image of the corresponding part.

[0035] The brightness calculation unit 140 calculates brightness using pixels corresponding to feature points (step S110). Here, in the first embodiment, the "predetermined condition" is the use of pixels corresponding to feature points in the region of the face (detection target). Specifically, the brightness calculation unit 140 calculates the average value of pixel values ​​of the pixel corresponding to the feature point and its surrounding pixels as brightness (brightness of the feature point). For example, the brightness calculation unit 140 may calculate the average value of pixel values ​​of all feature points, including one pixel corresponding to the feature point and eight adjacent pixels surrounding the pixel, for a total of nine pixels (pixels corresponding to feature points). When there are 14 feature points as in the example of FIG. 6, the brightness calculation unit 140 may calculate the average value of pixel values ​​of 9 × 14 pixels as brightness. Note that when pixels of neighboring feature points are adjacent to each other, there is a possibility that the surrounding pixels of these pixels overlap. In this case, the brightness calculation unit 140 does not need to use the pixel values ​​of the overlapping pixels in calculating brightness.

[0036] Furthermore, if the image is a grayscale image, the pixel value may be represented by a value between 0 and 255. If the image is a color image, the pixel value may be represented by a value between 0 and 255 for each of R, G, and B. In this case, the brightness calculation unit 140 may average the pixel values ​​of the feature point and its surrounding pixels for each of R, G, and B, and calculate the root mean square of the obtained average values ​​for each of R, G, and B as the brightness. Alternatively, the brightness calculation unit 140 may average the pixel values ​​of the feature point and its surrounding pixels without distinguishing between R, G, and B, and calculate the brightness as the average value.

[0037] The control parameter determination unit 150 determines the control parameters of the imaging device 30 based on the calculated brightness (step S122). Then, the control unit 160 controls the imaging device 30 so that the imaging device 30 captures an image using the determined control parameters (step S124). As a result, the imaging device 30 captures an image. Then, the process returns to S102, and the processes of S102 to S124 are repeated.

[0038] The control parameters are, for example, exposure time (shutter speed), gain, and iris. An example of determining the exposure time and gain as the control parameters will be described. First, ideal brightness (brightness of the feature point) is set as a predetermined brightness. If the calculated brightness is darker than the predetermined brightness, the control parameter determination unit 150 determines to extend the exposure time by a predetermined small time. Then, the control unit 160 controls the image capture device 30 to capture an image at the determined exposure time, so that the image capture device 30 captures an image (S124). Then, the processes of S102 to S110 are performed again. If the recalculated brightness is still darker than the predetermined brightness, the control parameter determination unit 150 determines to further extend the exposure time by a predetermined small time. After repeating this process, when the length of the exposure time reaches a predetermined upper limit, the control parameter determination unit 150 gradually increases the gain, similar to adjusting the length of the exposure time.

[0039] As described above, the control device 100 according to the first embodiment is configured to calculate brightness using pixels corresponding to feature points. This reduces the number of pixels used when calculating brightness compared to when brightness is calculated using all pixels included in the face area. Therefore, the control device 100 according to the first embodiment can reduce the processing time required for brightness calculation processing.

[0040] Note that if the size of the face area detected in the image is large, for example because the face is close to the imaging device 30, the number of pixels included in the face area will be large. Therefore, when calculating brightness using all pixels included in the face area, the larger the size of the face area detected in the image, the longer the processing time. In contrast, a certain number of feature points can be detected regardless of the size of the face area in the image. Therefore, even if the size of the face area detected in the image is large, the processing time required for the brightness calculation process can remain the same. Therefore, even if the size of the face area detected in the image is large, it is possible to reduce the processing time required for the brightness calculation process. Furthermore, feature points may exist on the image of the face in the face area. Therefore, by calculating brightness using pixels corresponding to feature points, it is possible to adjust the control parameters so that the face in the image is brighter with greater accuracy.

[0041] <Modification of the First Embodiment> The above description of the first embodiment does not mention the case where a part of the face is covered by a mask or the like. However, the first embodiment can also be applied to the case where a part of the face is covered by a mask or the like. This will be explained below. Note that in the above-mentioned face feature point detection process, even if parts related to feature points (eyes, nose, mouth, etc.) are covered, feature points corresponding to those parts can be detected.

[0042] 7 is a flowchart showing a first modified example of the process (S110) of the brightness calculation unit 140 according to the first embodiment. The brightness calculation unit 140 determines whether a part of a face is occluded in the detected face region (step S112A). The brightness calculation unit 140 determines, for example, whether the detected face region includes an image of a mask. The brightness calculation unit 140 also determines, for example, whether the detected face region includes an image of glasses (sunglasses). The determination of whether an image of a mask or glasses is included may be made using existing technology. For example, the determination may be made using a machine learning model such as a neural network that has learned images of a mask or glasses.

[0043] If it is determined that no part of the face is occluded (NO in S112A), the brightness calculation unit 140 calculates the brightness by calculating the average of the pixel values ​​of the pixels corresponding to all the feature points (step S114A). Specifically, similar to the process of S110 described above, the brightness calculation unit 140 may calculate, as the brightness, the average of the pixel values ​​of a total of nine pixels (pixels corresponding to the feature point) consisting of one pixel corresponding to the feature point and eight adjacent pixels surrounding that pixel.

[0044] On the other hand, if it is determined that part of the face is occluded (YES in S112A), the brightness calculation unit 140 calculates the brightness by calculating the average pixel values ​​of pixels corresponding to feature points other than those located in the occluded areas (step S116A). That is, when part of the face is occluded, the brightness calculation unit 140 calculates the brightness using pixels corresponding to feature points other than those located in the occluded areas. In other words, the brightness calculation unit 140 calculates the brightness excluding the feature points located in the occluded areas.

[0045] For example, when the mouth and nose are covered by a mask, the brightness calculation unit 140 calculates the brightness by calculating the average pixel values ​​of pixels corresponding to feature points other than the feature points related to the mouth and nose. In this case, the brightness calculation unit 140 may calculate the brightness using pixels corresponding to feature points related to the eyes and eyebrows. Furthermore, when the eyebrows are covered by glasses, the brightness calculation unit 140 calculates the brightness by calculating the average pixel values ​​of pixels corresponding to feature points other than the feature points related to the eyebrows. Furthermore, when the eyes are covered by sunglasses, the brightness calculation unit 140 calculates the brightness by calculating the average pixel values ​​of pixels corresponding to feature points other than the feature points related to the eyes.

[0046] In the first modification, when a part of the face is occluded, brightness is calculated using pixels corresponding to feature points other than those located in the occluded area, thereby excluding the feature points located in the occluded area. Here, since the occluded area is a part of the face area that is not actually photographed, the pixel value of the pixel at that position may not reflect the brightness of the face (brightness of the feature points). Therefore, with the above configuration, it is possible to adjust the control parameters so that the face in the image is brighter with higher accuracy.

[0047] 8 is a flowchart showing a second modified example of the process (S110) of the brightness calculation unit 140 according to the first embodiment. As in S112A, the brightness calculation unit 140 determines whether or not a part of the face is occluded in the detected face region (step S112B). If it is determined that a part of the face is not occluded (NO in S112B), the brightness calculation unit 140 calculates the brightness by calculating the average pixel values ​​of pixels corresponding to all feature points, as in S114A (step S114B).

[0048] On the other hand, if it is determined that a part of the face is occluded (YES in S112B), the brightness calculation unit 140 calculates the brightness by reducing the weight of the pixel corresponding to the feature point located in the occluded part. That is, the brightness calculation unit 140 calculates the brightness by reducing the weight of the pixel corresponding to the feature point located in the occluded part and calculating a weighted average of the pixel values ​​of the pixels corresponding to the feature point (step S116B). In other words, the brightness calculation unit 140 reduces the weight of the pixel corresponding to the feature point located in the occluded part compared to the weight of the pixel corresponding to the feature point located in the unoccluded part. Then, the brightness calculation unit 140 may calculate the brightness by calculating a weighted average of the pixel values ​​of the pixels corresponding to the feature point. That is, in the second modified example, the brightness calculation unit 140 calculates the brightness by reducing the weight of the pixel related to the feature point without excluding the feature point located in the occluded part.

[0049] For example, when the mouth and nose are covered by a mask, the brightness calculation unit 140 may calculate brightness by weighting pixels corresponding to feature points related to the mouth and nose less than the weighting pixels corresponding to feature points related to the eyes and eyebrows, and calculating a weighted average of pixel values. When the eyebrows are covered by glasses, the brightness calculation unit 140 may calculate brightness by weighting pixels corresponding to feature points related to the eyebrows less than the weighting pixels corresponding to feature points related to the eyes, nose, and mouth, and calculating a weighted average of pixel values. When the eyes are covered by sunglasses, the brightness calculation unit 140 may calculate brightness by weighting pixels corresponding to feature points related to the eyes less than the weighting pixels corresponding to feature points related to the eyebrows, nose, and mouth, and calculating a weighted average of pixel values.

[0050] In the second modified example, when a part of the face is occluded, the weight of the pixel corresponding to the feature point located in the occluded part is reduced, and the brightness is calculated by calculating the weighted average of the pixel values ​​of the pixels corresponding to the feature point. Here, the occluded part is a part of the face area that is not actually photographed, but the pixel value of the pixel at that position may not completely reflect the brightness of the face (brightness of the feature point). Therefore, with the above configuration, it is possible to adjust the control parameters so that the face in the image is brighter with greater accuracy.

[0051] (Embodiment 2) Next, a second embodiment will be described. The configurations of the control system 20 and the control device 100 according to the second embodiment are substantially the same as those according to the first embodiment, and therefore description thereof will be omitted. In the second embodiment, the predetermined condition for identifying a part of pixels used when calculating brightness is different from that in the first embodiment.

[0052] Specifically, in the second embodiment, the "predetermined condition" is that pixels corresponding to feature points related to a region with a high priority are used preferentially over pixels corresponding to feature points related to a region with a low priority, where the priority is set in advance for each region.

[0053] 9 is a flowchart showing a control method executed by the control device 100 according to the second embodiment. The image acquisition unit 120 acquires an image in the same manner as in S102 (step S202). The detection unit 130 detects a face, which is a detection target, from the acquired image in the same manner as in S104 (step S204).

[0054] The brightness calculation unit 140 calculates the brightness by preferentially using pixels corresponding to feature points of parts with high priority (step S210). That is, the brightness calculation unit 140 calculates the brightness by preferentially using pixels corresponding to feature points related to parts with high priority that are set in advance for parts of the face (detection target) over pixels corresponding to feature points related to parts with low priority.

[0055] For example, assume that the priority of the mouth is "high," the priority of the nose is "medium," and the priority of the eyes is "low." In this case, the brightness calculation unit 140 may calculate the brightness using pixels corresponding to feature points related to the mouth and nose, which are parts other than the eyes that have the lowest priority. That is, the brightness calculation unit 140 may calculate the brightness using pixel values ​​of pixels corresponding to feature points related to the mouth and nose, using a method substantially similar to the method of S110 described above. Note that if there is an upper limit on the number of feature points used in brightness calculation, the brightness calculation unit 140 may perform such processing when the number of detected feature points exceeds the upper limit.

[0056] Alternatively, the brightness calculation unit 140 may calculate the brightness using pixels corresponding to feature points related to the mouth that have the highest priority. That is, the brightness calculation unit 140 may calculate the brightness using pixel values ​​of pixels corresponding to feature points related to the mouth using a method substantially similar to the method of S110 described above. Note that if there is an upper limit on the number of feature points used in brightness calculation, and the number of detected feature points exceeds the upper limit, the brightness calculation unit 140 may perform such processing.

[0057] Alternatively, the brightness calculation unit 140 may calculate brightness by setting weights for pixels corresponding to feature points corresponding to each part according to the priority and calculating a weighted average of the pixel values ​​of the pixels corresponding to the feature points. In other words, the brightness calculation unit 140 may calculate brightness by setting a weight for pixels corresponding to feature points related to parts with high priority higher than a weight for pixels corresponding to feature points related to parts with low priority and calculating a weighted average of the pixel values ​​of pixels corresponding to the feature points.

[0058] The control parameter determination unit 150 determines the control parameters of the image capture device 30 based on the calculated brightness, similar to S122 (step S222). Then, the control unit 160 controls the image capture device 30 so that the image capture device 30 captures an image using the determined control parameters, similar to S124 (step S224). This causes the image capture device 30 to capture an image. Then, the process returns to S202, and the processes of S202 to S224 are repeated.

[0059] As described above, the control device 100 according to the second embodiment is configured to calculate brightness by preferentially using pixels corresponding to feature points related to a part having a high priority that is preset for the part to be detected. This allows pixels related to feature points of more distinctive parts to be used preferentially. That is, since eyes do not vary much between individuals, the priority of eyes may be low. Furthermore, since mouths and noses (and eyebrows) vary greatly between individuals, the priority of mouths and noses (and eyebrows) may be high. This prevents the use of pixels corresponding to feature points related to less distinctive parts, thereby further reducing processing time. Furthermore, by preferentially using pixels related to feature points of more distinctive parts, it is possible to adjust control parameters so that more distinctive parts in the image are brighter. Therefore, it is possible to adjust control parameters so that the face area in the image is brighter.

[0060] (Embodiment 3) Next, a third embodiment will be described. The configurations of the control system 20 and the control device 100 according to the third embodiment are substantially the same as those according to the first embodiment, and therefore will not be described again. In the third embodiment, the predetermined condition for identifying a portion of pixels used when calculating brightness is different from that of the above-described embodiments. Specifically, in the third embodiment, the "predetermined condition" is to use pixels in an unobstructed area.

[0061] 10 is a flowchart showing a control method executed by the control device 100 according to the third embodiment. The image acquisition unit 120 acquires an image in the same manner as in S102 etc. (step S302). The detection unit 130 detects a face, which is a detection target, from the acquired image in the same manner as in S104 etc. (step S304). Here, in the third embodiment, the detection unit 130 does not need to detect feature points.

[0062] The brightness calculation unit 140 calculates the brightness using pixels in the unoccluded region (step S310). Specifically, the brightness calculation unit 140 detects an occluded region (an occluded region) in the face region. For example, as described above, the brightness calculation unit 140 may determine whether the detected face region includes an image of a mask or glasses (sunglasses), etc. Then, the brightness calculation unit 140 may calculate the brightness using pixels in the face region other than the occluded region, with the image of the mask or glasses (sunglasses), etc., being the occluded region. In this case, the brightness calculation unit 140 may calculate the brightness using all pixels in the unoccluded region of the face region using a method substantially similar to the method of S110 described above. In other words, the brightness calculation unit 140 may calculate the brightness as the average value of the pixel values ​​of all pixels in the unoccluded region of the face region.

[0063] If an occluded area is not detected, the brightness calculation unit 140 may calculate the brightness using all pixels in the face area. Alternatively, if an occluded area is not detected, the brightness calculation unit 140 may perform the process of S110 in the first embodiment described above. In other words, the brightness calculation unit 140 may calculate the brightness using pixels corresponding to pixels related to feature points.

[0064] The control parameter determination unit 150 determines the control parameters of the image capture device 30 based on the calculated brightness, similar to S122 etc. (step S322). Then, the control unit 160 controls the image capture device 30 so that the image capture device 30 captures an image using the determined control parameters, similar to S124 etc. (step S324). This causes the image capture device 30 to capture an image. Then, the process returns to S302, and the processes of S302 to S324 are repeated.

[0065] As described above, the control device 100 according to the third embodiment is configured to calculate brightness using pixels in an unobstructed area of ​​the detection target area (face area). This allows brightness to be calculated excluding pixels in the obscured area, thereby reducing processing time compared to calculating brightness using all pixels in the face area. Here, because the obscured area is a part of the face area that is not actually photographed, the pixel values ​​of the pixels at that position may not reflect the brightness of the face. Therefore, with the above configuration, it becomes possible to adjust the control parameters so that the face in the image is brighter with greater accuracy.

[0066] In addition, if the shape of the face region is rectangular, the face region may also include the background image surrounding the face image. Therefore, in the third embodiment, there is a possibility that brightness is calculated using pixels corresponding to the background image. Therefore, there is a risk that the control parameters cannot be adjusted accurately so that the face in the image becomes brighter. On the other hand, as described above, by calculating brightness using pixels corresponding to feature points as in the first embodiment, it is possible to adjust the control parameters more accurately so that the face in the image becomes brighter than in the third embodiment.

[0067] (Fourth embodiment) Next, a fourth embodiment will be described. The configurations of the control system 20 and the control device 100 according to the fourth embodiment are substantially the same as those according to the first embodiment, and therefore will not be described again. In the fourth embodiment, the predetermined condition for identifying some pixels used when calculating brightness is different from that of the above-described embodiments. Specifically, in the fourth embodiment, the "predetermined condition" is to use pixels located at predetermined intervals.

[0068] 11 is a flowchart showing a control method executed by the control device 100 according to the fourth embodiment. The image acquisition unit 120 acquires an image (step S402), similar to S102 etc. The detection unit 130 detects a face, which is a detection target, from the acquired image (step S404), similar to S104 etc. Here, in the fourth embodiment, the detection unit 130 does not need to detect feature points.

[0069] The brightness calculation unit 140 calculates the brightness using pixels located at predetermined intervals in the face region (step S410). Specifically, the brightness calculation unit 140 calculates the brightness using pixels located at predetermined intervals among the pixels in the face region. In this case, the brightness calculation unit 140 may calculate the brightness using a method substantially similar to the method of S110 described above. In other words, the brightness calculation unit 140 may calculate the brightness as the average value of the pixel values ​​of the pixels located at predetermined intervals in the face region.

[0070] For example, the brightness calculation unit 140 may calculate the brightness using pixels located at a predetermined interval among the pixels in the face region. For example, if the interval is predetermined to be "a length of five pixels," the brightness calculation unit 140 calculates the brightness using images located at intervals of five pixels. Alternatively, the brightness calculation unit 140 may determine the predetermined interval so that the number of pixels used is a predetermined number. Then, the brightness calculation unit 140 may calculate the brightness using pixels located at the determined predetermined interval.

[0071] The control parameter determination unit 150 determines the control parameters of the image capture device 30 based on the calculated brightness, similar to S122 etc. (step S322). Then, the control unit 160 controls the image capture device 30 so that the image capture device 30 captures an image using the determined control parameters, similar to S124 etc. (step S324). This causes the image capture device 30 to capture an image. Then, the process returns to S302, and the processes of S302 to S324 are repeated.

[0072] As described above, the control device 100 according to the fourth embodiment is configured to calculate brightness using pixels located at predetermined intervals in the detection target area (face area). This makes it possible to reduce processing time compared to when brightness is calculated using all pixels in the face area. Furthermore, compared to the above-described embodiments, processing such as feature point detection or occluded area detection is not required, and therefore the processing content can be simplified.

[0073] In addition, if the shape of the face region is rectangular, the face region may also include the background image surrounding the face image. Therefore, in the fourth embodiment, there is a possibility that brightness is calculated using pixels corresponding to the background image. Therefore, there is a risk that the control parameters cannot be adjusted accurately so that the face in the image becomes brighter. On the other hand, as described above, by calculating brightness using pixels corresponding to feature points as in the first embodiment, it becomes possible to adjust the control parameters more accurately so that the face in the image becomes brighter than in the fourth embodiment.

[0074] (Variation) It should be noted that the present embodiment is not limited to the above embodiment, and can be appropriately modified within the scope of the spirit thereof. For example, the above-described multiple embodiments can be mutually applied. Furthermore, for example, the order of each process in the flowchart shown in FIG. 5 etc. can be appropriately changed. Furthermore, one or more of the processes in the flowchart shown in FIG. 5 etc. can be omitted.

[0075] Furthermore, in the above-described embodiment, the detection target is a face, but the detection target does not have to be a face. The detection target may be any object present in the environment. When the detection target is an object other than a face, feature points (key points) may be detected by a technique used in image recognition processing, such as SIFT (Scale-Invariant Feature Transform) or SURF (Speed-Up Robust Features).

[0076] Furthermore, in the first embodiment and the like, the brightness calculation unit 140 calculates brightness using pixels in the area of ​​the detection target that correspond to one or more feature points that characterize the detection target, but this is not limited to this. The brightness calculation unit 140 may calculate brightness using pixels in the area of ​​the detection target that correspond to one or more parts (constituent parts) that make up the detection target, rather than feature points. These constituent parts may be parts that characterize the detection target.

[0077] In this case, the detection unit 130 may detect the positions (areas) of the component parts from the area of ​​the detection target. Furthermore, the brightness calculation unit 140 may calculate the brightness using all (or some) of the pixels corresponding to the component parts. For example, if the detection target is a face, the component parts may be the eyes, nose, mouth, eyebrows, etc. The brightness calculation unit 140 may calculate the brightness using pixels corresponding to the eyes, nose, mouth, eyebrows, etc. The brightness calculation unit 140 may then calculate the brightness using all (or some) of the pixels corresponding to the eyes, nose, mouth, eyebrows, etc. By calculating the brightness using pixels corresponding to the component parts in this way, the number of pixels used in calculating the brightness is reduced compared to when the brightness is calculated using all the pixels included in the area of ​​the detection target. Therefore, similar to the above-described embodiment, it is possible to reduce the processing time.

[0078] On the other hand, if the area of ​​the detection target relative to the entire image is large, the size of the area of ​​the above-mentioned part may also be large. Therefore, the larger the area of ​​the detection target detected in the image, the longer the processing time. In contrast, as described above, a certain number of feature points can be detected regardless of the size of the face area in the image. Therefore, by calculating brightness using pixels corresponding to feature points as in the first embodiment, it is possible to reduce the processing time required for brightness calculation processing regardless of the size of the area of ​​the detection target detected in the image.

[0079] The above-mentioned program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0080] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) image acquisition means for acquiring an image obtained by photographing the surrounding environment; a detection means for performing a process of detecting a detection target from the acquired image; a calculation means for calculating brightness using a portion of pixels identified based on a predetermined condition in the region of the detection target detected from the image; a determining means for determining a control parameter to be used in the imaging device based on the calculated brightness; a control means for controlling the imaging device so as to perform imaging using the determined control parameters; A control device having: (Appendix 2) the calculation means calculates brightness using pixels in the region of the detection target that correspond to one or more parts that constitute the detection target; 10. The control device of claim 1. (Appendix 3) the calculation means calculates brightness using pixels in the region of the detection target that correspond to one or more feature points that characterize the detection target; 3. The control device according to claim 1 or 2. (Appendix 4) when a part of the detection target is occluded, the calculation means calculates brightness using pixels corresponding to the feature points other than the feature points located in the occluded part of the detection target. 4. The control device according to claim 3. (Appendix 5) when a part of the detection target is occluded, the calculation means calculates brightness by setting a weight for a pixel corresponding to the feature point located in the occluded part of the detection target to be smaller than a weight for a pixel corresponding to the feature point located in the unoccluded part, and calculating a weighted average of pixel values ​​of the pixels corresponding to the feature point. 4. The control device according to claim 3. (Appendix 6) the calculation means calculates brightness by preferentially using pixels corresponding to the feature points of a part having a high priority that is preset with respect to the part to be detected over pixels corresponding to the feature points of a part having a low priority. 4. The control device according to claim 3. (Appendix 7) the calculation means calculates brightness using pixels in an unobstructed area of ​​the detection target area. 10. The control device of claim 1. (Appendix 8) the calculation means calculates brightness using pixels positioned at predetermined intervals in the detection target area. 10. The control device of claim 1. (Appendix 9) The detection target is a human face. 9. The control device according to any one of claims 1 to 8. (Appendix 10) Acquire images of the surrounding environment, performing a process of detecting a detection target from the acquired image; calculating brightness using a portion of pixels identified based on a predetermined condition in the region of the detection target detected from the image; determining a control parameter to be used in the imaging device based on the calculated brightness; controlling the imaging device so as to perform imaging using the determined control parameters; Control method. (Appendix 11) Calculating brightness using pixels in the region of the detection target that correspond to one or more parts that constitute the detection target; 11. The control method of claim 10. (Appendix 12) calculating brightness using pixels in the region of the detection target that correspond to one or more feature points that characterize the detection target; 12. The control method according to claim 10 or 11. (Appendix 13) When a part of the detection target is occluded, brightness is calculated using pixels corresponding to the feature points other than the feature points located in the occluded part of the detection target. 13. The control method of claim 12. (Appendix 14) When a part of the detection target is occluded, the weight of the pixel corresponding to the feature point located in the occluded part of the detection target is made smaller than the weight of the pixel corresponding to the feature point located in the unoccluded part, and brightness is calculated by calculating a weighted average of the pixel values ​​of the pixels corresponding to the feature point. 13. The control method of claim 12. (Appendix 15) calculating brightness by using pixels corresponding to the feature points of a part having a high priority that is set in advance for the part to be detected, in preference to pixels corresponding to the feature points of a part having a low priority; 13. The control method of claim 12. (Appendix 16) Calculating brightness using pixels in an unobstructed area of ​​the detection target area; 11. The control method of claim 10. (Appendix 17) Calculating brightness using pixels positioned at predetermined intervals in the detection target region; 11. The control method of claim 10. (Appendix 18) The detection target is a human face. 18. The control method according to any one of appendices 10 to 17. (Appendix 19) acquiring an image of the surrounding environment; performing a process for detecting a detection target from the acquired image; calculating brightness using a portion of pixels identified based on a predetermined condition in the region of the detection target detected from the image; determining control parameters to be used in the imaging device based on the calculated brightness; controlling the imaging device to perform imaging using the determined control parameters; A non-transitory computer-readable medium storing a program that causes a computer to execute the program. [Explanation of symbols]

[0081] 1. Control device 2. Image acquisition unit 4. Detection unit 6 Calculation section 8 Decision Section 10 Control Unit 20 Control System 22 Network 30 Imaging device 100 control device 120 Image acquisition unit 130 Detector 140 Brightness calculation unit 150 Control parameter determination unit 160 control section

Claims

1. image acquisition means for acquiring an image obtained by photographing the surrounding environment; a detection means for performing a process of detecting a detection target from the acquired image; a calculation means for calculating brightness using pixels corresponding to one or more feature points that characterize the detection target in a region of the detection target detected from the image, and for calculating brightness when a portion of the detection target is occluded by making a weight for pixels corresponding to the feature points located in the occluded portion of the detection target smaller than a weight for pixels corresponding to the feature points located in the unoccluded portion, thereby calculating a weighted average of pixel values ​​of pixels corresponding to the feature points; a determining means for determining a control parameter to be used in the imaging device based on the calculated brightness; a control means for controlling the imaging device so as to perform imaging using the determined control parameters; A control device having:

2. the calculation means calculates brightness using pixels in the region of the detection target that correspond to one or more parts that constitute the detection target; The control device according to claim 1 .

3. the calculation means calculates brightness by preferentially using pixels corresponding to the feature points of a part having a high priority that is preset with respect to the part to be detected over pixels corresponding to the feature points of a part having a low priority. The control device according to claim 1 .

4. the calculation means calculates brightness using pixels positioned at predetermined intervals in the detection target area. The control device according to claim 1 .

5. Acquire images of the surrounding environment, performing a process of detecting a detection target from the acquired image; calculating brightness using pixels corresponding to one or more feature points that characterize the detection target in the region of the detection target detected from the image, and when a part of the detection target is occluded, calculating a weighted average of pixel values ​​of pixels corresponding to the feature points by making the weight of pixels corresponding to the feature points located in the occluded part of the detection target smaller than the weight of pixels corresponding to the feature points located in the unoccluded part, thereby calculating brightness; determining a control parameter to be used in the imaging device based on the calculated brightness; controlling the imaging device so as to perform imaging using the determined control parameters; Control method.

6. acquiring an image of the surrounding environment; performing a process for detecting a detection target from the acquired image; a step of calculating brightness using pixels corresponding to one or more feature points that characterize the detection target in the region of the detection target detected from the image, and when a part of the detection target is occluded, calculating a weighted average of pixel values ​​of pixels corresponding to the feature points by making the weight of pixels corresponding to the feature points located in the occluded part of the detection target smaller than the weight of pixels corresponding to the feature points located in the unoccluded part; determining control parameters to be used in the imaging device based on the calculated brightness; controlling the imaging device to perform imaging using the determined control parameters; A program that causes a computer to execute the following.

Images