Imaging control device, imaging control method, and program

Abstract

The present imaging apparatus includes an exposure control unit that performs exposure control based on information of a region of interest detected from an image captured by a region of interest detection unit. If the region of interest is detected, a region of interest exposure target value calculated based on a region of interest metering value obtained from a first metering region including at least the region of interest in an input frame is set as an exposure control value. If the region of interest is not detected, a setting region exposure target value calculated based on a setting region metering value obtained from a second metering region predetermined in the input frame is set as the exposure control value. In a predetermined period during which the region of interest is continuously not detected from a time of region of interest loss in which the region of interest is not detected, an exposure correction value is calculated based on the region of interest exposure target value before the time of region of interest loss, and exposure control is performed with an exposure control value obtained by correcting the setting region exposure target value with the exposure correction value.

Description

TECHNICAL FIELD

[0001] The present technology relates to an imaging apparatus and to settings for imaging. BACKGROUND

[0002] A technology that performs various imaging-related processing, such as exposure control on an image captured by an imaging apparatus, is known.

[0003] LIST OF CITATIONS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Laid-Open No. 2018-33013 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] Here, as exposure control in an imaging apparatus, there is a face auto exposure (AE) technology in which exposure control is performed so that a face region of a subject has appropriate luminance.

[0008] However, in face AE, control is performed assuming that a face is detected, so when a face is not detected due to the subject facing sideways or other reasons, appropriate exposure control can not be performed.

[0009] Therefore, the present disclosure proposes a technology for improving the exposure control function of an imaging apparatus.

[0010] SOLUTION TO PROBLEM

[0011] An imaging control apparatus according to the present technology includes an exposure control unit configured to perform exposure control based on information of a region of interest detected from a captured image by a region of interest detection unit, wherein the exposure control unit is configured to: in a case where the region of interest is detected from the captured image, perform exposure control in which a region of interest exposure target value calculated based on a region of interest photometry value obtained from a first photometry region including at least the region of interest in an input frame is set as an exposure control value; in a case where the region of interest is not detected from the captured image, perform exposure control in which a setting region exposure target value calculated based on a setting region photometry value obtained from a second photometry region predetermined in advance is set as the exposure control value; and in a predetermined period during which the region of interest is not detected continuously from a time of loss of the region of interest in which the region of interest is not detected from the captured image, obtain an exposure correction value based on a region of interest exposure target value before the time of loss of the region of interest, and then perform exposure control with an exposure control value obtained by correcting the setting region exposure target value with the exposure correction value.

[0012] The region of interest referred to here is a region in the captured image in which an object that is predefined as a target to be focused on is present. Examples of the object that is a target in the region of interest include independent objects such as a person, an animal, a plant, a car, a train, an airplane, and furniture, and specific portions in the independent objects, for example, a face or a hand of a person, a license plate of a car, and the like. According to the above-described configuration, for example, in a case where exposure control is performed so that the region of interest has an appropriate brightness in face auto exposure (face AE), when the region of interest is lost, for a predetermined period of time after the loss of the region of interest, exposure control is performed using an exposure control value obtained by correcting a set region exposure target value with an exposure correction value obtained using a region of interest exposure target value before the time at which the region of interest is lost. Since the exposure control value obtained by correcting the set region exposure target value with the exposure correction value using the region of interest exposure target value before the time at which the region of interest is lost is used as the exposure control value for the predetermined period of time after the loss of the region of interest, when the brightness of the entire composition does not change for the predetermined period of time, the brightness of the subject present in the region of interest can be appropriately maintained even after the loss of the region of interest, and even when the brightness of the entire composition changes for the predetermined period of time, it is possible to prevent a case in which the change in the brightness of the entire composition cannot be followed as in a case where AE locking is performed using the region of interest exposure target value before the time at which the region of interest is lost.

[0013] In the above-described imaging control device according to the present technology, the exposure correction value can be configured to be a value indicating a relationship between the set region exposure target value before the time at which the region of interest is lost and the region of interest exposure target value before the time at which the region of interest is lost.

[0014] Accordingly, for the predetermined period of time after the loss of the region of interest, the set region exposure target value can be corrected with the exposure target value to reproduce the relationship between the set region exposure target value before the time at which the region of interest is lost and the region of interest exposure target value before the time at which the region of interest is lost.

[0015] In the above-described imaging control device according to the present technology, for the predetermined period of time, the exposure control unit can be configured to perform exposure control using the obtained exposure correction value so that a specific numerical relationship between the set region exposure target value and the set region exposure target value corrected with the exposure correction value is equivalent to a specific numerical relationship between the set region exposure target value before the time at which the region of interest is lost and the region of interest exposure target value before the time at which the region of interest is lost.

[0016] The "specific numerical relationship" referred to herein means a specific numerical relationship among numerical relationships between two numerical values, and the "numerical relationship" means a numerical relationship such as a difference between two numerical values, a ratio between two numerical values, or the like. According to the above-described configuration, the exposure control is performed so that a specific numerical relationship such as a difference or a ratio between the set region exposure target value and the set region exposure target value corrected with the exposure correction value is equivalent to a specific numerical relationship between the set region exposure target value before the time of the loss of the region of interest and the region of interest exposure target value before the time of the loss of the region of interest (so as to reproduce the relationship) in a predetermined period after the loss of the region of interest.

[0017] In the above-described imaging control apparatus according to the present technology, the exposure correction value can be configured as a difference between the set region exposure target value before the time of the loss of the region of interest and the region of interest exposure target value before the time of the loss of the region of interest.

[0018] Accordingly, the exposure control is performed so that a difference between the set region exposure target value and the set region exposure target value corrected with the exposure correction value is equivalent to a difference between the set region exposure target value before the time of the loss of the region of interest and the region of interest exposure target value before the time of the loss of the region of interest in a predetermined period after the loss of the region of interest.

[0019] In the above-described imaging control apparatus according to the present technology, in a case where the region of interest is detected inside and outside the specific region in the captured image by the region of interest detection unit, the exposure control unit can be configured to calculate the region of interest exposure target value used for calculating the exposure correction value based on at least one of a size of the detected region of interest, a number of the detected region of interest, or a position of the detected region of interest.

[0020] Regarding the region of interest detected outside the specific region, in a case where the region size is small, the number of regions is small, or the distance from the region of interest detected inside the specific region is long, there is a possibility that the luminance of the subject in the region of interest detected inside the specific region cannot be made appropriate when the exposure target value of the region of interest is considered in calculating the exposure correction value.

[0021] In the above-described imaging control apparatus according to the present technology, in a case where the size of the region of interest detected outside the specific region is greater than a reference size, the exposure control unit can be configured to calculate the exposure correction value using the region of interest exposure target value calculated for the region of interest, and in a case where the size of the region of interest detected outside the specific region is not greater than the reference size, the exposure control unit can be configured to calculate the exposure correction value without using the region of interest exposure target value calculated for the region of interest.

[0022] Accordingly, regarding regions of interest detected outside a specific area, if the size of the region of interest is large and cannot be ignored in the correction of the exposure target value of the set area after the region of interest is lost, the exposure target value of the region of interest can be used to calculate the exposure correction value. Conversely, if the size is small and the region of interest can be ignored in the correction of the exposure target value of the set area after the region of interest is lost, the use of the exposure target value of the region of interest to calculate the exposure correction value can be prevented.

[0023] In the imaging control device according to the present technology described above, the reference size can be configured as the size of the region of interest detected in a specific region.

[0024] Accordingly, the exposure correction value is calculated based on the relative size relationship of the regions of interest detected inside / outside the specific region.

[0025] In the imaging control device according to the present technology described above, when the number of regions of interest detected outside a specific region is greater than a threshold, the exposure control unit can be configured to calculate the exposure correction value using the region of interest exposure target value calculated for the regions of interest detected outside the specific region, and when the number of regions of interest detected outside the specific region is not greater than the threshold, the exposure control unit can be configured not to calculate the exposure correction value using the region of interest exposure target value calculated for the regions of interest detected outside the specific region.

[0026] Accordingly, regarding regions of interest detected outside a specific region, when the number of regions of interest is large and the regions of interest outside the specific region cannot be ignored in the correction of the exposure target value of the set area after the loss of regions of interest, the exposure target value of the regions of interest can be used to calculate the exposure correction value. Conversely, when the number of regions of interest is small and the regions of interest outside the specific region can be ignored in the correction of the exposure target value of the set area after the loss of regions of interest, the use of the exposure target value of the regions of interest to calculate the exposure correction value can be prevented.

[0027] In the imaging control device according to the present technology described above, when a region of interest is detected outside a specific region, and there exists a region of interest within a certain distance from the region of interest detected within the specific region, the exposure control unit can be configured to calculate an exposure correction value using an exposure target value for the region of interest calculated for the region of interest. When no region of interest exists within a certain distance from the region of interest detected within the specific region, the exposure control unit can be configured not to calculate the exposure correction value using an exposure target value for the region of interest detected outside the specific region.

[0028] Accordingly, regarding regions of interest detected outside a specific region, in the case of regions of interest at a distance close to regions of interest within the specific region (in other words, regions of interest where the brightness difference with the region of interest within the specific region may be significant), the brightness difference occurs without using the exposure target value of the region of interest to calculate the exposure correction value, and the exposure target value of the region of interest can be used to calculate the exposure correction value. Conversely, in the case of regions of interest far from the region of interest within the specific region where the brightness difference is unlikely to be significant, the use of the exposure target value of the region of interest to calculate the exposure correction value can be prevented.

[0029] In the imaging control device according to the present technology described above, the exposure control unit can be configured to change a predetermined time period based on the size of the region of interest detected by the region of interest detection unit.

[0030] For example, when the region of interest (ROI) is small, the area outside the ROI dominates the composition, and when the predetermined time period (i.e., the exposure control period using exposure correction values) is long, there is a possibility that the area outside the ROI will be improperly exposed for a long time. In other words, there is a possibility that an unnatural exposure state will persist throughout the entire composition for a long time. As described above, when the predetermined time period is changed according to the size of the ROI, the predetermined time period can be shortened when the ROI is small, and it is possible to prevent an unnatural exposure state from persisting throughout the entire composition for a long time.

[0031] In the imaging control device according to the present technology described above, the exposure control unit can be configured to change a predetermined time period based on the number of regions of interest detected by the region of interest detection unit.

[0032] For example, when the number of regions of interest (ROIs) is small, the areas outside the ROIs dominate the composition, and when the predetermined time period (i.e., the exposure control period using exposure correction values) is long, there is a possibility that the exposure of the areas outside the ROIs will be inappropriate for a long time. In other words, there is a possibility that an unnatural exposure state will persist as part of the entire composition for a long time. As described above, when the predetermined time period is changed according to the number of ROIs, the predetermined time period can be shortened when the number of ROIs is small, and it is possible to prevent an unnatural exposure state from persisting as part of the entire composition for a long time.

[0033] In the imaging control device according to the present technology described above, the exposure control unit can be configured to change the predetermined time period based on the number of times the region of interest is lost within a certain time period.

[0034] For example, the fact that the region of interest (ROI) is lost a large number of times within a certain period means that when the ROI is the human face, the frequency of turning the face backward or away is high, and it can be said that the lost ROI is likely to be detected again immediately after it is lost. According to the above configuration, as mentioned above, corresponding to the high frequency of ROI loss, the predetermined time period (i.e., the exposure control period using the exposure correction value) can be shortened.

[0035] In the imaging control device according to the present technology described above, the exposure control unit can be configured to change a predetermined time period based on user operation.

[0036] Accordingly, the duration of exposure control using the exposure correction value after the region of interest is lost can be set according to the user's preferences.

[0037] The imaging control method according to this technology is an imaging control method in which an imaging control device that performs exposure control based on information of a region of interest detected from an image by a region of interest detection unit is configured to: perform exposure control when a region of interest is detected from the image, wherein an exposure target value of the region of interest calculated based on a metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame is set as an exposure control value; perform exposure control when no region of interest is detected from the image, wherein an exposure target value of a set region calculated based on a metering value of a set region obtained from a predetermined second metering region in the input frame is set as an exposure control value; and, during a predetermined period of time during which no region of interest is detected from the image, an exposure correction value is obtained based on the exposure target value of the region of interest prior to the time of loss of the region of interest, and exposure control is performed by correcting the exposure target value of the set region with the exposure correction value.

[0038] This information processing method can also achieve effects similar to those of the information processing apparatus according to the present technology described above.

[0039] The program according to this technology is a computer-readable program that enables the computer device to perform the following functions as exposure control based on information of a region of interest detected from an image by a region of interest detection unit: When a region of interest is detected from the image, exposure control is performed, wherein a target exposure value for the region of interest calculated based on a metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame is set as an exposure control value; when no region of interest is detected from the image, exposure control is performed, wherein a target exposure value for a set region calculated based on a metering value of a set region obtained from a predetermined second metering region in the input frame is set as an exposure control value; and, during a predetermined period of time during which no region of interest is detected from the image, an exposure correction value is obtained based on the target exposure value of the region of interest prior to the time of the loss of the region of interest, and exposure control is performed using the exposure control value obtained by correcting the target exposure value of the set region with the exposure correction value.

[0040] Using this procedure, the information processing apparatus described above according to this technology was realized. Attached Figure Description

[0041] Figure 1 These are plan views, front views, side views, and rear views of an imaging control device according to embodiments of the present technology.

[0042] Figure 2 This is a block diagram of the internal configuration of the imaging control device according to an embodiment.

[0043] Figure 3 This is a functional block diagram illustrating the functional configuration of exposure control according to an embodiment.

[0044] Figure 4 This is a schematic diagram illustrating the effect of exposure control based on the appearance of a captured image according to an embodiment.

[0045] Figure 5 This is an explanatory diagram of the effect of exposure control according to an embodiment.

[0046] Figure 6 This is an explanatory diagram of the effect of exposure control according to an embodiment.

[0047] Figure 7 This is a flowchart illustrating an example of a specific processing procedure for implementing exposure control according to the first embodiment.

[0048] Figure 8 It is a calculation Figure 7 The flowchart shown illustrates the processing of facial exposure correction values.

[0049] Figure 9 This is an explanatory diagram of exposure control according to the second embodiment.

[0050] Figure 10 This is a flowchart illustrating an example of a specific processing procedure for implementing exposure control according to the second embodiment.

[0051] Figure 11 This is an explanatory diagram illustrating another example of exposure control according to the second embodiment.

[0052] Figure 12 This is a diagram illustrating the process of skin beautification effect processing when capturing moving images according to an embodiment.

[0053] Figure 13 This is an explanatory diagram illustrating an example configuration of an imaging apparatus for achieving skin beautification effects according to an embodiment.

[0054] Figure 14 This is a flowchart illustrating an example of a processing procedure for achieving a skin beautification effect according to an embodiment.

[0055] Figure 15 This is an explanatory diagram illustrating the smoothing process used to achieve the skin beautification effect according to an embodiment.

[0056] Figure 16 This is a timing diagram of the smoothing process according to an embodiment.

[0057] Figure 17 This is an explanatory diagram illustrating the intensity of skin beautification effects. Detailed Implementation

[0058] The embodiments will be described below in the following order.

[0059] <1. First Embodiment>

[0060] (1-1. Configuration of the imaging device)

[0061] (1-2. AE control when region of interest is lost)

[0062] (1-3. Processing Procedure)

[0063] <2. Second Embodiment>

[0064] <3. Motion Image Skin Beautification Effect Function>

[0065] <4. Modification Example>

[0066] <5. Program>

[0067] <6. Summary of the Implementation Examples>

[0068] <7. This technology>

[0069] <1. First Embodiment>

[0070] (1-1. Configuration of the imaging device)

[0071] Figure 1 These are plan views, front views, side views, and rear views of the imaging device 1, which is an imaging control device according to an embodiment of the present technology.

[0072] Imaging device 1 is a so-called digital camera, and it can perform both still image capture and moving image capture.

[0073] In the imaging device 1, the lens unit 102 is deployed on the front side of the main body housing 100 that constitutes the camera body. During imaging, the shutter on the front surface side is opened, and the lens used for imaging is exposed.

[0074] On the rear surface side (user side) of the imaging device 1, for example, a display panel 101 is provided, which includes a display device such as a liquid crystal display (LCD) or an organic electroluminescent (EL) display.

[0075] In this example, the display panel 101 is held open / closed and rotatable by the shaft portion 101a, and is a so-called variable-angle display panel. In the figure, the display panel 101 is shown in a state where the display surface is not exposed.

[0076] Users can visually identify the image during imaging, the image during reproduction, and various other information by using the display panel 101.

[0077] Various operating elements 110 are provided on the main housing 100 of the imaging device 1.

[0078] For example, as operating element 110, various operating elements such as keys, dials, and combinations of press / rotate operating elements are provided to realize various operating functions. For example, menu operation, playback operation, mode selection operation, focus operation, zoom operation, and selection operation of parameters such as shutter speed and F-number can be performed.

[0079] Although a detailed description of each operating element 110 will be omitted, in this embodiment, several operators 110, including the shutter button 110S and the video recording button 110R, are deployed on the upper surface side of the main housing 100.

[0080] In this case, the video recording button 110R is a relatively large button and easy to operate. For example, the video recording button 110R has a size that is basically the same as that of the shutter button 110S.

[0081] For example, among the operating elements 110 provided on the upper surface side of the main housing 100, the shutter button 110S is the largest, and the video recording button 110R is the second largest. The reverse can also be applied. Alternatively, the shutter button 110S and the video recording button 110R can have the same size.

[0082] In any case, since the video recording button 110R is a large operating element on the upper surface, operation becomes easy, for example, when the imaging device 1 is placed on a table or the like and imaging is performed on a moving image poster.

[0083] Custom buttons 110C1 and 110C2 are provided as operating elements 110.

[0084] Custom buttons 110C1 and 110C2 are also known as assignable buttons. They are buttons that are assigned a predetermined operation function in the initial state and can be assigned any operation function by the user.

[0085] Note that the number of custom buttons is not limited to two, and can be one, three or more.

[0086] Reference Figure 2 Describe an example of the internal configuration of imaging device 1.

[0087] The imaging device 1 includes, for example, a lens system 11, an imaging element unit 12, a camera signal processing unit 13, a recording control unit 14, a display unit 15, a communication unit 16, an operation unit 17, a camera control unit 18, a memory unit 19, a driver unit 22, a sensor unit 23, and a power supply unit 24.

[0088] The lens system 11 includes lenses such as zoom lenses and focusing lenses, aperture mechanisms, etc. Through the lens system 11, light from the subject (incident light) is guided and focused onto the imaging element unit 12.

[0089] The imaging element unit 12 is configured, for example, to have an image sensor 12a (imaging element) such as a complementary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor.

[0090] The imaging element unit 12 performs, for example, correlated double sampling (CDS) processing and automatic gain control (AGC) processing, and further performs analog-to-digital (A / D) conversion processing on the electrical signal obtained from the light received by the image sensor 12a through photoelectric conversion. Then, the imaging signal, as digital data, is output to the camera signal processing unit 13 and the camera control unit 18 in the subsequent stages.

[0091] The camera signal processing unit 13 is configured, for example, as a digital signal processor (DSP) that functions as an image processing processor. The camera signal processing unit 13 performs various signal processing operations on the digital signals (captured image signals) from the imaging element unit 12. For example, as camera processing, the camera signal processing unit 13 performs preprocessing, synchronization processing, YC generation processing, resolution conversion processing, file formation processing, etc.

[0092] In the preprocessing, clamping processing, such as clamping the black level of RGB to a predetermined level, and correction processing between the RGB color channels are performed on the image signal captured from the imaging element unit 12.

[0093] During synchronization processing, color separation is performed so that the image data for each pixel has all RGB color components. For example, in the case of an imaging element using a Bayer array color filter, demosaicing is performed as a color separation process.

[0094] In the YC generation process, the luminance (Y) signal and color (C) signal are generated (separated) based on the RGB image data.

[0095] In resolution conversion processing, resolution conversion is performed on image data that has undergone various signal processing steps.

[0096] In file creation processing, for example, image data that has undergone the various processes described above may be compressed, encoded, formatted, have metadata generated or added for recording or communication, in order to generate a file for recording or communication.

[0097] For example, image files in formats such as JPEG, Tagged Image File Format (TIFF), or Graphics Interchange Format (GIF) can be generated as still image files. Furthermore, it is conceivable to generate image files in formats such as MP4, which record moving images and sound conforming to MPEG-4.

[0098] Note that it is also possible to generate raw image data from image files.

[0099] The camera signal processing unit 13 generates metadata, which includes information about the processing parameters in the camera signal processing unit 13, various control parameters obtained from the camera control unit 18, information indicating the operating status of the lens system 11 or the operating status of the imaging element unit 12, mode setting information, imaging environment information (date and time, location, etc.), identification information for the imaging device itself, information about the installed lens, information about the pre-registered photographer (name and identification information), and metadata from the International Press and Telecommunications Commission (IPTC).

[0100] It is important to note that IPTC metadata is metadata in a format designed by the Association of Media Companies, and it can describe various information such as "description / caption", "description author", "title", "keywords", etc.

[0101] The recording control unit 14 performs recording and reproduction, for example, with respect to a recording medium such as non-volatile memory. For instance, the recording control unit 14 performs the process of recording images, such as moving image data, still image data, or metadata, onto the recording medium.

[0102] Various practical forms of the recording control unit 14 can be considered. For example, the recording control unit 14 can be configured as a flash memory and its write / read circuitry built into the imaging device 1. Furthermore, the recording control unit 14 can be in the form of a card recording / reproducing unit that performs recording and reproducing access for the imaging device 1 and recording media that can be attached to or separated from it, such as a memory card (portable flash memory, etc.). Additionally, the recording control unit 14 can be implemented as a hard disk drive (HDD) or the like, as built into the imaging device 1.

[0103] Display unit 15 is a display unit that performs various displays on the person being imaged, and is, for example, a display panel or viewfinder of a display device such as a liquid crystal display (LCD) or an organic electroluminescent (EL) display deployed in the housing of imaging device 1.

[0104] The display unit 15 performs various displays on the display screen based on the instructions of the camera control unit 18.

[0105] For example, display unit 15 displays a reproduced image of image data read from the recording medium by recording control unit 14.

[0106] Furthermore, image data of the captured image, whose resolution has been converted by the camera signal processing unit 13 for display, can be supplied to the display unit 15, and the display unit 15 can perform display based on the image data of the captured image in response to instructions from the camera control unit 18. Thus, a so-called through image (monitoring image of the subject) is displayed, which is an image captured while undergoing composition confirmation, motion picture recording, etc.

[0107] In addition, the display unit 15 performs various operation menus, icons, messages, etc. (i.e., graphical user interface (GUI)) display on the screen based on the instructions of the camera control unit 18.

[0108] The communication unit 16 refers to all kinds of communication devices and communication processing circuits installed on the imaging device 1.

[0109] Various communication circuits and various communication devices are provided, which can perform communication via an external communication network (external network communication), perform local communication with the mobile terminal 9, and perform further master / slave communication with corresponding devices (e.g., with another imaging device 1 as a party to local communication), as communication unit 16.

[0110] Accordingly, for example, the imaging device 1 performs the transmission and reception of captured image data (still image files or moving image files), metadata, various parameters, etc., with respect to external information processing devices, imaging devices, display devices, recording devices, reproducing devices, etc.

[0111] More specifically, the communication unit 16, as a network communication unit, includes some or all of the following functions: performing communication by using mobile communication networks such as 4G, 5G, Internet lines, home networks, local area networks (LANs), performing short-range wireless communication such as Bluetooth, Wi-Fi, or near-field communication (NFC), performing infrared communication, and performing wired connection communication with another device.

[0112] Operation unit 17 generally refers to input devices that allow users to perform various input operations. Specifically, operation unit 17 refers to various operating elements (keys, dials, touch panels, touchpads, etc.) provided in the housing of imaging device 1.

[0113] The user's operation is detected by the operation unit 17, and the signal corresponding to the input operation is transmitted to the camera control unit 18.

[0114] The camera control unit 18 includes a microcomputer (arithmetic processing device) containing a central processing unit (CPU).

[0115] The memory unit 19 stores information for processing by the camera control unit 18. The memory unit 19 illustrated generally refers to, for example, read-only memory (ROM), random access memory (RAM), flash memory, etc.

[0116] The memory unit 19 may be a memory region built into the microcomputer chip that serves as the camera control unit 18, or it may be configured by a separate memory chip.

[0117] The camera control unit 18 controls the entire imaging device 1 by executing programs stored in the ROM, flash memory, etc. of the memory unit 19.

[0118] For example, the camera control unit 18 controls the operation of necessary units, such as the shutter speed control of the imaging element unit 12, various signal processing instructions in the camera signal processing unit 13, imaging and recording operations according to user operations, reproduction operations of recorded image files, operation of the lens system 11 (such as zoom, focus and aperture adjustment in the lens barrel), user interface operation, and the setting of communication scheme and transmission destination by the communication unit 16.

[0119] Furthermore, particularly in this embodiment, the camera control unit 18 performs exposure control based on metering values ​​on the captured image. Note that the details of the exposure control according to this embodiment will be described again later.

[0120] Furthermore, the camera control unit 18 in this example performs the process of detecting the region of interest from the captured image, and the region of interest and detection process will also be described later.

[0121] The RAM in memory unit 19 is used to temporarily store data, programs, etc., and serves as the working area for the CPU of camera control unit 18 when performing various data processing.

[0122] The ROM and flash memory (non-volatile memory) in memory unit 19 are used to store the operating system (OS) that allows the CPU to control each unit, content files such as image files, applications for various operations, firmware, various settings information, etc.

[0123] Examples of various settings include communication settings, settings for imaging operations such as exposure settings (shutter speed or F-number) and mode settings, settings for image processing such as white balance settings, color settings and settings for image effects, and settings for operability such as custom key settings and display settings, and so on.

[0124] The driver unit 22 may include, for example, a motor driver for a zoom lens drive motor, a motor driver for a focusing lens drive motor, and a motor driver for an aperture mechanism motor.

[0125] These motor drivers respond to commands from the camera control unit 18 by applying drive current to the corresponding drivers, and causing the drivers to perform actions such as moving the focusing lens or zoom lens, opening and closing the aperture blades of the aperture mechanism.

[0126] Sensor unit 23 refers to all kinds of sensors installed on the imaging device.

[0127] For example, an inertial measurement unit (IMU) is installed as sensor unit 23. For example, an angular velocity sensor with three axes of pitch, yaw and roll can detect angular velocity, and an acceleration sensor can detect acceleration.

[0128] Furthermore, as a sensor unit 23, for example, a location information sensor, an illuminance sensor, etc. can be installed.

[0129] Furthermore, it is assumed that a distance measurement sensor is provided as sensor unit 23. The distance from the imaging device 1 to the subject can be measured by the distance measurement sensor during imaging, and distance information can be added to the captured image as metadata.

[0130] Various information detected by the sensor unit 23 (e.g., location information, distance information, illumination information, IMU data, etc.) is added as metadata along with date and time information managed by the camera control unit 18 to the captured image.

[0131] The power supply unit 24 outputs the necessary power supply voltage Vcc for each unit by using battery 24a as a power source. The ON / OFF of the power supply voltage Vcc from the power supply unit 24 (i.e., the on / off state of the power supply to the imaging device 1) is controlled by the camera control unit 18. Furthermore, the camera control unit 18 can detect the capacity of battery 24a, i.e., the remaining battery capacity.

[0132] Note that the power supply unit 24 can be configured to output a power supply voltage Vcc based on an external power supply, for example, by connecting an AC adapter or receiving a DC power supply voltage.

[0133] (1-2. AE control when region of interest is lost)

[0134] Figure 3 This is a functional block diagram illustrating the exposure control function configuration according to an embodiment included in the camera control unit 18.

[0135] As shown in the figure, the camera control unit 18 includes a region of interest detection unit F1 and an exposure control unit F2 as functional configurations related to exposure control in this embodiment.

[0136] The region of interest detection unit F1 detects the region of interest from the image captured by the imaging element unit 12.

[0137] The region of interest (ROI) mentioned here refers to the area in an image where an object, predefined as a target of interest, exists. Examples of objects that are targets within the ROI include standalone objects such as people, animals, plants, cars, trains, airplanes, furniture, and traffic lights, as well as specific parts of standalone objects, such as a person's face or hand, or a car's license plate.

[0138] Various methods can be considered for detecting regions of interest. As an example, a method can be illustrated by using a template image of the target object to detect regions of interest through template matching. Alternatively, for example, it is also possible to detect regions of interest from captured images using an image recognition artificial intelligence (AI) engine such as a convolutional neural network (CNN).

[0139] In the following text, as an example, assume that the region of interest detection unit F1 detects a person's facial region as the region of interest. The region of interest that is the person's facial region will be referred to as "facial region Af".

[0140] The exposure control unit F2 performs exposure control based on information about the region of interest detected by the region of interest detection unit F1 from the captured image.

[0141] Specifically, when a facial region Af is detected in the captured image, the exposure control unit F2 in this example calculates a “facial exposure target value” as the exposure target value of the facial region Af based on the metering value obtained from the first metering region that includes at least the facial region Af in the input frame, and uses the facial exposure target value as the exposure control value to perform exposure control.

[0142] In this example, it is assumed that the first metering area is the same as the face area Af. Note that the first metering area can include areas other than the face area Af, such as the background area of ​​the face area Af in the input frame, and only requires that it be an area in the input frame that includes at least the face area Af. For example, the target exposure value for the face can be obtained based on the metering value obtained from the face area Af and the metering value obtained from the entire frame (the entire composition). Therefore, it is possible to prevent the entire composition from becoming too bright (or too dark).

[0143] In the following text, the function of using the facial exposure target value as the exposure control value to perform exposure control when detecting the facial region Af as described above will be referred to as the "facial AE function".

[0144] Note that the "exposure" values ​​indicated in this manual (such as the target exposure value and exposure control value mentioned above) are values ​​that indicate at least a combination of shutter speed and aperture value (F number).

[0145] Note that if no facial region Af is detected in the captured image, the exposure control unit F2 calculates the "set area exposure target value" based on the set area metering value obtained from the second metering area predetermined in the input frame, and uses the set area exposure target value as the exposure control value to perform exposure control.

[0146] Here, the aforementioned second metering area is determined by the metering mode. Examples of metering modes include multi-metering modes in which metering is performed by dividing the interior of the frame into multiple parts and exposure is controlled by balancing the entire frame; center-weighted metering modes in which metering is performed by emphasizing the central portion of the frame and exposure is determined based on the brightness near the central portion; and spot metering modes in which exposure control is performed based on metering values ​​of a specified portion of the frame.

[0147] The second metering area is the entire area of ​​the frame when the metering mode is multi-metering or center-weighted metering, and the specified area when the metering mode is spot metering.

[0148] In the following description, it is assumed that the metering mode is a multi-metering mode or a center-weighted metering mode, and that the “second metering area” is the entire area of ​​the frame.

[0149] Here, in a typical face-based after-effect (AE) function, for example, when the subject is facing to the side and the face cannot be detected, the exposure control is switched to exposure control for the entire composition. That is, the exposure control is switched to an exposure control where the "overall exposure target value," which serves as the exposure target value for the entire composition, is set to the exposure control value. Therefore, when the face cannot be detected, there is a problem of the face appearing dark.

[0150] As a measure to address this problem, it is conceivable to fix the brightness during face detection for a period of time, even when a face cannot be detected. Another method for fixing brightness is to perform after-effect (AE) locking. However, when the brightness of the entire composition changes significantly, there is a possibility that the brightness cannot keep up, resulting in overexposure or underexposure (see [link to image description]). Figure 4 (A and 4B).

[0151] In this embodiment, instead of fixing the facial exposure target value during facial detection, an exposure control method is adopted that calculates the exposure correction value (hereinafter referred to as "facial exposure correction value") based on the facial exposure target value and maintains the facial exposure correction value for a predetermined period of time.

[0152] Specifically, during a predetermined period of time during which no facial region Af is detected in the captured image, the exposure control unit F2 obtains a facial exposure correction value based on the facial exposure target value prior to the time of facial region loss, and performs exposure control using the exposure control value obtained by correcting the overall exposure target value with the facial exposure correction value.

[0153] In this example, the exposure control unit F2 obtains the value relating the overall exposure target value before the time of facial region loss to the facial exposure target value before the time of facial region loss as the aforementioned facial exposure correction value. Specifically, the exposure control unit F2 obtains the difference between the overall exposure target value before the time of facial region loss and the facial exposure target value before the time of facial region loss as the facial exposure correction value.

[0154] Here, the facial exposure target value and the overall exposure target value before the time of facial region loss can be considered as being calculated in the frames immediately preceding the frame in which the facial region Af is lost. Alternatively, the average of the exposure target values ​​from the immediately preceding frame to a predetermined number of frames can also be considered.

[0155] The target exposure values ​​for the face and the overall exposure before the time of face region loss only require the target exposure values ​​for the face and the overall exposure obtained in the frame immediately preceding or in the frame immediately preceding the frame of face region Af loss (the frame near the previous side).

[0156] In the following text, the facial exposure target value before the time of facial region loss and the overall exposure target value before the time of facial region loss are assumed to be the facial exposure target value and the overall exposure target value calculated in the frame immediately preceding the frame in which the facial region Af is lost.

[0157] In this example, the facial exposure correction value is obtained as [“Face exposure target value before the time of facial area loss” - “Overall exposure target value before the time of facial area loss”].

[0158] In this case, within a predetermined time period starting from the timing of the loss of facial region Af, a correction is performed by adding the facial exposure correction value to the overall exposure target value, and exposure control is performed using the exposure control value obtained through the correction.

[0159] Figure 5 and 6 This is an explanatory diagram illustrating the effect of exposure control according to the above embodiment.

[0160] By performing exposure control as described above, when the brightness of the background in the facial region Af does not change, such as Figure 5 As shown, even when repeated detection is performed in face detection ( Figure 5 The “D” in the text) and not detected ( Figure 5The “L” in the image also maintains the correction using the facial exposure correction value, so the brightness of the face (the brightness of the head when facing to the side or back) can be kept at an appropriate level.

[0161] Furthermore, a method is used to correct the overall exposure target value within a predetermined time period after the loss of facial regions, using a facial exposure correction value (a value indicating the relationship between the overall exposure target value before the time of facial region loss and the facial exposure target value before the time of facial region loss). Figure 6 As shown in A and 6B, even when the brightness of the entire composition changes after the face is lost, it can follow the change in brightness. Specifically, as Figure 6 As shown in A, even when the entire composition brightens after face loss and the brightness changes in the direction of decreasing overall exposure target value, this change can be followed. Furthermore, as... Figure 6 As shown in B, even after the face is lost, the entire composition darkens and the brightness changes in the direction of increasing overall exposure target value, it can follow this change.

[0162] After maintaining a predetermined period of time from the time the face is lost, exposure control is performed to make the overall brightness of the composition appropriate by deactivating the exposure correction using the face exposure correction value.

[0163] Note that in the above Figure 4 In the middle, through from Figure 4 A to Figure 4 The transition of C is illustrated schematically by the appearance of the captured image as an action of exposure control in this embodiment.

[0164] As described above, the exposure control according to this embodiment can achieve appropriate exposure control by taking into account both the brightness of the face and the brightness of the entire composition in the case of face loss, and improve the function of exposure control.

[0165] (1-3. Processing Procedure)

[0166] Figure 7 and 8 This is a flowchart illustrating an example of a specific processing procedure for implementing exposure control according to the first embodiment described above.

[0167] Note that, Figure 7 and 8 The processing shown is performed by the camera control unit 18 in this example. The camera control unit 18 repeats this process for each frame of the image captured by the imaging element unit 12. Figure 7 The series of processes shown.

[0168] exist Figure 7In step S101, the camera control unit 18 calculates the overall exposure target value. That is, the overall exposure target value is calculated based on the metering value of the second metering area (the entire area of ​​the input frame in this example) in the input frame.

[0169] In step S102, following step S101, the camera control unit 18 performs a process to acquire face detection information. That is, it performs a process to acquire information about the face region Af detected by the region of interest detection unit F1 (at least information about its position and extent (size) in the input frame).

[0170] In step S103, following step S102, the camera control unit 18 performs a process to calculate the facial exposure correction value. In this example, the process of calculating the facial exposure correction value can also handle situations where multiple facial regions Af are detected.

[0171] Figure 8 This is a flowchart of the process for calculating facial exposure correction values ​​performed in step S103.

[0172] As shown in the figure, in the process of calculating the facial exposure correction value, the camera control unit 18 first sets the target face identifier n to "1" in step S201. The target face identifier n is used to identify the value of the facial region Af to be processed in the process of calculating the facial exposure correction value.

[0173] Although not illustrated here, each time a new facial region Af is detected, the camera control unit 18 manages the newly detected facial region Af using a processing target face identifier n. Specifically, management is performed by adding a new processing target face identifier n.

[0174] In step S202, following step S201, the camera control unit 18 determines whether the nth face (face region Af) has been detected. In step S202, when the nth face region Af is detected, the camera control unit 18 advances the process to step S203, sets the face AE permission flag to ON, and performs the setting and holding counter process in the subsequent step S204.

[0175] The facial AE clearance flag is a status management flag used to detect a facial region Af. Specifically, the facial AE clearance flag is used to identify whether the target facial region Af has been detected after... Figure 6 The flags indicate the state up to the predetermined time period (holding period) shown.

[0176] The process of setting the hold counter in step S204 is a process of setting a counter for setting a predetermined time period as the hold period. The value of the hold counter is decremented for each frame by the process in step S208 (described later), and the determination of the elapsed hold period is performed when it is determined in step S207 whether the value of the hold counter has reached "zero". If the hold period has elapsed, the face AE permission flag is set to OFF in step S209 (described later).

[0177] In this example, the camera control unit 18 sets a predetermined fixed value to the value of the hold counter. In this case, the hold period is a fixed period.

[0178] In step S205, following step S204, the camera control unit 18 calculates the exposure target value (face exposure target value) for the nth face. Then, in the subsequent step S206, the camera control unit 18 calculates the face exposure correction value for face detection. Specifically, in this example, ["face exposure target value of the current frame" - "overall exposure target value of the current frame"] is calculated as the face exposure correction value for face detection. To confirm that the "face exposure target value of the current frame" is the exposure target value calculated in step S205, and that the "overall exposure target value of the current frame" is... Figure 7 The exposure target value calculated in step S101.

[0179] As described above, for the detected facial region Af, a facial exposure correction value is calculated during face detection based on the "facial exposure target value of the current frame" and the "overall exposure target value of the current frame". As will be described later, in the case of multiple detected facial regions Af, the facial exposure correction values ​​calculated for the multiple facial regions Af during face detection are averaged and used to correct the overall exposure target value (see [link to documentation]). Figure 7 Steps S104 and S105 in the process.

[0180] When performing the calculation process in step S206, the camera control unit 18 advances the process to step S213 and determines whether the target facial identifier n is equal to or greater than the maximum value N. Here, the maximum value N is the total number of facial regions Af to which the target facial identifier n is added.

[0181] In step S213, when the target face identifier n is not equal to or greater than the maximum value N, the camera control unit 18 advances the processing to step S214, increments the target face identifier n by one, and then returns to step S202.

[0182] If, in step S202, it is determined that the nth face is not detected, the camera control unit 18 proceeds the process to step S207 and determines whether the hold counter is greater than zero, i.e., whether the value of the hold counter is greater than zero. When the hold counter is greater than zero, the camera control unit 18 proceeds the process to step S208, decrements the value of the hold counter (e.g., decrements by one), and proceeds the process to step S210. In this case, the face AE permission flag of the nth face region Af remains in the ON state.

[0183] On the other hand, when the value of the counter is not greater than zero, the camera control unit 18 advances the process to step S209, sets the face AE permission flag to OFF, and advances the process to step S210.

[0184] In step S210, the camera control unit 18 determines whether the face AE permission flag is in the ON state. This corresponds to determining whether the retention period after the face region is lost has elapsed for the nth face region Af.

[0185] In step S210, when the face AE permission flag is ON, the camera control unit 18 advances the process to step S211 and calculates the face exposure correction value for holding. Specifically, in this example, [“face exposure target value before the time of face region loss” - “overall exposure target value before the time of face region loss”] is calculated as the face exposure correction value for holding. For confirmation, the “time of face region loss” mentioned here refers to the time when the nth face region Af is lost.

[0186] When the calculation process in step S211 is executed, the camera control unit 18 advances the process to step S213 as described above.

[0187] On the other hand, when it is determined in step S210 that the face AE permission flag is not in the ON state, the camera control unit 18 advances the process to step S212, sets "zero" to the face exposure correction value, and then advances the process to step S213. As described above, for face region Af where the face AE permission flag is set to OFF, that is, for face region Af where the holding period has passed, the face exposure target value of face region Af is not used to correct the overall exposure target value of the current frame.

[0188] When it is determined in step S213 that the target facial identifier n is equal to or greater than the maximum value N, the camera control unit 18 terminates. Figure 8 The series of processes shown.

[0189] Through the above Figure 8The process involves a series of steps, including calculating the facial exposure correction value for face detection based on the "facial exposure target value of the current frame" and the "overall exposure target value of the current frame" (S206) for the detected facial region Af, and calculating the facial exposure correction value for retention based on the "facial exposure target value before the time of facial region loss" and the "overall exposure target value before the time of facial region loss" (S211) for the facial region Af within a predetermined time period after the facial region is lost.

[0190] Description Return to Figure 7 .

[0191] exist Figure 7 In step S103, when the facial exposure correction value is calculated, the camera control unit 18 calculates the average facial exposure correction value in step S104. That is, the average of the facial exposure correction values ​​(including detection time and holding time) calculated in step S103 is calculated. Note that if there is only one facial region Af managed using the processing target facial identifier n, the facial exposure correction value calculated for facial region Af in step S104 is set as the average facial exposure correction value.

[0192] Then, in step S105 following step S104, the camera control unit 18 performs a process of calculating the exposure control value by ["overall exposure target value" + "average face exposure correction value"] as the process of calculating the exposure control value.

[0193] When the processing in step 105 is executed, the camera control unit 18 ends. Figure 7 The series of processes shown.

[0194] When all facial regions Af are detected through the above series of processes, the average of the facial exposure correction values ​​obtained for all facial regions Af at the time of detection is used to correct the overall exposure target value of the current frame, and the corrected overall exposure target value is used as the exposure control value to perform exposure control.

[0195] Furthermore, when all facial regions Af exist during the hold period after facial regions are lost, the overall exposure target value of the current frame is corrected using the average of the facial exposure correction values ​​obtained for all facial regions Af during the hold period, and the corrected overall exposure target value is used as the exposure control value to perform exposure control.

[0196] Furthermore, when the facial region Af during the detection state and the facial region Af during the hold period after facial region loss are mixed, a facial exposure correction value at the time of detection is calculated for the facial region Af during the detection state, and a facial exposure correction value at the time of hold is calculated for the facial region Af during the hold period after facial region loss. The average of the facial exposure correction value at the time of detection and the facial exposure correction value at the time of hold is used to correct the overall exposure target value of the current frame, and exposure control is performed using the corrected overall exposure target value as the exposure control value.

[0197] Note that even in the case of facial region Af under the mixed detection state as described above, for facial region Af during the hold period after facial region loss, the overall exposure target value is always corrected based on the facial exposure correction value at the hold time (i.e., the facial exposure correction value calculated based on the "facial exposure target value before the time of facial region loss" and the "overall exposure target value before the time of facial region loss").

[0198] <2. Second Embodiment>

[0199] Next, a second embodiment will be described. In the second embodiment, when multiple facial regions Af are detected, a target facial exposure value for calculating the facial exposure correction value is calculated based on at least one of the size, number, or location of the detected facial regions Af.

[0200] Note that in the second embodiment, since the hardware configuration of the imaging device 1 is similar to that of the first embodiment, repeated descriptions will be avoided.

[0201] Furthermore, in the following description, the same reference numerals are given to parts similar to those already described, and their descriptions will be omitted.

[0202] Figure 9 Figure A illustrates a composition where a relatively large facial area Af, located in the center of the frame of the captured image, is in sunlight, while a relatively smaller facial area Af, located near the ends of the frame, is in shadow. Generally, when photographing multiple subjects of the same type (such as "people") in a frame, a relatively large subject in the center of the frame, such as... Figure 9 As in example A, and with relatively small subjects at the ends of the frame, it can be said that the subject in the center of the frame is the main subject in the composition.

[0203] exist Figure 9 In the composition of case A, when the facial exposure target value of the facial region Af near the end of the frame is used to calculate the facial exposure correction value, the small face near the end of the frame is dark, so there is a possibility that the facial exposure correction value will be large due to overexposure.

[0204] Therefore, for the captured image, within the image region, the area where the main subject will be photographed is defined as a specific region As, and for the facial region Af outside the specific region As, the facial exposure target value is not used to calculate the facial exposure correction value (see [reference]). Figure 9 B). Therefore, the exposure correction value can be calculated using only the exposure target value of the face region Af detected in a specific region As (i.e., the subject region considered as the main subject region in the composition), and appropriate exposure control can be achieved.

[0205] In this example, a specific region As is defined as a rectangular portion of the image center within the region of the captured image. More specifically, a specific region As is set to have a size in which the center of the specific region As coincides with the image center and its area relative to the entire captured image is at least equal to or greater than half.

[0206] It should be noted that the position and size of the specific region As are not limited to these. It is only required that the specific region As be determined by estimating the position or size of the main subject being photographed in the composition, and the specific region As can be set at a position offset from the center of the captured image in at least one of the top, bottom, right, or left directions. Furthermore, the area ratio of this region As to the entire captured image can be equal to or less than one-half. Moreover, the specific region As is not limited to a region with a fixed position and size, and can, for example, be made variable based on user operation, image analysis results, etc.

[0207] Figure 10 This is a flowchart illustrating an example of a specific processing procedure for implementing exposure control according to the second embodiment described above.

[0208] Note that, Figure 10 The processing shown is performed by the camera control unit 18 in conjunction with the previous Figure 7 The processing shown is repeated for each frame of the captured image in a similar manner.

[0209] First, the camera control unit 18 executes the process of step S102 after the processing in step S101. These processes are related to... Figure 7 The descriptions are similar to those in the text, thus avoiding repetitive descriptions.

[0210] In this case, when face detection information is acquired in step S102, the camera control unit 18 executes the processing of step S301. In step S301, the camera control unit 18 performs the process of setting the face in the specific region As as the face to be processed. That is, only the face region Af located in the specific region As among the face regions Af detected in the captured image is set as the face region Af to be used as the processing target for calculating the face exposure correction value in step S103.

[0211] In this case, when the setting process in step S301 is executed, the camera control unit 18 advances the process to step S103. The process from step S103 to step S105 is similar to... Figure 7 The descriptions are similar to those in the text, thus avoiding repetitive descriptions.

[0212] By using the processing described above, for a facial region Af detected outside the specific region As, the facial exposure target value can be prevented from being used to calculate the facial exposure correction value for both the holding period before and after the facial region is lost. Accordingly, it is possible to prevent the brightness of the main subject from becoming inappropriate due to the brightness of a non-main subject detected outside the specific region As, and to achieve appropriate exposure control.

[0213] Here, in the above description, for a facial region Af detected outside a specific region As, the target facial exposure value of facial region Af is not unconditionally used to calculate the facial exposure correction value. However, as... Figure 11 As shown, it is also possible to determine whether to use the facial exposure target value to calculate the facial exposure correction value based on the size, number, and location of the facial region Af detected outside a specific region As.

[0214] Specifically, such as Figure 11 As shown in Figure A, when the size of the facial region Af detected outside the specific region As is larger than the size of the facial region Af detected within the specific region As, the facial exposure target value of the facial region Af detected outside the specific region As is used to calculate the facial exposure correction value.

[0215] In this case, Figure 10In step S301 of the process shown, the camera control unit 18 determines the dimensional relationship between the facial region Af detected within the specific region As and the facial region Af detected outside the specific region As, and determines whether to process the facial region Af detected outside the specific region As based on the determination result of the dimensional relationship. That is, when the size of the facial region Af detected outside the specific region As is large, the facial region Af is set as the processing target, and when the size of the facial region Af detected outside the specific region As is not large, the facial region Af is not set as the processing target.

[0216] Note that the above description has already described an example of comparing the size of the facial region Af detected within a specific region As with the size of the facial region Af detected outside the specific region As. However, the size of the facial region Af outside the specific region As can be compared with at least some reference size. For example, the reference size can be a fixed size or a size that can vary according to certain conditions.

[0217] In addition, such as Figure 11 As shown in B, when the number of facial regions Af detected outside a specific region As is equal to or greater than a certain number, the facial exposure target value of the facial regions Af detected outside the specific region As can be used to calculate the facial exposure correction value.

[0218] In this case, Figure 10 In step S301 shown, the camera control unit 18 determines whether the number of facial regions Af detected outside the specific region As is equal to or greater than a predetermined threshold. When the number of facial regions Af is equal to or greater than the threshold, the facial regions Af detected outside the specific region As will be set as processing targets. When the number of facial regions Af is not equal to or greater than the threshold, the facial regions Af detected outside the specific region As will not be set as processing targets.

[0219] In addition, such as Figure 11 As shown in C, when the distance between the facial region Af detected outside the specific region As and the facial region Af detected inside the specific region As is within a certain distance, the facial exposure target value of the facial region Af detected outside the specific region As is used to calculate the facial exposure correction value.

[0220] In this case, Figure 10In step S301 shown, the camera control unit 18 calculates the distance between the face region Af detected outside the specific region As and the face region Af detected inside the specific region As. When the distance is within a certain distance, the face region Af detected outside the specific region As is set as the processing target, and when the distance is not within a certain distance, the face region Af detected outside the specific region As is not set as the processing target.

[0221] Note that combinations are possible. Figure 11 The processing shown in A to 11C. For example, regarding a facial region Af detected outside a specific region As, it is conceivable to use a facial exposure target value to calculate the desired result. Figure 11 The condition for the number of times detected in B and satisfying it. Figure 11 The facial exposure correction value for the facial region Af, given the size condition in A. Alternatively, it is conceivable to use the facial exposure target value to calculate the value that satisfies the condition. Figure 11 The distance condition of C and Figure 11 Facial exposure correction values ​​for the facial region under size conditions A, etc.

[0222] <3. Motion Image Skin Beautification Effect Function>

[0223] Next, the motion image skin beautification effect function of imaging device 1 will be described.

[0224] The motion image skin beautification effect function performs skin beautification processing in real time when shooting motion images, just as it would when shooting still images.

[0225] Here, skin beautification effect processing refers to image processing that gives the face of a subject a beautified skin appearance, and is performed, for example, as an adjustment processing of at least one of brightness, contrast, color reproduction, white balance or noise reduction, or an adjustment processing of all or some of brightness, contrast, color reproduction, white balance and noise reduction.

[0226] Noise reduction processing includes, for example, removing high-frequency components such as low-pass filters (LPF).

[0227] In the following description, as an example, it is assumed that the skin beautification effect processing is noise reduction processing.

[0228] When performing skin beautification effects, the subject's face is detected, but the processing for detecting the facial region Af has already been described, so a repetitive description will be avoided.

[0229] When capturing moving images, if a facial region Af is detected, skin beautification effects are applied only to the facial region Af (see [link]). Figure 12 (A to 12C).

[0230] At this point, skin beautification can be performed by varying the intensity of the effect based on the distance from the center of the facial region Af, which is shown as the face detection area in the figure. Applying skin beautification to the cheeks, which are close to the center of the face, is effective. However, when the resolution decreases more significantly at locations farther from the center of the face, there is a possibility of creating an unnatural image. In particular, unnatural images may form when the perceived resolution of the hair and chin decreases. Therefore, it is conceivable that the intensity of the skin beautification effect decreases as the distance from the center of the face increases.

[0231] Figure 13 This is an explanatory diagram illustrating an example configuration of an imaging device 1 used to achieve the skin beautification effect processing described above, where the intensity of the effect changes according to the distance from the center of the face.

[0232] Note that, with Figure 1 The difference in the configuration of the imaging device 1 shown is that a camera control unit 18A is provided instead of a camera control unit 18, and a memory unit 19A is provided instead of a memory unit 19. Therefore, in Figure 13 In the figure, only the camera control unit 18A and the memory unit 19A are extracted and illustrated.

[0233] Here, the size of the facial region Af is not constant, and it can change during the capture of moving images. When the size of the facial region Af changes, the areas such as the cheeks to which skin beautification effects are applied also change. Therefore, the correspondence between the distance from the center of the face and the intensity of the skin beautification effect should be adjusted (e.g., how much the distance from the center of the face should be increased and how much the skin beautification effect should be reduced).

[0234] Therefore, in this example, a reliability table 30 is used, which sets the reliability corresponding to the distance from the center of the face based on the size of the detected facial region Af, and an effect intensity table 31 is used, which sets the correspondence between the reliability and effect intensity of the skin beautification effect processing. In the reliability table 30, information indicating the correspondence between the distance (in pixels) from the center of the facial region Af and the reliability is stored for each size of the facial region Af. Since the intensity of the skin beautification effect decreases as the distance from the center of the facial region Af increases, the reliability is set to decrease as the distance from the center of the facial region Af increases.

[0235] Furthermore, in Table 31, the higher the reliability, the higher the effect intensity is set.

[0236] Reliability table 30 and effect intensity table 31 are stored in memory unit 19A, and camera control unit 18A can refer to the information content of reliability table 30 and effect intensity table 31.

[0237] The camera control unit 18A, together with the aforementioned region of interest detection unit F1, functions as a skin beautification effect processing unit F3.

[0238] The skin beautification effect processing unit F3 performs skin beautification effect processing on the facial region Af detected by the region of interest detection unit F1. Specifically, for the detected facial region Af, the skin beautification effect processing unit F3 obtains the effect intensity for each pixel based on the reliability table 30 and the effect intensity table 31, and performs skin beautification effect processing according to the obtained effect intensity.

[0239] Figure 14 This is a flowchart of the processing performed by the camera control unit 18A, which functions as the skin beautification effect processing unit F3. Note that the camera control unit 18A repeats this process for each frame of the captured image. Figure 14 The series of processes shown.

[0240] First, as part of the face detection determination process in step S401, the camera control unit 18A determines whether the region of interest detection unit F1 has detected a face region Af in the captured image. If no face region Af is detected, the camera control unit 18A terminates the process. Figure 14 The processing shown.

[0241] When a facial region Af is detected, the camera control unit 18A advances the processing to step S402 and detects the size of the facial region. That is, it detects the size of the detected facial region Af.

[0242] In step S403, following step S402, the camera control unit 18A obtains the reliability of each pixel based on the size of the face region according to the reliability table 30. That is, the reliability of each pixel of the face region Af is obtained by referring to the correspondence information corresponding to the size of the detected face region Af from the correspondence information stored in the reliability table 30 for each face size (i.e., information indicating the correspondence between the distance from the center of the face region Af and the reliability).

[0243] In step S404, following step S403, the camera control unit 18A obtains the effect intensity of each pixel based on the effect intensity table 31. That is, based on the reliability of each pixel obtained in step S403, the corresponding enhancement intensity is obtained for each pixel from the effect intensity table 31.

[0244] Then, in step S405 following step S404, as a skin beautification effect processing for facial region Af, skin beautification effect processing corresponding to the effect intensity obtained in step S404 is performed on facial region Af.

[0245] When the processing in step S405 is executed, the camera control unit 18A ends. Figure 14 The series of processes shown.

[0246] Through the processing described above, even if the size of the facial region Af changes during the capture of a moving image, the facial region Af can be appropriately beautified.

[0247] In the skin beautification effect processing, it is also conceivable to use the timing of the loss of facial region Af or the timing of the detection of facial region Af as a trigger to apply a smoothing filter in the time direction.

[0248] Specifically, such as Figure 15 As shown in Figure A, skin beautification processing is performed in the frame where the facial region Af is detected, and then the skin beautification effect is applied to achieve the desired effect. Figure 15 In frame B, where facial region Af was not detected due to the face being turned to the side, the detection rate gradually decreases over time. Furthermore, in frames such as... Figure 15 As shown in C, after the face is detected again, the skin beautification effect gradually recovers (the effect intensity gradually increases), as... Figure 15 C to Figure 15 The transition of D is shown.

[0249] At this point, the target of smoothing can be at least one of brightness, contrast, color reproduction, white balance, or noise reduction, or a combination of all or some of brightness, contrast, color reproduction, white balance, and noise reduction.

[0250] Figure 16 The timing diagram of the smoothing process described above is illustrated.

[0251] As shown in the figure, the intensity of the skin beautification effect gradually increases in a predetermined number of frames after the detection of the facial region Af. During the detection of the facial region Af, skin beautification effect processing is performed on the location of the detected facial region Af in each frame.

[0252] Furthermore, in frames following the loss of facial region Af, while the position of the target area for skin beautification remains immediately before the loss of facial region, the intensity of the skin beautification effect gradually decreases.

[0253] Note that the slope of the effect intensity can be arbitrarily set when the effect intensity gradually changes during smoothing. In this case, the slope of the effect intensity can be the same or different on the side where the effect intensity increases (from which frame smoothing of the detected facial region Af is achieved) and the side where the effect intensity decreases (from which frame smoothing of the lost facial region Af is achieved).

[0254] Incidentally, regarding skin beautification effects, the optimal effect intensity differs between still and moving images. Therefore, it is also conceivable to perform skin beautification effect processing with an independent effect intensity.

[0255] Figure 17 A to Figure 17 D is a diagram illustrating the intensity of the skin beautification effect, and it shows the percentages of skin beautification effects. Figure 17 A, Figure 17 B Figure 17 C Figure 17 Examples of D with progressively increasing order.

[0256] In still images, the subject's face is stationary, while in moving images, the subject's face moves to some extent, thus the optimal intensity of the skin beautification effect differs. Specifically, in moving images, the skin beautification effect tends to be difficult to perceive unless the effect intensity is further increased. Therefore, in moving images, a higher skin beautification effect intensity is applied than in still images. For example, when there are three settings for the skin beautification effect intensity in both moving and still images—"strong," "weak," and "OFF"—the intensity of the skin beautification effect in the moving image is set to be stronger than that in the still image when comparing the "strong" setting for the skin beautification effect intensity in the moving image with that in the still image. The same applies to the "weak" setting. Under these same settings, it is preferable to set the intensity of the skin beautification effect in the moving image to be stronger than that in the still image.

[0257] In addition, the skin beautification effect can not only be set to ON / OFF, but also the effect amount can be set according to user preferences.

[0258] It is worth noting that similar applications of the aforementioned facial afterimage (AE) control can be used to address skin beautification effects. Specifically, for at least one of brightness, contrast, color reproduction, white balance, or noise reduction, or a combination of all or some of these, adjustment processing based on correction values ​​is performed. These correction values ​​are calculated based on appropriate values ​​for the facial area and appropriate values ​​for the entire composition.

[0259] Here, the skin beautification effect can also be controlled based on the imaging status information indicating the imaging status.

[0260] Imaging status information includes selfie confirmation information indicating whether a selfie is being taken.

[0261] In addition, examples of controlling skin beautification effects include switching control between ON and OFF of skin beautification effect processing, or determining control of the intensity of skin beautification effect.

[0262] When it is determined that the user is taking a selfie, set the skin beautification effect to ON or increase the intensity of the skin beautification effect.

[0263] Here, "increase the intensity of the skin beautification effect" means "increase the intensity of the skin beautification effect to a value higher than a predetermined value" or "when it is determined that a selfie is being taken, increase the intensity of the skin beautification effect to a higher value compared to when it is determined that no selfie is being taken".

[0264] On the other hand, when it is determined that the user is not taking a selfie, the skin beautification effect is set to OFF or the intensity of the skin beautification effect is reduced.

[0265] Here, "reducing the intensity of the skin beautification effect" means "reducing the intensity of the skin beautification effect to below a predetermined value" or "reducing the intensity of the skin beautification effect to a lower level when it is determined that no selfie is being taken, compared to when it is determined that a selfie is being taken."

[0266] Examples of selfie determination information include information from "Information Example 1" to "Information Example 8" below. However, skin beautification effect processing can also be controlled based on any one of the information from "Information Example 1" to "Information Example 8", or it can be controlled based on multiple pieces of selfie determination information.

[0267] "Information Example 1" is a fixed imaging device determination information indicating whether imaging device 1 is fixed.

[0268] The information for fixing the imaging device includes the following.

[0269] <1> Tripod connection status information indicating whether the imaging device is connected to the tripod.

[0270] When the tripod connection status information indicates that the imaging device is connected to a tripod, the skin beautification effect processing is set to ON or the intensity of the skin beautification effect is increased.

[0271] When the tripod connection status information indicates that the imaging device is connected to a tripod, the skin beautification effect processing is set to OFF or the intensity of the skin beautification effect is reduced.

[0272] <2> Image change information indicating changes in the captured image

[0273] When the image change information indicates that the value of the change in the captured image is equal to or less than a predetermined value, the skin beautification effect is set to ON or the intensity of the skin beautification effect is increased.

[0274] If the image change information indicates that the value of the change in the captured image is equal to or greater than a predetermined value, the skin beautification effect processing is set to OFF or the intensity of the skin beautification effect is reduced.

[0275] Here, for example, changes in the captured image are determined based on the correlation between the current frame and past frames (e.g., frames that are temporally preceding frames). If the correlation is equal to or greater than a certain value, the change in the image is determined to be within a threshold; if the correlation is less than a certain value, the change in the image is determined to be greater than the threshold.

[0276] <3> Imaging device orientation information indicating the orientation of the imaging device

[0277] The orientation of the imaging device 1 is detected by using a gyroscope sensor or similar device.

[0278] When the change in the orientation of the imaging device 1 indicated by the imaging device orientation information is equal to or less than a predetermined value, the skin beautification effect processing is set to ON or the intensity of the skin beautification effect is increased.

[0279] When the change in the orientation of the imaging device, as indicated by the imaging device orientation information, is equal to or less than a predetermined value, the skin beautification effect processing is set to OFF or the intensity of the skin beautification effect is reduced.

[0280] "Information Example 2" is line-of-sight information that indicates the subject's line of sight.

[0281] When the change in the subject's gaze, as indicated by the gaze information, is equal to or less than a predetermined value, the skin beautification effect is set to ON or the intensity of the skin beautification effect is increased.

[0282] When the change in the subject's gaze, as indicated by the gaze information, is equal to or less than a predetermined value, the skin beautification effect is set to OFF or the intensity of the skin beautification effect is reduced.

[0283] "Information Example 3" is display unit orientation information indicating the orientation of the display panel 101 of the imaging device 1.

[0284] As for the display panel 101 of the imaging device 1, there are the aforementioned variable angle type and tilt type.

[0285] When the direction of the display panel 101 indicated by the display unit direction information is on the lens side of the imaging device 1 (when the display panel 101 is opened horizontally), the skin beautification effect processing is set to ON or the intensity of the skin beautification effect is increased.

[0286] When the direction of the display panel 101 indicated by the display unit direction information is on the side opposite to the lens of the imaging device 1 (when the display panel 101 is in the normal position), the skin beautification effect processing is set to OFF or the intensity of the skin beautification effect is reduced.

[0287] "Information Example 4" is sound directionality information that indicates the directionality of a sound.

[0288] The sound directivity can be obtained through one or more microphones provided in the imaging device 1.

[0289] When the change in the sound directionality of the subject, as indicated by the sound directionality information, is equal to or less than a predetermined value, the skin beautification effect is set to ON or the intensity of the skin beautification effect is increased.

[0290] When the change in the sound directionality of the subject, as indicated by the sound directionality information, is equal to or greater than a predetermined value, the skin beautification effect is set to OFF or the intensity of the skin beautification effect is reduced.

[0291] Note that the sound source location direction information, which indicates the direction of the sound source location, can also be used as sound-based information. When the change in sound source location is determined to be equal to or less than a predetermined amount based on the sound source location direction information, the skin beautification effect is set to ON or the intensity of the skin beautification effect is increased. Conversely, when the change in sound source location is determined to be equal to or greater than a predetermined amount based on the sound source location direction information, the skin beautification effect is set to OFF or the intensity of the skin beautification effect is decreased.

[0292] "Information Example 5" is facial position information that indicates the position of the subject's face in the captured image.

[0293] It can detect the position of a subject's face in the captured image, and can utilize changes in the detected position of the face in the captured image.

[0294] When the facial position information indicates that the position of the subject's face in the captured image has changed by a value equal to or less than a predetermined value, the skin beautification effect is set to ON or the intensity of the skin beautification effect is increased.

[0295] When the facial position information indicates that the position of the subject's face in the captured image has changed by a value equal to or less than a predetermined value, the skin beautification effect is set to OFF or the intensity of the skin beautification effect is reduced.

[0296] "Information Example 6" is face frame size information that indicates the size of the subject's face frame in the captured image.

[0297] When the face frame size information indicates that the size of the face frame in the captured image is equal to or greater than a predetermined value, the skin beautification effect is set to ON or the intensity of the skin beautification effect is increased.

[0298] When the face frame size information indicates that the size of the face frame in the captured image is equal to or smaller than a predetermined value, the skin beautification effect is set to OFF or the intensity of the skin beautification effect is reduced.

[0299] "Information Example 7" is the focus target distance information indicating the distance between the focus target and the imaging device 1.

[0300] When taking a selfie, the imaging device is held in hand or a selfie stick is used. In this case, the distance between the imaging device and the subject is often within a predetermined range corresponding to the selfie. By using this, it is determined that a selfie is being taken when the focus target distance information indicates that the distance between the focus target and the imaging device is within a predetermined range.

[0301] For example, it indicates a distance of A cm to B cm within a predetermined range, or a distance outside the predetermined range that is shorter than A cm or longer than B cm.

[0302] Furthermore, the case where the distance within the predetermined range is equal to or less than the predetermined distance indicates the case where the distance outside the predetermined range is longer than the predetermined distance.

[0303] When the distance information indicating the target distance to the subject to be focused is within a predetermined range, the skin beautification effect is set to ON or the intensity of the skin beautification effect is increased.

[0304] When the distance indicated by the focus target distance information to the subject to be focused is outside the predetermined range, the skin beautification effect is set to OFF or the intensity of the skin beautification effect is reduced.

[0305] "Information Example 8" is personal information indicating whether the subject is a specific person.

[0306] When a face registered based on the registered facial information registered in imaging device 1 matches the face of the subject, the person information indicates that the subject is a specific person. Furthermore, when a person's face is automatically determined based on facial information, the person information indicates that the subject is a specific person (e.g., the owner of imaging device 1). Automatically determining a person's face based on facial information includes, for example, generating facial information as information about a person's face based on information about the subject in previously captured images and determining whether the subject's face is that of a specific person based on the facial information.

[0307] When the person information indicates that the person is a specific individual, the skin beautification effect is set to ON or the intensity of the skin beautification effect is increased.

[0308] When the person information indicates that the person is not a specific individual, the skin beautification effect is set to OFF or the intensity of the skin beautification effect is reduced.

[0309] Furthermore, the skin beautification effect processing can be controlled based on the determination of whether the imaging device 1 is in a state of being placed to perform imaging (imaging while placed on a stationary object). Here, the imaging device fixation determination information in "Information Example 1" above can be used to determine whether the imaging device is in a state of being placed to perform imaging. Furthermore, if a hand shake correction function is provided, it is conceivable to perform the determination based on whether hand shake correction is performed.

[0310] When it is determined that the imaging device 1 is in a position to perform imaging, the skin beautification effect processing is set to ON or the intensity of the skin beautification effect is increased (because the imaging device 1 is not moving, image retention is unlikely to occur). On the other hand, when it is determined that the imaging device 1 is not in a position to perform imaging, the skin beautification effect processing is set to OFF or the intensity of the skin beautification effect is decreased (because the imaging device 1 is moving, image retention may occur).

[0311] <4. Modification Example>

[0312] Here, the embodiments are not limited to the specific examples described above, and various configurations can be used as examples of modifications.

[0313] For example, in the above description, the difference between the overall exposure target value before the time of facial area loss and the facial exposure target value before the time of facial area loss is obtained as the facial exposure correction value. However, the facial exposure correction value can also be obtained as a ratio instead of the difference between the overall exposure target value and the facial exposure target value. That is, the facial exposure correction value can be obtained as a ratio between the overall exposure target value before the time of facial area loss and the facial exposure target value before the time of facial area loss, for example, the value of ["facial exposure target value before the time of facial area loss" / "overall exposure target value before the time of facial area loss"]. In this case, it is only required to perform the correction during the retention period (a predetermined period after facial area loss) by multiplying the overall exposure target value by the facial exposure correction value as the ratio.

[0314] As mentioned above, the facial exposure correction value can be the ratio, rather than the difference, between the overall exposure target value before the time the facial area was lost and the facial exposure target value before the time the facial area was lost. Given this, the facial exposure correction value can be conceptually defined as follows.

[0315] That is, the "specific numerical relationship" between the overall exposure target value after the loss of facial area and the overall exposure target value corrected by the exposure correction value can be defined as the same as the "specific numerical relationship" between the overall exposure target value before the time of facial area loss and the facial exposure target value before the time of facial area loss.

[0316] The “specific numerical relationship” mentioned in this article refers to a specific numerical relationship between two numerical values, and “numerical relationship” refers to numerical relationships such as the difference between two numerical values ​​or the ratio between two numerical values.

[0317] Furthermore, the term "equivalent" as used in this article refers not only to strict coincidence but also to the concept of falling within a predetermined error range.

[0318] Furthermore, in the description above, it is assumed that the holding period (a predetermined period) during which the correction with facial exposure correction values ​​is performed is fixed, but the holding period can be a variable period.

[0319] For example, the duration of the hold can be varied depending on the size of the facial region Af. In this case, the camera control unit 18 (exposure control unit F2) changes the duration of the hold based on the size of the facial region Af detected by the region of interest detection unit F1.

[0320] For example, when the size of the face region Af is small, the area outside Af dominates the composition, and when the hold period (i.e., the exposure control period using the face exposure correction value) is long, there is a possibility that the area outside Af will be improperly exposed for a long time. In other words, there is a possibility that an unnatural exposure state will persist as part of the entire composition for a long time. As mentioned above, when the hold period is changed according to the size of the face region Af, the hold period can be shortened when the face region Af is small, and it is possible to prevent an unnatural exposure state from persisting as part of the entire composition for a long time.

[0321] Therefore, as an exposure control method after the loss of facial region Af, it is possible to achieve appropriate exposure control based on the size of facial region Af.

[0322] Alternatively, the duration of the hold can be varied based on the number of facial regions Af. In this case, the camera control unit 18 (exposure control unit F2) changes the duration of the hold based on the number of facial regions Af detected by the region of interest detection unit F1.

[0323] For example, when the number of facial area Afs is small, the areas outside of facial area Afs dominate the composition, and when the predetermined time period (i.e., the exposure control period using facial exposure correction values) is long, there is a possibility that the exposure of areas outside of facial area Afs will be inappropriate for a long time. In other words, there is a possibility that an unnatural exposure state will persist as part of the entire composition for a long time. As mentioned above, when the holding time is changed according to the number of facial area Afs, the holding time can be shortened when the number of facial area Afs is small, and it is possible to prevent an unnatural exposure state from persisting as part of the entire composition for a long time.

[0324] Therefore, as an exposure control method after the loss of facial region Af, it is possible to achieve appropriate exposure control based on the number of detected facial region Af.

[0325] Furthermore, the duration of the hold can be varied based on the number of times the facial region Af is lost within a certain time period. In this case, the camera control unit 18 (exposure control unit F2) changes the duration of the hold based on the number of times the facial region Af is lost within a certain time period in the past.

[0326] For example, the fact that the number of times facial region Af is lost within a certain period of time means that the face is turned to the back or the face is turned away frequently, and it can be said that the lost facial region Af is likely to be detected again immediately after the facial region Af is lost. According to the above configuration, corresponding to the case of high frequency of facial region loss as described above, the retention period (i.e., the exposure control period using the facial exposure correction value) can be shortened.

[0327] Therefore, it is possible to prevent exposure control using facial exposure correction values ​​from unnecessarily persisting for an extended period after facial area Af is lost.

[0328] Furthermore, the duration of the hold can be varied based on the user's actions. That is, in this case, the camera control unit 18 (exposure control unit F2) changes the duration of the hold, for example, based on the user's input (such as menu operations).

[0329] Therefore, the duration of exposure control using facial exposure correction values ​​after facial area Af loss can be set according to the user's preferences.

[0330] Therefore, the exposure control function can be improved.

[0331] Furthermore, it is conceivable that the duration of the hold could vary depending on, for example, the blur width and acceleration of the face within a certain time, whether the scene in which the imaging is performed is indoors or outdoors, or the stability of the main body of the imaging device 1 (e.g., held by hand or fixed on a tripod).

[0332] Furthermore, in the above description, the configuration of providing a region of interest detection unit F1 in the imaging apparatus 1 has been described as an example. However, in this art, the imaging control apparatus including the exposure control unit F2 is not limited to the configuration including the region of interest detection unit F1, and a configuration excluding the region of interest detection unit F1 may be adopted.

[0333] Furthermore, in the above description, it is assumed that the frame in which the region of interest is to be detected coincides with the frame in which metering is to be performed to obtain the exposure target value, but these frames can be different frames and can be synchronized at least in terms of timing.

[0334] Furthermore, the above description has already described an example in which metering for exposure control is performed based on an image captured by imaging element 12a. However, metering can also be performed based on the output of a metering sensor provided separately from the sensor used to acquire the captured image.

[0335] <5. Program>

[0336] The imaging control device (imaging device 1) as an embodiment has been described above, but the program of the embodiment is a program that causes a computer device such as a CPU to perform the processing of the imaging device 1.

[0337] The program according to the embodiment is a program that can be read by a computer device, and enables the computer device to perform the following functions as a function related to exposure control performed based on information from the region of interest detected in the image captured by the region of interest detection unit.

[0338] That is, as described in the function, when a region of interest is detected in the captured image, the following exposure control is performed, wherein a target exposure value for the region of interest calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame is set as an exposure control value; when no region of interest is detected in the captured image, the following exposure control is performed, wherein a target exposure value for a set area calculated based on the metering value of a set area obtained from a predetermined second metering region in the input frame is set as an exposure control value; and, during a predetermined period of time during which no region of interest is detected in the captured image, an exposure correction value is obtained based on the target exposure value of the region of interest before the time of the loss of the region of interest, and exposure control is performed by correcting the target exposure value of the set area with the exposure correction value.

[0339] That is, this program, for example, is related to causing computer devices to perform... Figure 7 , 8 The processing procedure described in 10 corresponds to that described in 10.

[0340] Such programs can be pre-stored on computer-readable storage media, such as ROM, hard disk drives (HDDs), solid-state drives (SSDs), etc. Alternatively, the program can be temporarily or permanently stored on removable storage media, such as semiconductor memory, memory cards, optical discs, magneto-optical discs, or magnetic disks. Furthermore, such removable storage media can be provided as so-called software packages.

[0341] Furthermore, such programs can be installed on personal computers or other devices from removable storage media, or downloaded from download sites via networks such as LANs or the Internet to the desired information processing devices (such as smartphones).

[0342] <6. Summary of Examples>

[0343] As described above, the imaging control device (imaging device 1) in the embodiment includes an exposure control unit (F2: camera control unit 18), which performs exposure control based on information of the region of interest detected from the captured image by the region of interest detection unit (F1).

[0344] Then, if a region of interest is detected in the captured image, the exposure control unit performs the following exposure control: a target exposure value for the region of interest calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame is set as an exposure control value; if no region of interest is detected in the captured image, the exposure control unit performs the following exposure control: a target exposure value for a set area calculated based on the metering value of a set area obtained from a predetermined second metering region in the input frame is set as an exposure control value; and, for a predetermined period of time during which no region of interest is detected in the captured image, an exposure correction value is obtained based on the target exposure value of the region of interest before the time of the loss of the region of interest, and exposure control is performed using the exposure control value obtained by correcting the target exposure value of the set area with the exposure correction value.

[0345] According to the above configuration, for example, when performing exposure control to ensure appropriate brightness in the region of interest (ROI), such as in a face AE, if the ROI is lost, exposure control is performed for a predetermined period thereafter using an exposure control value obtained by correcting the target exposure value of the set area using an exposure correction value based on the ROI exposure target value prior to the time the ROI was lost. Since the exposure control value obtained by correcting the target exposure value of the set area using an exposure correction value based on the ROI exposure target value prior to the time the ROI was lost is used as the exposure control value for the predetermined period after the ROI is lost, the brightness of the subject present in the ROI can be appropriately maintained even after the ROI is lost, provided the overall brightness of the composition does not change within the predetermined period. Furthermore, even if the overall brightness of the composition changes within the predetermined period, it can prevent situations where the brightness does not follow the overall composition changes as it would when AE locking is performed using the ROI exposure target value prior to the time the ROI was lost.

[0346] Therefore, it is possible to maintain proper exposure control even when the region of interest is lost, and to improve the exposure control function of the imaging device.

[0347] Furthermore, in the imaging control device as an example, the exposure correction value is a value that indicates the relationship between the target exposure value of the set area before the time of loss of the region of interest and the target exposure value of the region of interest before the time of loss of the region of interest.

[0348] Therefore, within a predetermined time period after the region of interest is lost, the set area exposure target value can be corrected using the exposure target value to reproduce the relationship between the set area exposure target value before the time of region of interest loss and the region of interest exposure target value before the time of region of interest loss.

[0349] Therefore, proper exposure control can be maintained even when the region of interest is lost.

[0350] Furthermore, in the imaging control device as an example, the exposure control unit performs exposure control with an exposure correction value within a predetermined time period. The exposure correction value is obtained in such a way that a specific numerical relationship between the exposure target value of the set area and the exposure target value of the set area corrected by the exposure correction value is equivalent to a specific numerical relationship between the exposure target value of the set area before the time of loss of the region of interest and the exposure target value of the region of interest before the time of loss of the region of interest.

[0351] According to the above configuration, during a predetermined period of time after the region of interest is lost, exposure control is performed such that a specific numerical relationship, such as difference or ratio, between the target exposure value of the set region and the target exposure value of the set region corrected by the exposure correction value is equivalent to a specific numerical relationship between the target exposure value of the set region before the time of the region of interest loss and the target exposure value of the region of interest before the time of the region of interest loss (in order to reproduce that relationship).

[0352] Therefore, proper exposure control can be maintained even when the region of interest is lost.

[0353] Furthermore, in the imaging control device of this embodiment, the exposure correction value is the difference between the target exposure value of the set region before the time of region of interest loss and the target exposure value of the region of interest before the time of region of interest loss.

[0354] Accordingly, exposure control is performed within a predetermined time period after the region of interest is lost, so that the difference between the target exposure value of the set region and the target exposure value of the set region corrected by the exposure correction value is equal to the difference between the target exposure value of the set region before the time of the region of interest loss and the target exposure value of the region of interest before the time of the region of interest loss.

[0355] Therefore, proper exposure control can be maintained even when the region of interest is lost.

[0356] Furthermore, in the imaging control apparatus as an example, when the region of interest detection unit detects a region of interest both inside and outside a specific region in the captured image, the exposure control unit calculates a target exposure value for the region of interest based on at least one of the size, number, or location of the detected regions of interest (see [reference]). Figures 9 to 11 ).

[0357] Regarding regions of interest detected outside a specific area, when the area size is small, the number of areas is small, or the distance from the region of interest detected within the specific area is long, there is a possibility that the brightness of the subject in the region of interest detected within the specific area may not be appropriate when considering the exposure target value for the region of interest when calculating the exposure correction value.

[0358] Based on the above configuration, in the event that such a possibility exists, the exposure target value of the region of interest detected outside the specific area can be prevented from being used to calculate the exposure correction value, and appropriate exposure control can be maintained even when the region of interest is lost.

[0359] Furthermore, in the imaging control device of the embodiment, when the size of the region of interest detected outside the specific region is larger than the reference size, the exposure control unit uses the region of interest exposure target value calculated for the region of interest to calculate the exposure correction value, and when the size of the region of interest detected outside the specific region is not larger than the reference size, the exposure control unit does not use the region of interest exposure target value calculated for the region of interest to calculate the exposure correction value.

[0360] Accordingly, regarding a region of interest detected outside a specific area, if the size of the region of interest is large and cannot be ignored in the correction of the exposure target value of the set area after the region of interest is lost, the exposure target value of the region of interest can be used to calculate the exposure correction value. Conversely, if the size is small and the region of interest can be ignored in the correction of the exposure target value of the set area after the region of interest is lost, the use of the exposure target value of the region of interest to calculate the exposure correction value can be prevented.

[0361] Therefore, as an exposure control after the loss of region of interest, appropriate exposure control can be achieved based on the size of the region of interest detected outside the specific region.

[0362] Furthermore, in the imaging control device used as an example, the reference size is the size of the region of interest detected within a specific area.

[0363] Accordingly, the exposure correction value is calculated based on the relative size relationship of the regions of interest detected inside / outside the specific region.

[0364] Therefore, even if the size of the region of interest detected outside the specific area is large, it is possible to prevent inconveniences such as the fact that the exposure target value of the region of interest is not used to calculate the exposure correction value, and appropriate exposure control can be achieved based on the size relationship between the size of the region of interest detected within the specific area and the size of the region of interest detected outside the specific area.

[0365] Furthermore, in the imaging control device of the embodiment, when the number of regions of interest detected outside the specific region is greater than a threshold, the exposure control unit uses the region of interest exposure target value calculated for the regions of interest detected outside the specific region to calculate the exposure correction value, and when the number of regions of interest detected outside the specific region is not greater than the threshold, the exposure control unit does not use the region of interest exposure target value calculated for the regions of interest detected outside the specific region to calculate the exposure correction value.

[0366] Accordingly, regarding regions of interest detected outside a specific region, when the number of regions of interest is large and the regions of interest outside the specific region cannot be ignored in the correction of the exposure target value of the set region after the loss of regions of interest, the exposure target value of the regions of interest can be used to calculate the exposure correction value. Conversely, when the number of regions of interest is small and the regions of interest outside the specific region can be ignored in the correction of the exposure target value of the set region after the loss of regions of interest, the use of the exposure target value of the regions of interest to calculate the exposure correction value can be prevented.

[0367] Therefore, as an exposure control method following the loss of the region of interest, it is possible to achieve appropriate exposure control based on the number of regions of interest detected outside the specific region.

[0368] Furthermore, in the imaging control device of the embodiment, when there is a region of interest within a certain distance from the region of interest detected within the specific region, which is a region of interest detected outside the specific region, the exposure control unit uses the region of interest exposure target value calculated for the region of interest to calculate the exposure correction value; and when there is no region of interest within a certain distance from the region of interest detected within the specific region, the exposure control unit does not use the region of interest exposure target value calculated for the region of interest detected outside the specific region to calculate the exposure correction value.

[0369] Accordingly, regarding regions of interest detected outside a specific region, in the case of regions of interest at a distance close to regions of interest within the specific region (in other words, regions of interest where the brightness difference with the region of interest within the specific region may be significant), the brightness difference occurs without using the exposure target value of the region of interest to calculate the exposure correction value, and the exposure target value of the region of interest can be used to calculate the exposure correction value. Conversely, in the case of regions of interest at a distance far from the region of interest within the specific region where the brightness difference is unlikely to be significant, the use of the exposure target value of the region of interest to calculate the exposure correction value can be prevented.

[0370] Therefore, as an exposure control method after the loss of region of interest, it is possible to achieve appropriate exposure control based on the separation distance between the region of interest detected within a specific region and the region of interest detected outside the specific region.

[0371] Furthermore, in the imaging control device as an example, the exposure control unit changes a predetermined time period based on the size of the region of interest detected by the region of interest detection unit.

[0372] For example, when the region of interest (ROI) is small, the area outside the ROI dominates the composition, and when the predetermined time period (i.e., the exposure control period using exposure correction values) is long, there is a possibility that the area outside the ROI will be improperly exposed for a long time. In other words, there is a possibility that an unnatural exposure state will persist throughout the entire composition for a long time. As mentioned above, when the predetermined time period is changed according to the size of the ROI, the predetermined time period can be shortened when the ROI is small, and it is possible to prevent an unnatural exposure state from persisting throughout the entire composition for a long time.

[0373] Therefore, as an exposure control method after the region of interest is lost, appropriate exposure control based on the size of the region of interest can be achieved.

[0374] Furthermore, in the imaging control device as an example, the exposure control unit changes a predetermined time period based on the number of regions of interest detected by the region of interest detection unit.

[0375] For example, when the number of regions of interest (ROIs) is small, the areas outside ROIs dominate the composition, and when the predetermined time period (i.e., the exposure control period using exposure correction values) is long, there is a possibility that the exposure of areas outside ROIs will be inappropriate for a long time. In other words, there is a possibility that unnatural exposure conditions will persist throughout the entire composition for a long time. As mentioned above, when the predetermined time period is changed according to the number of ROIs, the predetermined time period can be shortened when the number of ROIs is small, and it is possible to prevent unnatural exposure conditions from persisting throughout the entire composition for a long time.

[0376] Therefore, as an exposure control method after the region of interest is lost, it is possible to achieve appropriate exposure control based on the number of times the region of interest is detected.

[0377] Furthermore, in the imaging control device as an example, the exposure control unit changes a predetermined time period based on the number of times the region of interest is lost within a certain time period.

[0378] For example, the fact that the region of interest (ROI) is lost a large number of times within a certain period means that when the ROI is the human face, the frequency of turning the face backward or away is high, and it can be said that the lost ROI is likely to be detected again immediately after it is lost. According to the above configuration, as mentioned above, corresponding to the high frequency of ROI loss, the predetermined time period, i.e., the exposure control period using the exposure correction value, can be shortened.

[0379] Therefore, it is possible to prevent exposure control using exposure correction values ​​from unnecessarily persisting for an extended period after the region of interest has been lost.

[0380] Furthermore, in the imaging control device as an example, the exposure control unit changes a predetermined time period based on the user's operation.

[0381] Accordingly, the duration of exposure control using the exposure correction value after the region of interest is lost can be set according to the user's preferences.

[0382] Therefore, the exposure control function can be improved.

[0383] Furthermore, as an embodiment, the imaging control method is an imaging control method in which an imaging control device that performs exposure control based on information of a region of interest detected from an image by a region of interest detection unit is configured to: when a region of interest is detected from an image, perform exposure control in which a target exposure value of the region of interest calculated based on a metering value of the region of interest obtained from a first metering region that includes at least the region of interest in an input frame is set as an exposure control value; when no region of interest is detected from an image, perform exposure control in which a target exposure value of a set region calculated based on a metering value of a set region obtained from a predetermined second metering region in an input frame is set as an exposure control value; and during a predetermined period of time during which no region of interest is detected from an image, an exposure correction value is obtained based on the target exposure value of the region of interest prior to the time the region of interest was lost, and exposure control is performed by correcting the target exposure value of the set region with the exposure correction value.

[0384] This imaging control method can also achieve similar effects to the imaging control device described in the above embodiments.

[0385] Furthermore, the program in this embodiment is a computer-readable program that enables the computer device to perform the following functions as exposure control based on information of the region of interest detected from the captured image by the region of interest detection unit: when a region of interest is detected from the captured image, exposure control is performed in which a target exposure value of the region of interest calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame is set as an exposure control value; when no region of interest is detected from the captured image, exposure control is performed in which a target exposure value of a set region calculated based on the metering value of a set region obtained from a predetermined second metering region in the input frame is set as an exposure control value; and, during a predetermined period of time during which no region of interest is detected from the captured image, an exposure correction value is obtained based on the target exposure value of the region of interest before the time of the loss of the region of interest, and exposure control is performed by correcting the target exposure value of the set region with the exposure correction value.

[0386] Using this program, a computer device can be used as an imaging control device in the above embodiments.

[0387] It should be noted that the effects described in this manual are merely examples and are not limited thereto, and other effects may be provided.

[0388] <7. This technology>

[0389] Note that this technology can also have the following configurations. (1)

[0391] An imaging control device, comprising:

[0392] The exposure control unit is configured to perform exposure control based on information about the region of interest detected from the captured image by the region of interest detection unit.

[0393] The exposure control unit is configured as follows:

[0394] If a region of interest is detected from the captured image, exposure control is performed, wherein the target exposure value of the region of interest, calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame, is set as the exposure control value;

[0395] If no region of interest is detected in the captured image, exposure control is performed, wherein a target exposure value for the set area, calculated based on metering values ​​of the set area obtained from a pre-determined second metering area in the input frame, is set as the exposure control value; and

[0396] During a predetermined period of time in which no region of interest is detected from the captured image, an exposure correction value is obtained based on the exposure target value of the region of interest before the time of loss of the region of interest. Exposure control is then performed using the exposure control value obtained by correcting the exposure target value of the set region of interest with the exposure correction value. (2)

[0398] According to the imaging control device described in (1),

[0399] The exposure correction value is a value that indicates the relationship between the target exposure value of the set area before the time the region of interest is lost and the target exposure value of the region of interest before the time the region of interest is lost. (3)

[0401] According to the imaging control device described in (2),

[0402] During the predetermined time period, the exposure control unit performs exposure control using an exposure correction value, which is obtained in such a way that a specific numerical relationship between the set area exposure target value and the set area exposure target value corrected by the exposure correction value is equivalent to a specific numerical relationship between the set area exposure target value before the time of the loss of the region of interest and the region of interest exposure target value before the time of the loss of the region of interest. (4)

[0404] According to the imaging control device described in (2) or (3),

[0405] The exposure correction value is the difference between the target exposure value of the set region before the region of interest is lost and the target exposure value of the region of interest before the region of interest is lost. (5)

[0407] The imaging control device according to any one of (1) to (4),

[0408] In cases where the region of interest detection unit detects a region of interest both inside and outside a specific region in the captured image, the exposure control unit calculates the region of interest exposure target value for calculating the exposure correction value based on at least one of the size of the detected region of interest, the number of detected regions of interest, or the location of the detected regions of interest. (6)

[0410] According to the imaging control device described in (5),

[0411] If the size of the region of interest detected outside the specific area is larger than the reference size, the exposure control unit uses the region of interest exposure target value calculated for the region of interest to calculate the exposure correction value. If the size of the region of interest detected outside the specific area is not larger than the reference size, the exposure control unit does not use the region of interest exposure target value calculated for the region of interest to calculate the exposure correction value. (7)

[0413] According to the imaging control device described in (6),

[0414] The reference size is the size of the region of interest detected within a specific area. (8)

[0416] The imaging control device according to any one of (5) to (7),

[0417] If the number of regions of interest detected outside the specific area is greater than a threshold, the exposure control unit uses the region of interest exposure target value calculated for the regions of interest detected outside the specific area to calculate the exposure correction value. If the number of regions of interest detected outside the specific area is not greater than the threshold, the exposure control unit does not use the region of interest exposure target value calculated for the regions of interest detected outside the specific area to calculate the exposure correction value. (9)

[0419] The imaging control device according to any one of (5) to (8),

[0420] Specifically, when a region of interest is detected outside a specific region, and a region of interest exists within a certain distance of a region of interest detected within a specific region, the exposure control unit uses the region of interest exposure target value calculated for that region of interest to calculate the exposure correction value. When no region of interest exists within a certain distance of a region of interest detected within a specific region, the exposure control unit does not use the region of interest exposure target value calculated for a region of interest detected outside the specific region to calculate the exposure correction value. (10)

[0422] The imaging control device according to any one of (1) to (9),

[0423] The exposure control unit changes the predetermined time period based on the size of the region of interest detected by the region of interest detection unit. (11)

[0425] The imaging control device according to any one of (1) to (10),

[0426] The exposure control unit changes the predetermined time period based on the number of regions of interest detected by the region of interest detection unit. (12)

[0428] The imaging control device according to any one of (1) to (11),

[0429] The exposure control unit adjusts the predetermined time period based on the number of times the region of interest is lost within a certain time period. (13)

[0431] The imaging control device according to any one of (1) to (12),

[0432] The exposure control unit changes the predetermined time period based on the user's operation. (14)

[0434] An imaging control method,

[0435] The imaging control device that performs exposure control based on information about the region of interest detected from the captured image by the region of interest detection unit is configured as follows:

[0436] If a region of interest is detected from the captured image, exposure control is performed, wherein the target exposure value of the region of interest, calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame, is set as the exposure control value;

[0437] If no region of interest is detected in the captured image, exposure control is performed, wherein a target exposure value for the set area, calculated based on metering values ​​of the set area obtained from a pre-determined second metering area in the input frame, is set as the exposure control value; and

[0438] During a predetermined period of time in which no region of interest is detected from the captured image, an exposure correction value is obtained based on the exposure target value of the region of interest before the time of loss of the region of interest. Exposure control is then performed using the exposure control value obtained by correcting the exposure target value of the set region of interest with the exposure correction value. (15)

[0440] A computer device-readable program that enables the computer device to perform the following functions:

[0441] This is an exposure control function performed based on information from the region of interest detected in the captured image by the region of interest detection unit:

[0442] If a region of interest is detected from the captured image, exposure control is performed, wherein the target exposure value of the region of interest, calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame, is set as the exposure control value;

[0443] If no region of interest is detected in the captured image, exposure control is performed, wherein a target exposure value for the set area, calculated based on metering values ​​of the set area obtained from a pre-determined second metering area in the input frame, is set as the exposure control value; and

[0444] During a predetermined period of time in which no region of interest is detected from the captured image, an exposure correction value is obtained based on the exposure target value of the region of interest before the time of loss of the region of interest. Exposure control is then performed using the exposure control value obtained by correcting the exposure target value of the set region of interest with the exposure correction value.

[0445] List of reference numerals

[0446] 1 Imaging device

[0447] 11 Lens System

[0448] 12 Imaging element units

[0449] 12a Imaging element

[0450] 13 Camera signal processing unit

[0451] 14 Recording Control Unit

[0452] 15 display units

[0453] 16 Communication Units

[0454] 17 Operation Units

[0455] 18, 18A Camera Control Unit

[0456] 19,19A memory cell

[0457] 22 driver units

[0458] 23 Sensor Units

[0459] 24 power supply units

[0460] 24A battery

[0461] F1 Region of Interest Detection Unit

[0462] F2 Exposure Control Unit

[0463] Af facial area

[0464] As a specific area

Claims

1. An imaging control device, comprising: The exposure control unit is configured to perform exposure control based on information about the region of interest detected from the captured image by the region of interest detection unit. The exposure control unit is configured as follows: If a region of interest is detected from the captured image, exposure control is performed, wherein the target exposure value of the region of interest, calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame, is set as the exposure control value; If no region of interest is detected in the captured image, exposure control is performed, wherein the target exposure value of the set area, calculated based on the set area metering value obtained from a pre-determined second metering area in the input frame, is set as the exposure control value. as well as During a predetermined period of time during which no region of interest is detected in the captured image, starting from the time when the region of interest is lost, an exposure correction value is obtained based on the exposure target value of the region of interest before the time of loss of the region of interest. Exposure control is then performed using the exposure control value obtained by correcting the exposure target value of the set region of interest with the exposure correction value. The exposure correction value indicates the relationship between the target exposure value of the set area before the time of region of interest loss and the target exposure value of the region of interest before the time of region of interest loss. The exposure correction value is the difference between the target exposure value of the set region before the region of interest is lost and the target exposure value of the region of interest before the region of interest is lost.

2. The imaging control device according to claim 1, During the predetermined time period, the exposure control unit performs exposure control using an exposure correction value, which is obtained in such a way that a specific numerical relationship between the set area exposure target value and the set area exposure target value corrected by the exposure correction value is equivalent to a specific numerical relationship between the set area exposure target value before the time of the loss of the region of interest and the region of interest exposure target value before the time of the loss of the region of interest.

3. The imaging control device according to claim 1, In cases where the region of interest detection unit detects a region of interest both inside and outside a specific region in the captured image, the exposure control unit calculates the region of interest exposure target value for calculating the exposure correction value based on at least one of the size of the detected region of interest, the number of detected regions of interest, or the location of the detected regions of interest.

4. The imaging control device according to claim 3, If the size of the region of interest detected outside the specific area is larger than the reference size, the exposure control unit uses the region of interest exposure target value calculated for the region of interest detected outside the specific area to calculate the exposure correction value; if the size of the region of interest detected outside the specific area is not larger than the reference size, the exposure control unit does not use the region of interest exposure target value calculated for the region of interest detected outside the specific area to calculate the exposure correction value.

5. The imaging control device according to claim 4, The reference size is the size of the region of interest detected within a specific area.

6. The imaging control device according to claim 3, If the number of regions of interest detected outside the specific area is greater than a threshold, the exposure control unit uses the region of interest exposure target value calculated for the regions of interest detected outside the specific area to calculate the exposure correction value. If the number of regions of interest detected outside the specific area is not greater than the threshold, the exposure control unit does not use the region of interest exposure target value calculated for the regions of interest detected outside the specific area to calculate the exposure correction value.

7. The imaging control device according to claim 3, in, When a region of interest is detected outside a specific region, and a region of interest exists within a certain distance of the region of interest detected within the specific region, the exposure control unit uses the region of interest exposure target value calculated for the region of interest detected outside the specific region to calculate the exposure correction value. When no region of interest exists within a certain distance of the region of interest detected within the specific region, the exposure control unit does not use the region of interest exposure target value calculated for the region of interest detected outside the specific region to calculate the exposure correction value.

8. The imaging control device according to claim 1, The exposure control unit changes the predetermined time period based on the size of the region of interest detected by the region of interest detection unit.

9. The imaging control device according to claim 1, The exposure control unit changes the predetermined time period based on the number of regions of interest detected by the region of interest detection unit.

10. The imaging control device according to claim 1, The exposure control unit adjusts the predetermined time period based on the number of times the region of interest is lost within a certain time period.

11. The imaging control device according to claim 1, The exposure control unit changes the predetermined time period based on the user's operation.

12. An imaging control method, in, An imaging control device that performs exposure control based on information about the region of interest detected from the captured image by the region of interest detection unit is configured as follows: If a region of interest is detected from the captured image, exposure control is performed, wherein the target exposure value of the region of interest, calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame, is set as the exposure control value; If no region of interest is detected in the captured image, exposure control is performed, wherein the target exposure value of the set area, calculated based on the set area metering value obtained from a pre-determined second metering area in the input frame, is set as the exposure control value. as well as During a predetermined period of time during which no region of interest is detected in the captured image, starting from the time when the region of interest is lost, an exposure correction value is obtained based on the exposure target value of the region of interest before the time of loss of the region of interest. Exposure control is then performed using the exposure control value obtained by correcting the exposure target value of the set region of interest with the exposure correction value. The exposure correction value indicates the relationship between the target exposure value of the set area before the time of region of interest loss and the target exposure value of the region of interest before the time of region of interest loss. The exposure correction value is the difference between the target exposure value of the set region before the region of interest is lost and the target exposure value of the region of interest before the region of interest is lost.

13. A computer program product containing a program that, when executed by a processor, performs the following functions: This is an exposure control function performed based on information from the region of interest detected in the captured image by the region of interest detection unit: If a region of interest is detected from the captured image, exposure control is performed, wherein the target exposure value of the region of interest, calculated based on the metering value of the region of interest obtained from a first metering region that includes at least the region of interest in the input frame, is set as the exposure control value; If no region of interest is detected in the captured image, exposure control is performed, wherein a target exposure value for the set area, calculated based on metering values ​​of the set area obtained from a pre-determined second metering area in the input frame, is set as the exposure control value; and During a predetermined period of time during which no region of interest is detected in the captured image, starting from the time when the region of interest is lost, an exposure correction value is obtained based on the exposure target value of the region of interest before the time of loss of the region of interest. Exposure control is then performed using the exposure control value obtained by correcting the exposure target value of the set region of interest with the exposure correction value. The exposure correction value indicates the relationship between the target exposure value of the set area before the time of region of interest loss and the target exposure value of the region of interest before the time of region of interest loss. The exposure correction value is the difference between the target exposure value of the set region before the region of interest is lost and the target exposure value of the region of interest before the region of interest is lost.

Images