Exposure control device, imaging device, exposure control method, and program

Abstract

To provide an exposure control device, an imaging device, an exposure control method, and a program, capable of suppressing an unsuitable variation of brightness in a set area set in an image.SOLUTION: An exposure control device acquires an image signal outputted from an imaging element, to acquire a specific subject area in the image represented by the image signal, calculates a first exposure value of the set area set in the image and a second exposure value of a subject area, decides first weight for the first exposure value and second weight for the second exposure value on the basis of first determination for determining importance of the subject area with respect to the set area and second determination for determining a positional relation between the set area and the subject area, and uses the first weight and the second weight, to calculate a third exposure value by performing weighted addition of the first exposure value and the second exposure value.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The technology disclosed herein relates to an exposure control device, an imaging device, an exposure control method, and a program. [Background technology]

[0002] Patent document 1 discloses an aperture control device that adjusts the aperture opening based on the output signal of an image sensor that photoelectrically converts incident light through an aperture, the aperture control device being equipped with a face detection means that detects a face from a captured image based on the output signal of the image sensor, and that adjusts the opening based on at least one of the position, size, and orientation of a face contained in the captured image on the captured image.

[0003] Patent document 2 discloses an information processing device having an image acquisition means for acquiring an image obtained by imaging a subject using an imaging unit, a detection method setting means for setting a subject detection method for the image, a subject detection means for detecting a subject based on the detection method determined by the detection method setting means, and an exposure determination means for determining exposure based on the detection result obtained from the subject detection means, wherein the detection method setting means is capable of setting different detection methods for different areas in the image based on specified information when imaging was performed to obtain the image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-233470 A [Patent Document 2] JP 2020-072469 A Summary of the Invention [Problem to be solved by the invention]

[0005] The technology disclosed herein aims to provide an exposure control device, an imaging device, an exposure control method, and a program that are capable of suppressing inappropriate fluctuations in the brightness of a set area set within an image. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the exposure control device disclosed herein includes a processor and a memory, and the processor acquires an image signal output from an imaging element, acquires a specific subject area within the image represented by the image signal, calculates a first exposure value of a set area set within the image and a second exposure value of the subject area, determines a first weight for the first exposure value and a second weight for the second exposure value based on a first judgment that judges the importance of the subject area relative to the set area and a second judgment that judges the positional relationship between the set area and the subject area, and calculates a third exposure value by weighted addition of the first exposure value and the second exposure value using the first weight and the second weight.

[0007] When the processor determines in the second determination that the subject region is within the set region, it is preferable that the processor determines the first weight and the second weight regardless of the positional relationship.

[0008] When the processor determines in the second determination that the subject region is outside the set region, it is preferable that the processor determines the first weight and the second weight based on the positional relationship.

[0009] Preferably, the processor determines the weighting factors based on the result of the first determination, and determines the correction factors based on the result of the second determination.

[0010] If the first exposure value is E1, the second exposure value is E2, the third exposure value is E3, the first weight is W1, the second weight is W2, the weighting coefficient is α, and the correction coefficient is β, it is preferable that the processor determines the first weight and the second weight so as to satisfy W1 = 1 - α × β and W2 = α × β, and calculates the third exposure value based on E3 = E1 × W1 + E2 × W2.

[0011] It is preferable that the processor determines the correction coefficient to be 1 when the second determination determines that the subject region is within the set region.

[0012] When the processor determines in the second determination that the subject region is outside the set region, it is preferable that the processor determine the correction coefficient based on the positional relationship.

[0013] The positional relationship preferably includes the distance between the set region and the subject region.

[0014] It is preferable that the correction coefficient decreases as the distance increases.

[0015] The first determination is preferably a determination based on the ratio of the size of the subject region to the size of the set region.

[0016] It is preferable that the processor determines the first weight and the second weight regardless of the positional relationship when the ratio of the size of the subject area to the size of the set area is less than a first threshold value and the absolute value of the difference between the first exposure value and the second exposure value is less than a second threshold value.

[0017] The processor may determine the first weight and the second weight regardless of the positional relationship when the ratio of the size of the subject area to the size of the set area is less than a first threshold, the absolute value of the difference between the first exposure value and the second exposure value is less than a second threshold, and the subject included in the subject area is a specific subject.

[0018] The subject area is preferably a face area of ​​a person.

[0019] Preferably, the processor obtains the subject region using a machine learning model.

[0020] The set area is preferably an image display area for displaying an image within the set area, an image recording area for recording an image within the set area, or an image cut-out area for electronic zoom or image stabilization.

[0021] An imaging device according to the present disclosure includes the above-described exposure control device and an imaging element.

[0022] The exposure control method disclosed herein includes a processor acquiring an image signal output from an imaging element, acquiring a specific subject area within an image represented by the image signal, calculating a first exposure value of a set area set within the image and a second exposure value of the subject area, determining a first weight for the first exposure value and a second weight for the second exposure value based on a first determination that determines the importance of the subject area relative to the set area and a second determination that determines the positional relationship between the set area and the subject area, and calculating a third exposure value by weighted addition of the first exposure value and the second exposure value using the first weight and the second weight.

[0023] The program of the present disclosure includes acquiring an image signal output from an imaging element, acquiring a specific subject area within an image represented by the image signal, calculating a first exposure value of a set area set within the image and a second exposure value of the subject area, determining a first weight for the first exposure value and a second weight for the second exposure value based on a first judgment that judges the importance of the subject area relative to the set area and a second judgment that judges the positional relationship between the set area and the subject area, and calculating a third exposure value by weighted addition of the first exposure value and the second exposure value using the first weight and the second weight. [Brief description of the drawings]

[0024] [Figure 1] FIG. 2 is a diagram illustrating an example of the internal configuration of an imaging device. [Diagram 2] 2 is a block diagram showing an example of a functional configuration of a processor. FIG. [Diagram 3] FIG. 13 is a diagram illustrating an example of a setting region. [Figure 4] FIG. 11 is a diagram illustrating an example of a subject region acquisition process. [Diagram 5] FIG. 4 illustrates an example of the configuration of an exposure value calculation unit. [Figure 6] 5A to 5C are diagrams conceptually illustrating an exposure value calculation process. [Figure 7]FIG. 13 is a diagram showing an example of a case where a subject region is outside a set region. [Figure 8] FIG. 13 is a diagram showing an example of a case where a subject region is within a set region. [Figure 9] FIG. 13 is a diagram illustrating an example of a relationship between a weighting coefficient and a ratio. [Figure 10] FIG. 13 is a diagram illustrating an example of a relationship between a correction coefficient and a ratio. [Figure 11] 4 is a flowchart showing the flow of an entire process executed by the imaging device. [Figure 12] 10 is a flowchart showing an example of the flow of an exposure value calculation process. [Figure 13] FIG. 13 is a diagram showing a configuration of an exposure value calculation section according to a first modified example. [Figure 14] 13 is a flowchart showing a flow of an exposure value calculation process according to a first modified example. [Figure 15] 13 is a flowchart showing a flow of an exposure value calculation process according to a second modified example. [Figure 16] FIG. 13 is a diagram showing a subject area acquisition process according to a third modified example. [Figure 17] FIG. 13 is a diagram showing an example in which a set region is an image cropping region for electronic zoom. [Figure 18] FIG. 13 is a diagram showing an example in which a setting region is an image cutout region for camera shake correction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] An example of an embodiment of the technology of the present disclosure will be described with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following explanation, "IC" is an abbreviation for "Integrated Circuit". "CPU" is an abbreviation for "Central Processing Unit". "ROM" is an abbreviation for "Read Only Memory". "RAM" is an abbreviation for "Random Access Memory". "CMOS" is an abbreviation for "Complementary Metal Oxide Semiconductor".

[0028] "FPGA" is an abbreviation for "Field Programmable Gate Array". "PLD" is an abbreviation for "Programmable Logic Device". "ASIC" is an abbreviation for "Application Specific Integrated Circuit". "OVF" is an abbreviation for "Optical View Finder". "EVF" is an abbreviation for "Electronic View Finder". "CNN" is an abbreviation for "Convolutional Neural Network".

[0029] The technology of the present disclosure will be described by taking a digital camera with interchangeable lenses as an example of an embodiment of an imaging device. Note that the technology of the present disclosure is not limited to the digital camera with interchangeable lenses, and can also be applied to a digital camera with an integrated lens.

[0030] 1 shows an example of the configuration of an imaging device 10. The imaging device 10 is a digital camera with interchangeable lenses. The imaging device 10 is composed of a main body 11 and an imaging lens 12 that is replaceably attached to the main body 11. The imaging lens 12 is attached to the front side of the main body 11 via a camera side mount 11A and a lens side mount 12A.

[0031] The main body 11 is provided with an operation device 13 that includes a dial, a release button, a touch panel, etc. and accepts operations by a user. The operation modes of the imaging device 10 include, for example, a still image capturing mode, a video capturing mode, and an image display mode. The operation device 13 is operated by the user when setting the operation mode. The operation device 13 is also operated by the user when starting execution of still image capturing or video capturing.

[0032] The main body 11 is also provided with a finder 14. Here, the finder 14 is a hybrid finder (registered trademark). The hybrid finder refers to a finder in which, for example, an optical view finder (hereinafter, referred to as "OVF") and an electronic view finder (hereinafter, referred to as "EVF") are selectively used. The user can observe an optical image or a live view image of a subject projected by the finder 14 through a finder eyepiece (not shown).

[0033] A display 15 is provided on the rear side of the main body 11. Images based on image signals obtained by imaging, various menu screens, etc. are displayed on the display 15. Instead of the viewfinder 14, the user can also observe a live view image projected by the display 15.

[0034] The main body 11 and the imaging lens 12 are electrically connected by electrical contacts 11B provided on the camera side mount 11A coming into contact with electrical contacts 12B provided on the lens side mount 12A.

[0035] The imaging lens 12 includes a zoom lens 30, a focus lens 31, an aperture 32, and a driver 33. The zoom lens 30, the focus lens 31, and the aperture 32 are arranged in this order from the subject side along the optical axis A. The zoom lens 30 and the focus lens 31 are movable in the direction of the optical axis A. The zoom lens 30, the focus lens 31, and the aperture 32 constitute an imaging optical system. The type, number, and arrangement order of the lenses that constitute the imaging optical system are not limited to the example shown in FIG.

[0036] The imaging lens 12 also has a driver 33. The driver 33 is composed of, for example, a CPU, a RAM, and a ROM. The lens drive control unit 34 is electrically connected to a processor 40 in the main body 11 via the electrical contacts 12B and 11B.

[0037] The driver 33 drives the zoom lens 30, the focus lens 31, and the aperture 32 based on a control signal transmitted from the processor 40. In order to control the zoom magnification, the driver 33 drives the zoom lens 30 based on a control signal for zoom magnification control transmitted from the processor 40. In addition, the driver 33 drives the focus lens 31 based on a control signal for focus control transmitted from the processor 40 in order to adjust the focal position of the imaging lens 12.

[0038] The diaphragm 32 has an aperture whose diameter is variable around the optical axis A. The driver 33 drives the diaphragm 32 based on an aperture adjustment control signal transmitted from the processor 40 in order to adjust the amount of light incident on the light receiving surface 20A of the image sensor 20. A value representing the aperture diameter (i.e., the opening) of the diaphragm 32 is called the "aperture value."

[0039] Also provided inside the main body 11 are an image sensor 20, a processor 40, and a memory 42. The operations of the image sensor 20, the memory 42, the operation device 13, the viewfinder 14, and the display 15 are controlled by the processor 40.

[0040] The processor 40 is configured by, for example, a CPU. In this case, the processor 40 executes various processes based on a program 43 stored in the memory 42. The processor 40 may be configured by a collection of multiple IC chips. The processor 40 and the memory 42 configure an "exposure control device" according to the technology of the present disclosure.

[0041] The imaging sensor 20 is, for example, a CMOS image sensor. The imaging sensor 20 is disposed such that its optical axis A is perpendicular to the light receiving surface 20A and is located at the center of the light receiving surface 20A. Light that has passed through the imaging lens 12 is incident on the light receiving surface 20A. A plurality of pixels that generate signals by performing photoelectric conversion are formed on the light receiving surface 20A. The imaging sensor 20 photoelectrically converts the light incident on each pixel to generate and output an image signal D. The imaging sensor 20 is an example of an "imaging element" according to the technology of the present disclosure.

[0042] A color filter array in a Bayer pattern is arranged on the light receiving surface of the image sensor 20, and a color filter of any one of R (red), G (green), and B (blue) is arranged to face each pixel. Note that some of the pixels arranged on the light receiving surface of the image sensor 20 may be phase difference detection pixels for detecting a phase difference related to focus control.

[0043] The image sensor 20 also has an electronic shutter function, and the shutter speed can be changed electronically. The shutter speed is controlled by the processor 40.

[0044] Fig. 2 shows an example of the functional configuration of the processor 40. For example, the processor 40 realizes various functional units by executing processes according to a program 43 stored in the memory 42. As shown in Fig. 2, for example, the processor 40 realizes a main control unit 50, an imaging control unit 51, an image acquisition unit 52, a display control unit 53, a region determination unit 54, a subject region acquisition unit 55, and an exposure value calculation unit 56.

[0045] The main control unit 50 performs overall control of the operation of the imaging device 10 based on output information from the operation device 13. The imaging control unit 51 controls the imaging sensor 20 to execute imaging processing for causing the imaging sensor 20 to perform an imaging operation. The imaging control unit 51 drives the imaging sensor 20 in a still image imaging mode or a video imaging mode.

[0046] The image sensor 20 outputs an image signal D obtained by capturing an image. For example, the image signal D is a signal for one frame. The image sensor 20 outputs the image signal D for each frame period.

[0047] The image acquisition unit 52 acquires the image signal D output from the imaging sensor 20, and outputs the acquired image signal D to the display control unit 53, the subject area acquisition unit 55, the exposure value calculation unit 56, etc. The image acquisition unit 52 performs image processing such as demosaic processing on the acquired image signal D and then outputs it.

[0048] The region determination unit 54 determines the setting region RG based on output information from the operation device 13, and sets the setting region RG in the image represented by the image signal D. In this embodiment, the setting region RG is an image display region for displaying the image in the setting region RG on the display 15 or the finder 14. Of the image represented by the image signal D, only the image in the setting region RG is displayed on the display 15. The size or shape of the setting region RG is specified by the user operating the operation device 13. The region determination unit 54 may set the size or shape of the setting region RG based on the size of the recorded image specified by operating the operation device 13. The region determination unit 54 supplies information representing the setting region RG to the display control unit 53 and the exposure value calculation unit 56.

[0049] The display control unit 53 causes the display 15 to display an image within the set region RG based on the image signal D and the set region RG. In addition, the display control unit 53 causes the display 15 to display a live view image based on the image signal D periodically input from the image acquisition unit 52 during imaging preparation operations before still image capture or video capture.

[0050] The subject area acquisition unit 55 acquires a subject area RO including a subject to be acquired from the entire image represented by the image signal D. The user can specify the subject to be acquired (such as a person's face, a bird, an airplane, or a car) using the operation device 13. It is also possible to specify multiple types of subjects as the acquisition target. The subject area acquisition unit 55 supplies information representing the subject area RO to the exposure value calculation unit 56. Note that the entire image represented by the image signal D refers to the entire image represented by the signal output from the effective area where light is incident on the imaging sensor 20, and excludes signals output from optical black areas, etc. The subject area RO including the subject to be acquired is an example of a "specific subject area" according to the technology of the present disclosure. The specific subject area refers to an area including a subject arbitrarily selected by design or by a user.

[0051] The exposure value calculation unit 56 performs exposure value calculation processing based on the image signal D and information representing the set area RG and the subject area RO. As will be described in detail later, the exposure value calculation unit 56 calculates a first exposure value E1 of the set area RG and a second exposure value E2 of the subject area RO, and calculates a third exposure value E3 as a final exposure value by weighted addition of the first exposure value E1 and the second exposure value E2. The exposure value calculation unit 56 outputs the calculated third exposure value E3 to the main control unit 50. Note that the exposure value is a value determined by the amount of light incident on the light receiving surface 20A of the image sensor 20 during the exposure period, and is controlled by the aperture value (F value) and the shutter speed (exposure time). The exposure value is also called an EV value. The first exposure value E1 represents the brightness of the set area RG. The second exposure value E2 represents the brightness of the subject area RO.

[0052] The main controller 50 performs exposure control based on the third exposure value E3 to control the aperture diameter of the diaphragm 32 and the shutter speed of the image sensor 20. When performing exposure control, the main controller 50 determines a combination of the diaphragm value and the shutter speed that will yield the third exposure value E3, using a program diagram.

[0053] 2, the processor 40 is configured with a function of calculating a defocus amount based on a signal output from a phase difference detection pixel of the image sensor 20. The main controller 50 adjusts the in-focus position by controlling the focus lens 31 based on the calculated defocus amount.

[0054] 3 shows an example of the setting area RG. The image represented by the image signal D is rectangular. For example, the setting area RG is rectangular, and its center of gravity O coincides with the center of gravity of the image represented by the image signal D. The aspect ratio of the setting area RG may be the same as the aspect ratio of the image represented by the image signal D. In this embodiment, the setting area RG is a display area, and the size or aspect ratio changes based on the size of the recorded image, etc.

[0055] FIG. 4 shows an example of subject region acquisition processing by the subject region acquisition unit 55. In this embodiment, the subject region acquisition unit 55 acquires a subject region RO including a subject to be acquired by using a known template matching method. Specifically, the subject region acquisition unit 55 searches using a rectangular template image corresponding to the subject to be acquired, and acquires, as the subject region RO, a region having the highest degree of match with the template image from the entire image represented by the image signal D. FIG. 4 shows a case where the subject region RO is a person's face region. In this embodiment, the subject region RO is represented by a rectangular frame.

[0056] Fig. 5 shows an example of the configuration of the exposure value calculation unit 56. Fig. 6 conceptually shows the exposure value calculation process by the exposure value calculation unit 56. The exposure value calculation unit 56 is made up of a first exposure value calculation unit 61, a second exposure value calculation unit 62, a first determination unit 63, a second determination unit 64, a weight determination unit 65, and a weighted addition unit 66.

[0057] The first exposure value calculation unit 61 performs photometry on a set area RG in the image represented by the image signal D, and calculates a first exposure value E1 based on the obtained photometry value. The second exposure value calculation unit 62 performs photometry on a subject area RO in the image represented by the image signal D, and calculates a second exposure value E2 based on the obtained photometry value. For example, the first exposure value calculation unit 61 and the second exposure value calculation unit 62 perform multi-zone photometry.

[0058] The first determination unit 63 performs a first determination to determine the importance of the subject region RO relative to the set region RG. In this embodiment, the importance is a ratio R1 of the size of the subject region RO relative to the size of the set region RG. More specifically, the ratio R1 is the ratio of the area of ​​the subject region RO to the area of ​​the set region RG. The greater the ratio R1, the higher the importance.

[0059] The second determination unit 64 performs a second determination to determine the positional relationship between the set area RG and the subject area RO. In this embodiment, the second determination unit 64 determines whether or not the subject area RO is within the set area RG. For example, the second determination unit 64 obtains the center of gravity P of the subject area RO, and determines whether or not the center of gravity P is within the set area RG. Hereinafter, "the subject area RO is within the set area RG" means "the center of gravity P of the subject area RO is within the set area RG", and "the subject area RO is outside the set area RG" means "the center of gravity P of the subject area RO is outside the set area RG".

[0060] When the subject region RO is outside the set region RG, the second judgment unit 64 calculates the ratio R2 between the distance from the center of gravity O of the set region RG to the center of gravity P of the subject region RO and the distance from the outer periphery of the set region RG to the center of gravity P of the subject region RO.

[0061] Fig. 7 shows an example of a case where the subject region RO is outside the set region RG. In Fig. 7, B indicates the intersection point where the line connecting the centers O and P intersects with the outer periphery of the set region RG. In this case, the second determination unit 64 calculates the ratio R2 of the distance from the center O to the intersection point B to the distance from the center O to the center P. When the subject region RO is outside the set region RG, R2<1.

[0062] Fig. 8 shows an example of a case where the subject region RO is within the set region RG. In Fig. 8, B indicates the intersection point where an extension line of a straight line connecting the centers of gravity O and P intersects with the outer periphery of the set region RG. In this case, the second determination unit 64 calculates a ratio R2 of the distance from the center of gravity O to the intersection point B to the distance from the center of gravity O to the center of gravity P. When the subject region RO is within the set region RG, R2≧1.

[0063] The weight determination unit 65 determines a first weight W1 for the first exposure value E1 and a second weight W2 for the second exposure value E2 based on the result of the first determination by the first determination unit 63 and the result of the second determination by the second determination unit 64. Specifically, the weight determination unit 65 determines a weighting coefficient α based on the result of the first determination and determines a correction coefficient β based on the result of the second determination, and then determines the first weight W1 and the second weight W2 to satisfy the following equations (1) and (2). W1 = 1 − α × β (1) W2 = α × β (2)

[0064] Here, the weighting coefficient α is a coefficient representing the importance of the subject region RO relative to the set region RG obtained by the first judgment. The correction coefficient β is a coefficient representing the positional relationship between the set region RG and the subject region RO obtained by the second judgment. In this embodiment, the weighting coefficient α is a coefficient that depends on the above-mentioned ratio R1. The correction coefficient β is a coefficient that depends on the above-mentioned ratio R2.

[0065] Fig. 9 shows an example of the relationship between the weighting coefficient α and the ratio R1. The relationship between the weighting coefficient α and the ratio R1 is specified in advance and is stored, for example, as data in the memory 42. In the relationship shown in Fig. 9, when R1 ≥ 1, α = 1, and when R1 < 1, the weighting coefficient α decreases as the ratio R1 decreases.

[0066] The weight determination unit 65 determines the weight coefficient α based on the ratio R1 obtained by the first determination by the first determination unit 63 and data indicating the relationship between the weight coefficient α and the ratio R1. When the size of the subject region RO is equal to or larger than the size of the set region RG, the weight coefficient α is a fixed value ("1" in the example shown in FIG. 9). On the other hand, when the size of the subject region RO is smaller than the size of the set region RG, the weight coefficient α decreases as the size of the subject region RO decreases. In other words, the weight coefficient α increases monotonically as the ratio R1 increases.

[0067] Note that the relationship between the weighting coefficient α and the ratio R1 may be linear within the range of 0≦R1<1. Also, the relationship between the weighting coefficient α and the ratio R1 may be nonlinear within the range of 0≦R1<1.

[0068] Fig. 10 shows an example of the relationship between the correction coefficient β and the ratio R2. The relationship between the correction coefficient β and the ratio R2 is specified in advance and is stored as data in the memory 42, for example. In the relationship shown in Fig. 10, when R2 ≥ 1, β = 1, and when R2 < 1, the correction coefficient β decreases as the ratio R2 decreases.

[0069] The weight determination unit 65 determines the correction coefficient β based on the ratio R2 obtained by the second determination by the second determination unit 64 and data indicating the relationship between the correction coefficient β and the ratio R2. When the subject region RO is within the set region RG, the correction coefficient β is a fixed value ("1" in the example shown in FIG. 10). On the other hand, when the subject region RO is outside the set region RG, the correction coefficient β decreases as the distance between the subject region RO and the set region RG (for example, the distance between the center of gravity O and the center of gravity P) increases. That is, the correction coefficient β increases monotonically as the ratio R2 increases.

[0070] In addition, in the range of 0≦R2<1, the relationship between the correction coefficient β and the ratio R2 may be linear. In addition, in the range of 0≦R2<1, the relationship between the correction coefficient β and the ratio R2 may be nonlinear. In addition, in the range of 0≦R2<1, the correction coefficient β may be a fixed value (for example, "0").

[0071] In this way, when it is determined in the second determination that the subject region RO is within the set region RG, the weight determination unit 65 determines the correction coefficient β to be 1. In other words, when it is determined in the second determination that the subject region RO is within the set region RG, the weight determination unit 65 determines the first weight W1 and the second weight W2 regardless of the positional relationship between the set region RG and the subject region RO.

[0072] On the other hand, when the second judgment determines that the subject region RO is outside the set region RG, the weight determination unit 65 determines the correction coefficient β based on the positional relationship between the set region RG and the subject region RO. That is, when the second judgment determines that the subject region RO is outside the set region RG, the weight determination unit 65 determines the first weight W1 and the second weight W2 based on the positional relationship between the set region RG and the subject region RO.

[0073] The weighted addition unit 66 calculates the third exposure value E3 by performing weighted addition of the first exposure value E1 and the second exposure value E2 using the first weight W1 and the second weight W2. Specifically, the weighted addition unit 66 calculates the third exposure value E3 using the relational expression expressed by the following equation (3), where the relationships 0≦W1≦1 and 0≦W2≦1 are satisfied. E3 = E1 × W1 + E2 × W2 (3)

[0074] Fig. 11 shows the flow of the entire process executed by the imaging device 10. Fig. 11 shows an example of the flow of the process in the moving image imaging mode.

[0075] First, the main control unit 50 determines whether or not an image capture start instruction has been issued by the user operating the operation device 13 (step S10). If an image capture start instruction has been issued (step S10: YES), the main control unit 50 controls the image capture control unit 51 to cause the image sensor 20 to perform an image capture operation (step S11).

[0076] The image acquisition unit 52 acquires one frame's worth of image signal D output from the imaging sensor 20 (step S12). The region determination unit 54 determines a set region RG based on output information from the operation device 13, and sets the set region RG within the image represented by the image signal D (step S13). The display control unit 53 causes the display 15 to display the image within the set region RG, out of the image represented by the image signal D (step S14).

[0077] The subject region acquisition unit 55 acquires a subject region RO including a target subject from the image represented by the image signal D (step S15). The exposure value calculation unit 56 performs an exposure value calculation process based on the image signal D and information representing the set region RG and the subject region RO (step S16). The main control unit 50 performs exposure control based on a third exposure value E3 calculated by the exposure value calculation process (step S17).

[0078] 12 shows an example of the flow of the exposure value calculation process. First, the first exposure value calculation unit 61 calculates the first exposure value E1 as described above (step S20). Next, the exposure value calculation unit 56 determines whether or not the subject area RO has been acquired by the subject area acquisition unit 55 (step S21). If the subject area RO has not been acquired (step S21: NO), the exposure value calculation unit 56 sets the first exposure value E1 calculated in step S20 as the third exposure value E3 (step S22) and ends the exposure value calculation process.

[0079] On the other hand, if the subject region RO is acquired (step S21: YES), the second exposure value calculation unit 62 calculates the second exposure value E2 as described above (step S23). Next, the first judgment unit 63 performs the first judgment as described above (step S24). The weight determination unit 65 determines the weight coefficient α based on the result of the first judgment as described above (step S25). For example, step S25 is performed by applying the ratio R1 obtained in the first judgment to the relationship between the weight coefficient α and the ratio R1 (see FIG. 9).

[0080] Next, the second judgment unit 64 performs the second judgment described above (step S26). The weight determination unit 65 judges whether or not the subject region RO is outside the set region RG based on the result of the second judgment (step S27). If the subject region RO is outside the set region RG (step S27: YES), the weight determination unit 65 determines the correction coefficient β based on the positional relationship between the set region RG and the subject region RO as described above (step S28). On the other hand, if the subject region RO is within the set region RG (step S27: NO), the weight determination unit 65 determines the correction coefficient β to be 1 (step S29). For example, steps S27 to S29 are performed by applying the ratio R2 obtained in the second judgment to the relationship between the correction coefficient β and the ratio R2 (see FIG. 10).

[0081] After step S28 or step S29, the weight determination unit 65 determines the first weight W1 and the second weight W2 based on the determined weighting coefficient α and correction coefficient β and the above formulas (1) and (2) (step S30). Then, the weighted addition unit 66 calculates the third exposure value E3 based on the determined first weight W1 and second weight W2 and the above formula (3) (step S31). This ends the exposure value calculation process.

[0082] As described above, the technology disclosed herein is characterized in that it determines a first weight W1 for the first exposure value E1 of the set area RG and a second weight W2 for the second exposure value E2 of the subject area RO based on a first judgment that judges the importance of the subject area RO relative to the set area RG and a second judgment that judges the positional relationship between the set area RG and the subject area RO, and calculates a third exposure value E3 by weighted addition of the first exposure value E1 and the second exposure value E2 using the first weight W1 and the second weight W2.

[0083] In the conventional technology, when calculating the third exposure value E3, which is the final exposure value, the positional relationship between the set area RG and the subject area RO is not taken into consideration, so the third exposure value E3 does not depend on whether the subject area RO is within the set area RG. Therefore, even if the subject area RO does not exist within the set area RG, the third exposure value E3 is affected by the second exposure value E2 of the subject area RO. For example, when displaying an image of the set area RG, even if the subject area RO, such as a person, does not exist within the set area RG and the scene within the set area RG does not change, if the subject area RO exists outside the set area RG as shown in FIG. 7, the brightness of the displayed image may vary due to the influence of the subject area RO. That is, in the conventional technology, the brightness of the set area RG may vary inappropriately due to the presence of the subject area RO outside the set area RG.

[0084] According to the technology of the present disclosure, the third exposure value E3 is calculated taking into consideration the positional relationship between the set area RG and the subject area RO, so that inappropriate fluctuations in the brightness of the set area RG can be suppressed.

[0085] Various modifications of the above embodiment will be described below.

[0086] [First Modification] In the above embodiment, the weight determination unit 65 determines the first weight W1 and the second weight W2 based on the results of the first judgment and the second judgment, but it may also determine the first exposure value E1 and the second exposure value E2 by further taking into account the first exposure value E1 and the second exposure value E2.

[0087] Fig. 13 shows the configuration of the exposure value calculation unit 56 according to the first modified example. As shown in Fig. 13, in this modified example, the weight determination unit 65 determines the first weight W1 and the second weight W2 using the first exposure value E1 and the second exposure value E2 in addition to the results of the first and second judgments. Other configurations of the exposure value calculation unit 56 according to this modified example are the same as those of the above embodiment.

[0088] Fig. 14 shows a flow of exposure value calculation processing according to Modification 1. The exposure value calculation processing according to this modification differs from the exposure value calculation processing according to the above embodiment shown in Fig. 12 in that steps S40 and S41 are added between steps S25 and S26.

[0089] After determining the weight coefficient α in step S25, the weight determination unit 65 determines whether the ratio R1 obtained in the first determination is less than a first threshold value (step S40). If the ratio R1 is less than the first threshold value, it means that the size of the subject region RO relative to the size of the set region RG is small, and the importance of the subject region RO is low. That is, in step S40, it is determined that the importance of the subject region RO is less than a certain value.

[0090] If the ratio R1 is not less than the first threshold value (step S40: NO), the weight determination unit 65 shifts the process to step S26. That is, if the importance of the subject region RO is high, the same process as in the above embodiment is performed.

[0091] On the other hand, if the ratio R1 is less than the first threshold (step S40: YES), the weight determination unit 65 determines whether the absolute value of the difference between the first exposure value E1 and the second exposure value E2 is less than the second threshold (step S41). The absolute value being less than the second threshold means that the luminance difference between the subject region RO and its background is small. That is, in step S41, it is determined that the luminance difference between the subject region RO and its background is less than a certain value.

[0092] If the absolute value is not less than the second threshold (step S41: NO), the weight determination unit 65 shifts the process to step S26. That is, even if the importance of the subject region RO is low, if the luminance difference between the subject region RO and the background is large, the same process as in the above embodiment is performed.

[0093] On the other hand, if the absolute value is less than the second threshold (step S41: YES), the weight determination unit 65 shifts the process to step S29 and determines the correction coefficient β to be 1. In other words, if the importance of the subject region RO is low and the luminance difference with respect to the background is small, the correction coefficient β is determined to be 1 regardless of the positional relationship between the setting region RG and the subject region RO, and the first weight W1 and the second weight W2 are determined.

[0094] As described above, in this modified example, when the ratio R1 of the size of the subject area RO to the size of the setting area RG is less than the first threshold value and the absolute value of the difference between the first exposure value E1 and the second exposure value E2 is less than the second threshold value, the weight determination unit 65 determines the first weight W1 and the second weight W2 regardless of the positional relationship between the setting area RG and the subject area RO.

[0095] When the importance of the subject region RO is low and the brightness difference between the subject region RO and the background is small, the third exposure value E3 is less affected by the second exposure value E2 of the subject region RO. However, in such a case, if the correction coefficient β is determined based on the positional relationship between the set region RG and the subject region RO, the third exposure value E3 changes according to the change in the positional relationship, and unnecessary flickering of the brightness of the set region RG may occur. In this modification, when the importance of the subject region RO is low and the brightness difference between the subject region RO and the background is small, the first weight W1 and the second weight W2 are determined regardless of the positional relationship, so that unnecessary flickering of the brightness of the set region RG can be suppressed.

[0096] [Second modified example] In the first modified example, the weight determination unit 65 determines the first exposure value E1 and the second exposure value E2 using the first exposure value E1 and the second exposure value E2 in addition to the results of the first judgment and the second judgment, but the first exposure value E1 and the second exposure value E2 may also be determined taking into account information about the subject included in the subject area RO.

[0097] The configuration of the exposure value calculation section 56 according to this modification is similar to that of the first modification, except that the weight determination section 65 acquires information on the subject included in the subject region RO from the subject region acquisition section 55.

[0098] Fig. 15 shows the flow of the exposure value calculation process according to the second modified example. The exposure value calculation process according to this modified example differs from the exposure value calculation process according to the first modified example shown in Fig. 14 in that step S42 is added between step S41 and step S29.

[0099] When the absolute value of the difference between the first exposure value E1 and the second exposure value E2 is less than the second threshold (step S41: YES), the weight determination unit 65 determines whether or not the subject included in the subject area RO is a specific subject (step S42). The type of specific subject is stored in advance as data in the memory 42. For example, the specific subject is an object such as a drone or a bird that moves at high speed. The specific subject is a subject that can move at high speed within the image represented by the image signal D. Note that the configuration may be such that the user can set the type of specific subject using the operation device 13. The specific subject refers to a subject that is arbitrarily selected by design or by the user.

[0100] If the subject included in the subject region RO is not a specific subject (step S42: NO), the process proceeds to step S26. That is, even if the importance of the subject region RO is low and the luminance difference between the subject region RO and its background is small, if the subject included in the subject region RO is not a specific subject, the same process as in the above embodiment is performed.

[0101] On the other hand, if the subject included in the subject region RO is a specific subject (step S42: YES), the process proceeds to step S29, and the correction coefficient β is determined to be 1. In other words, if the importance of the subject region RO is low, the luminance difference with respect to the background is small, and the subject included in the subject region RO is a specific subject, the correction coefficient β is determined to be 1 regardless of the positional relationship between the set region RG and the subject region RO, and the first weight W1 and the second weight W2 are determined.

[0102] As described above, in this modified example, when the ratio R1 of the size of the subject area RO to the size of the setting area RG is less than the first threshold value, the absolute value of the difference between the first exposure value E1 and the second exposure value E2 is less than the second threshold value, and the subject included in the subject area RO is a specific subject, the weight determination unit 65 determines the first weight W1 and the second weight W2 regardless of the positional relationship between the setting area RG and the subject area RO.

[0103] When the subject included in the subject area RO is a specific subject moving at high speed, the positional relationship between the setting area RG and the subject area RO is likely to change, which may cause unnecessary flickering in the brightness of the setting area RG. In this modified example, when the subject included in the subject area RO is a specific subject, the first weight W1 and the second weight W2 are determined regardless of the positional relationship between the setting area RG and the subject area RO, so that unnecessary flickering in the brightness of the setting area RG can be suppressed.

[0104] [Third Modification] In the above embodiment, the subject region acquisition unit 55 acquires the subject region RO by a template matching method, but the subject region RO may be acquired by using a machine learning model.

[0105] Fig. 16 shows a subject region acquisition process according to the third modified example. As shown in Fig. 16, the machine-learned model LM is configured by a neural network having an input layer, an intermediate layer, and an output layer. The intermediate layer is configured by a plurality of neurons. The number of intermediate layers and the number of neurons in each intermediate layer can be changed as appropriate. For example, the machine-learned model LM is a CNN. The machine-learned model LM is machine-learned using a plurality of images including a specific subject region as training data.

[0106] The subject region acquisition unit 55 inputs the image signal D to the machine-learned model LM. The machine-learned model LM identifies and outputs a subject region RO from within the image represented by the image signal D. For example, the machine-learned model LM outputs the result of segmentation for each pixel as a heat map. For example, when the acquisition target is a person's face, a set of pixels whose class belongs to the person's face is identified as the subject region RO.

[0107] In this way, by acquiring the subject region RO using the machine learning model, the subject region RO can be acquired with high accuracy, so that the positional relationship between the set region RG and the subject region RO can be determined with high accuracy in the first determination described above. This makes it possible to suppress inappropriate fluctuations in the brightness of the set region RG with high accuracy.

[0108] [Other variations] In the above embodiment, the setting area RG is an image display area for displaying the image in the setting area RG, but the setting area RG may be an image recording area for the image in the setting area RG. In this case, the image in the setting area RG is recorded in a storage device such as the memory 42 as moving image data.

[0109] The set area RG may be an image cutout area for electronic zoom. In this case, the size of the set area RG changes according to the zoom magnification of the electronic zoom, as shown in Fig. 17. Specifically, the set area RG becomes smaller as the zoom magnification of the electronic zoom increases.

[0110] The set area RG may be an image cutout area for camera shake correction. In this case, as shown in Fig. 17, the position of the set area RG is changed so as to cancel camera shake based on an output signal from an angular velocity sensor (not shown) such as a gyro sensor provided in the imaging device 10.

[0111] In the above embodiment, the first determination unit 63 determines the ratio R1 of the size of the subject area RO to the size of the set area RG as the importance of the subject area RO to the set area RG, but the importance is not limited to the ratio R1. For example, the importance may be the type of subject. In this case, the weight determination unit 65 increases the weight coefficient α for a type of subject having a higher importance.

[0112] In the above embodiment, the positional relationship between the set region RG and the subject region RO is determined based on the center of gravity P of the subject region RO, but the criterion for determining the positional relationship is not limited to the center of gravity P of the subject region RO. For example, if at least a part of the subject region RO is outside the set region RG, the subject region RO may be determined to be outside the set region RG, and if the entire subject region RO is inside the set region RG, the subject region RO may be determined to be inside the set region RG. Conversely, if at least a part of the subject region RO is inside the set region RG, the subject region RO may be determined to be inside the set region RG, and if the entire subject region RO is outside the set region RG, the subject region RO may be determined to be outside the set region RG.

[0113] In the above embodiment, the imaging device 10 is a digital camera, but the imaging device 10 may be a surveillance camera. In this case, the display 15 may be a monitor of a personal computer externally connected to the surveillance camera. The operation device 13 may be a keyboard, a mouse, or the like of the personal computer externally connected to the surveillance camera.

[0114] The technology of the present disclosure is also applicable to electronic devices such as smartphones and tablet terminals that have an imaging function.

[0115] In the above embodiment, the hardware structure of the control unit, with processor 40 being an example, may be any of the following various processors. The above various processors include a CPU, which is a general-purpose processor that functions by executing software (programs), as well as a processor such as an FPGA, whose circuit configuration can be changed after manufacture. FPGAs include dedicated electric circuits, which are processors having a circuit configuration designed specifically for executing specific processing, such as PLDs or ASICs.

[0116] The control unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple control units may be configured with one processor.

[0117] There are several possible examples of configuring multiple control units with one processor. A first example is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as clients and servers, and this processor functions as multiple control units. A second example is a form in which a processor is used that realizes the functions of the entire system including multiple control units with one IC chip, as typified by system on chip (SOC). In this way, the control unit can be configured as a hardware structure using one or more of the various processors described above.

[0118] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0119] The program may also be stored in a non-transitory computer-readable storage medium.

[0120] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0121] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[0122] The above explanation makes it possible to understand the following techniques. [Additional note 1] A processor and a memory, The processor, Acquire the image signal output from the imaging element, Obtaining a specific object region within an image represented by the image signal; Calculating a first exposure value for a set area set in the image and a second exposure value for the subject area; determining a first weight for the first exposure value and a second weight for the second exposure value based on a first determination of an importance of the subject region relative to the set region and a second determination of a positional relationship between the set region and the subject region; calculating a third exposure value by performing a weighted addition of the first exposure value and the second exposure value using the first weight and the second weight; Exposure control device. [Additional note 2] The processor, when it is determined in the second determination that the subject region is within the set region, the first weight and the second weight are determined regardless of the positional relationship. Item 1. An exposure control device according to claim 1. [Additional note 3] The processor, when it is determined in the second determination that the subject region is outside the set region, determining the first weight and the second weight based on the positional relationship. 3. An exposure control device according to claim 1 or 2. [Additional note 4] The processor, determining a weighting factor based on a result of the first determination; determining a correction coefficient based on a result of the second determination; Item 3. An exposure control device according to any one of items 1 to 3. [Additional note 5] When the first exposure value is E1, the second exposure value is E2, the third exposure value is E3, the first weight is W1, the second weight is W2, the weighting coefficient is α, and the correction coefficient is β, The processor, determining the first weight and the second weight so as to satisfy W1=1-α×β and W2=α×β; Calculate the third exposure value according to E3=E1×W1+E2×W2; 5. The exposure control device according to claim 4. [Additional note 6] The processor, When it is determined in the second determination that the subject area is within the set area, the correction coefficient is determined to be 1. 6. An exposure control device according to claim 4 or 5. [Additional note 7] The processor, When it is determined in the second determination that the subject region is outside the set region, the correction coefficient is determined based on the positional relationship. 7. An exposure control device according to claim 4, wherein the exposure control device is a light-emitting diode. [Additional note 8] The positional relationship includes a distance between the setting area and the subject area. 8. An exposure control device according to any one of claims 4 to 7. [Additional note 9] The correction coefficient decreases as the distance increases. Item 9. The exposure control device according to item 8. [Additional Note 10] The first determination is based on a ratio of a size of the subject region to a size of the set region. 10. An exposure control device according to claim 1, wherein the exposure control device is a light-emitting diode. [Additional Note 11] The processor, when a ratio of the size of the subject region to the size of the set region is less than a first threshold value and an absolute value of a difference between the first exposure value and the second exposure value is less than a second threshold value, the first weight and the second weight are determined regardless of the positional relationship. Item 11. An exposure control device according to item 10. [Additional Note 12] The processor, when a ratio of the size of the subject region to the size of the set region is less than a first threshold value, an absolute value of the difference between the first exposure value and the second exposure value is less than a second threshold value, and the subject included in the subject region is a specific subject, determining the first weight and the second weight regardless of the positional relationship. Item 11. An exposure control device according to item 10. [Additional Note 13] The subject area is a face area of ​​a person. 13. An exposure control device according to any one of claims 1 to 12. [Additional Note 14] The processor, acquiring the object region using a machine learning model; An exposure control device according to any one of claims 1 to 13. [Additional Note 15] The set area is an image display area for displaying an image in the set area, an image recording area for recording an image in the set area, or an image cut-out area for electronic zoom or image stabilization. An exposure control device according to any one of claims 1 to 14. [Additional Note 16] An exposure control device according to any one of claims 1 to 15; The imaging element; An imaging device comprising: [Explanation of symbols]

[0123] 10. Imaging device 11 Main unit 11A Camera side mount 11B Electrical contacts 12 Imaging lens 12A Lens side mount 12B Electrical contact 13 Operating device 14 Finder 15 Display 20 Image sensor 20A light receiving surface 30 Zoom Lens 31 Focus Lens 33 Driver 34 Lens drive control unit 40 processors 42 Memory 43 Programs 50 Main control unit 51 Imaging control unit 52 Image acquisition unit 53 Display control unit 54 Area determination part 55 Subject area acquisition section 56 Exposure value calculation section 61 First exposure value calculation unit 62 Second exposure value calculation unit 63 1st judgment part 64 Second Judgment Section 65 Weight determination unit 66 Weighted addition unit A optical axis B intersection D Image signal E1 First exposure value E2 Second exposure value E3 3rd exposure value LM Machine Learning Models O,P Center of gravity RG setting area RO object area W1 First weight W2 Second weight

Claims

1. A processor and a memory, The processor, Acquire the image signal output from the imaging element, Obtaining a specific object region within an image represented by the image signal; Calculating a first exposure value of a set area set in the image and a second exposure value of the subject area; determining a first weight for the first exposure value and a second weight for the second exposure value based on a first determination of an importance of the subject region with respect to the set region and a second determination of a positional relationship between the set region and the subject region; calculating a third exposure value by performing a weighted addition of the first exposure value and the second exposure value using the first weight and the second weight; Exposure control device.

2. The processor, when it is determined in the second determination that the subject region is within the set region, the first weight and the second weight are determined regardless of the positional relationship. The exposure control device according to claim 1 .

3. The processor, when it is determined in the second determination that the subject region is outside the set region, the first weight and the second weight are determined based on the positional relationship. The exposure control device according to claim 2 .

4. The processor, determining a weighting factor based on a result of the first determination; determining a correction coefficient based on a result of the second determination; The exposure control device according to claim 1 .

5. When the first exposure value is E1, the second exposure value is E2, the third exposure value is E3, the first weight is W1, the second weight is W2, the weighting coefficient is α, and the correction coefficient is β, The processor, determining the first weight and the second weight so as to satisfy W1=1-α×β and W2=α×β; Calculate the third exposure value based on E3=E1×W1+E2×W2; The exposure control device according to claim 4 .

6. The processor, When it is determined in the second determination that the subject area is within the set area, the correction coefficient is determined to be 1. The exposure control device according to claim 5 .

7. The processor, when it is determined in the second determination that the subject region is outside the set region, determining the correction coefficient based on the positional relationship. The exposure control device according to claim 6 .

8. The positional relationship includes a distance between the setting area and the subject area. The exposure control device according to claim 7 .

9. The correction coefficient decreases as the distance increases. The exposure control device according to claim 8 .

10. the first determination is based on a ratio of a size of the subject region to a size of the set region; The exposure control device according to claim 1 .

11. The processor, when a ratio of a size of the subject region to a size of the set region is less than a first threshold value and an absolute value of a difference between the first exposure value and the second exposure value is less than a second threshold value, the first weight and the second weight are determined regardless of the positional relationship. The exposure control device according to claim 10.

12. The processor, determining the first weight and the second weight regardless of the positional relationship when a ratio of the size of the subject region to the size of the set region is less than a first threshold value, an absolute value of a difference between the first exposure value and the second exposure value is less than a second threshold value, and a subject included in the subject region is a specific subject. The exposure control device according to claim 10.

13. The subject area is a face area of ​​a person. The exposure control device according to claim 1 .

14. The processor, acquiring the object region using a machine learning model; The exposure control device according to claim 1 .

15. The set area is an image display area for displaying an image in the set area, an image recording area for recording an image in the set area, or an image cut-out area for electronic zoom or image stabilization. The exposure control device according to claim 1 .

16. An exposure control device according to any one of claims 1 to 15; The imaging element; An imaging device comprising:

17. The processor: Obtaining an image signal output from the imaging element; acquiring a particular subject area within an image represented by said image signal; Calculating a first exposure value of a set area set in the image and a second exposure value of the subject area; determining a first weight for the first exposure value and a second weight for the second exposure value based on a first determination of an importance of the subject region with respect to the set region and a second determination of a positional relationship between the set region and the subject region; calculating a third exposure value by performing a weighted addition of the first exposure value and the second exposure value using the first weight and the second weight; 23. An exposure control method comprising:

18. Obtaining an image signal output from the imaging element; acquiring a particular subject area within an image represented by said image signal; Calculating a first exposure value of a set area set in the image and a second exposure value of the subject area; determining a first weight for the first exposure value and a second weight for the second exposure value based on a first determination of an importance of the subject region with respect to the set region and a second determination of a positional relationship between the set region and the subject region; calculating a third exposure value by performing a weighted addition of the first exposure value and the second exposure value using the first weight and the second weight; A program that causes a processor to execute a process including the steps of:

Images