Electronic apparatus

Abstract

To set an appropriate imaging condition in an imaging region in which different object images are mixed.SOLUTION: An electronic apparatus includes: an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging an object; and a setting unit for setting a first imaging condition for a first imaging region where a first object image exists among the plurality of imaging regions, setting a second imaging condition different from the first imaging condition for a second imaging region where a second object image exists among the plurality of imaging regions, and setting a third imaging condition for a third imaging region where the first object image and the second object image exists among the plurality of imaging regions on the basis of at least one of the first imaging condition and the second imaging condition and a luminance value based on output from the third imaging region.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to electronic devices. [Background technology]

[0002] There is known an imaging device equipped with an imaging element that can set different imaging conditions for each area of ​​a screen (see Patent Document 1). However, the imaging conditions are not taken into consideration when processing image data generated for areas with different imaging conditions. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-197192 A Summary of the Invention

[0004] The electronic device disclosed in the present application has an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions that capture an image of a subject, and a setting unit that sets a first imaging condition in a first imaging region among the plurality of imaging regions in which a first subject image is present, sets a second imaging condition different from the first imaging condition in a second imaging region among the plurality of imaging regions in which a second subject image is present, and sets a third imaging condition for a third imaging region among the plurality of imaging regions in which the first subject image and the second subject image are present, based on at least one of the first imaging condition and the second imaging condition and a brightness value based on an output from the third imaging region.

[0005] The electronic device disclosed in the present application has an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions that capture an image of a subject, and a setting unit that sets a first imaging condition in a first imaging region among the plurality of imaging regions in which a first subject image is present, sets a second imaging condition different from the first imaging condition in a second imaging region among the plurality of imaging regions in which a second subject image is present, and sets a third imaging condition for a third imaging region among the plurality of imaging regions in which the first subject image and the second subject image are present, based on at least one of the first imaging condition and the second imaging condition.

[0006] The electronic device disclosed in the present application includes an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions that capture an image of a subject, and a setting unit that sets a first imaging condition for a first imaging region among the plurality of imaging regions, sets a second imaging condition for a second imaging region among the plurality of imaging regions, and sets an imaging condition between the first imaging condition and the second imaging condition for a third imaging region among the plurality of imaging regions that is sandwiched between the first imaging region and the second imaging region.

[0007] The electronic device disclosed in the present application has an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject, and a setting unit that sets a first imaging condition to an imaging region among the plurality of imaging regions in which a first subject image is present, and sets a second imaging condition that is different from the first imaging condition and in which image signals of the first subject image and the second subject image different from the first subject image are not saturated to an imaging region among the plurality of imaging regions in which the first subject image and a second subject image different from the first subject image are present.

[0008] The electronic device disclosed in the present application includes an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions that capture an image of a subject, and a setting unit that sets a first imaging condition in a first imaging region among the plurality of imaging regions, sets a second imaging condition different from the first imaging condition in a second imaging region among the plurality of imaging regions, and sets a third imaging condition in a third imaging region among the plurality of imaging regions, the third imaging condition being such that a pixel signal having a magnitude between that output under the first imaging condition and that output under the second imaging condition is output.

[0009] The electronic device disclosed in the present application comprises an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging a subject, and a setting unit that sets a first imaging condition in a first imaging region among the plurality of imaging regions in which a first subject image is present, sets a second imaging condition different from the first imaging condition in a second imaging region among the plurality of imaging regions in which a second subject image is present, and sets a third imaging condition in a third imaging region among the plurality of imaging regions sandwiched between the first imaging region and the second imaging region, where a pixel signal having a magnitude between that output under the first imaging condition and that output under the second imaging condition is output.

[0010] A setting program which is one aspect of the invention disclosed in the present application is a setting program which causes a processor to execute setting control of an image sensor which can set different imaging conditions for each of a plurality of imaging regions which capture an image of a subject, and causes the processor to set a first imaging condition for a first imaging region among the plurality of imaging regions in which a first subject image is present, set a second imaging condition different from the first imaging condition for a second imaging region among the plurality of imaging regions in which a second subject image is present, and set a third imaging condition for a third imaging region among the plurality of imaging regions in which the first subject image and the second subject image are present, based on at least one of the first imaging condition and the second imaging condition. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 is a cross-sectional view of a stacked type imaging element. [Diagram 2] FIG. 2 is a diagram illustrating the pixel arrangement of the imaging chip. [Diagram 3] FIG. 3 is a circuit diagram of the imaging chip. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of an imaging element. [Diagram 5] FIG. 5 is an explanatory diagram illustrating an example of a block configuration of an electronic device. [Figure 6] FIG. 6 is an explanatory diagram showing an example of the structure of an image file generated by capturing an image with an electronic device. [Figure 7] FIG. 7 is an explanatory diagram showing the relationship between the imaging surface and the subject image. [Figure 8] FIG. 8 is an explanatory diagram showing a specific example of the structure of an image file. [Figure 9] FIG. 9 is an explanatory diagram illustrating an example of a detailed block configuration of the electronic device. [Figure 10] FIG. 10 is an explanatory diagram showing a first setting example of the third imaging condition. [Figure 11] FIG. 11 is an explanatory diagram showing a second setting example of the third imaging condition. [Figure 12] FIG. 12 is an explanatory diagram showing a third setting example of the third imaging condition. [Figure 13] FIG. 13 is an explanatory diagram showing the degree of influence on the coexistence region. [Figure 14] FIG. 14 is an explanatory diagram showing an example of a setting unit of an indefinite block. [Figure 15] FIG. 15 is an explanatory diagram showing the boundary length of the coexistence region. [Figure 16] FIG. 16 is an explanatory diagram showing reduction in the effect of halos. [Figure 17] FIG. 17 is a graph showing an example of a prediction of halo occurrence using the γ function. [Figure 18] FIG. 18 is a flowchart showing a first example of a processing procedure for setting image capturing conditions in an electronic device. [Figure 19] FIG. 19 is a flowchart illustrating a detailed process procedure example 1 of the area priority setting process (step S1808) illustrated in FIG. [Figure 20] FIG. 20 is a flowchart illustrating a detailed process procedure example 2 of the area priority setting process (step S1808) illustrated in FIG. [Figure 21] FIG. 21 is a flowchart showing a detailed processing procedure example 2-1 of the area priority setting process (step S1808) shown in FIG. [Figure 22] FIG. 22 is a flowchart showing a detailed processing procedure example 2-2 of the area priority setting process (step S1808) shown in FIG. [Diagram 23] FIG. 23 is a flowchart showing a detailed processing procedure example 1 of the halo countermeasure processing (step S2202) shown in FIG. [Figure 24] FIG. 24 is a flowchart illustrating a detailed processing procedure example 2-3 of the area priority setting process (step S1808) illustrated in FIG. [Diagram 25] FIG. 25 is a flowchart illustrating a detailed process procedure example 1 of the complexity priority setting process (step S1809) illustrated in FIG. [Figure 26] FIG. 26 is a flowchart illustrating a detailed processing procedure example 2 of the complexity priority setting process (step S1809) illustrated in FIG. [Figure 27] FIG. 27 is a flowchart illustrating a second example of a processing procedure for setting image capturing conditions in an electronic device. [Figure 28] FIG. 28 is a flowchart illustrating a detailed processing procedure example 2 of the halo countermeasure processing (step S2202) illustrated in FIG. [Figure 29] FIG. 29 is a flowchart illustrating a detailed example of a processing procedure of the halo countermeasure processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS EXAMPLES

[0012] <Example of imaging element configuration> The imaging element mounted on the electronic device shown in the examples of this specification is a stacked imaging element. This stacked imaging element allows different imaging conditions to be set for each of a plurality of different imaging regions. First, the structure of the stacked imaging element that allows different imaging conditions to be set for each of a plurality of different imaging regions will be described. This stacked imaging element is described in Japanese Patent Application No. 2012-139026, previously filed by the applicant of this application. The electronic device is, for example, an imaging device such as a digital camera or a digital video camera.

[0013] 1 is a cross-sectional view of a stacked imaging element 100. The stacked imaging element (hereinafter simply referred to as "imaging element") 100 includes a back-illuminated imaging chip (hereinafter simply referred to as "imaging chip") 113 that outputs pixel signals corresponding to incident light, a signal processing chip 111 that processes the pixel signals, and a memory chip 112 that stores the pixel signals. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by conductive bumps 109 such as Cu.

[0014] As shown in FIG. 1, incident light is mainly incident in the Z-axis positive direction indicated by the white arrow. In this embodiment, the surface of the imaging chip 113 on which the incident light is incident is referred to as the back surface. As shown in the coordinate axes 120, the left direction on the paper surface perpendicular to the Z-axis is the X-axis positive direction, and the front direction on the paper surface perpendicular to the Z-axis and the X-axis is the Y-axis positive direction. In the following figures, the coordinate axes 120 in FIG. 1 are used as a reference, and the coordinate axes 120 are displayed so that the orientation of each figure can be understood.

[0015] An example of the imaging chip 113 is a back-illuminated MOS (Metal Oxide Semiconductor) image sensor. A PD (photodiode) layer 106 is disposed on the back side of a wiring layer 108. The PD layer 106 includes a plurality of PDs 104 that are two-dimensionally disposed and accumulate electric charges according to incident light, and transistors 105 that correspond to the PDs 104.

[0016] A color filter 102 is provided on the incident side of the PD layer 106 where the incident light is incident, via a passivation film 103. There are a plurality of types of color filters 102 that transmit different wavelength regions, and each has a specific arrangement corresponding to each PD 104. The arrangement of the color filters 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

[0017] A microlens 101 is provided corresponding to each pixel on the incident light side of the color filter 102. The microlens 101 condenses the incident light toward the corresponding PD 104.

[0018] The wiring layer 108 has wiring 107 that transmits pixel signals from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multi-layered, and may include passive and active elements.

[0019] A plurality of bumps 109 are arranged on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with a plurality of bumps 109 provided on the opposing surface of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed together, whereby the aligned bumps 109 are joined together and electrically connected.

[0020] Similarly, a plurality of bumps 109 are arranged on the opposing surfaces of the signal processing chip 111 and the memory chip 112. These bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressed together, whereby the aligned bumps 109 are bonded together and electrically connected.

[0021] The bonding between the bumps 109 is not limited to Cu bump bonding by solid-phase diffusion, and micro-bump bonding by solder melting may be adopted. Also, it is sufficient to provide about one bump 109 for each block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Also, in a peripheral region other than the pixel region where the pixels are arranged, a bump larger than the bump 109 corresponding to the pixel region may be provided.

[0022] The signal processing chip 111 has TSVs (Through Silicon Vias) 110 that connect the circuits provided on the front and back surfaces of the chip to each other. The TSVs 110 are preferably provided in the peripheral region. The TSVs 110 may also be provided in the peripheral region of the imaging chip 113 and the memory chip 112.

[0023] 2 is a diagram for explaining the pixel arrangement of the imaging chip 113. In particular, the imaging chip 113 is shown as observed from the back side. (a) is a plan view that shows a schematic image pickup surface 200, which is the back side of the imaging chip 113, and (b) is an enlarged plan view of a partial region 200a of the imaging surface 200. As shown in (b), a large number of pixels 201 are two-dimensionally arranged on the imaging surface 200.

[0024] Each pixel 201 has a color filter (not shown). The color filters are of three types: red (R), green (G), and blue (B), and the notations "R", "G", and "B" in (b) indicate the type of color filter that the pixel 201 has. As shown in (b), the pixels 201 equipped with such color filters are arranged in a so-called Bayer array on the imaging surface 200 of the image sensor 100.

[0025] The pixel 201 having a red filter photoelectrically converts light in the red wavelength band of the incident light and outputs a light reception signal (photoelectric conversion signal). Similarly, the pixel 201 having a green filter photoelectrically converts light in the green wavelength band of the incident light and outputs a light reception signal. Moreover, the pixel 201 having a blue filter photoelectrically converts light in the blue wavelength band of the incident light and outputs a light reception signal.

[0026] The image sensor 100 is configured to be able to individually control each block 202 consisting of a total of four pixels 201, ie, 2 pixels by 2 pixels adjacent to each other. For example, when charge accumulation is started simultaneously for two different blocks 202, charge reading, i.e., light reception signal reading, can be performed 1 / 30 seconds after the start of charge accumulation in one block 202, and charge reading can be performed 1 / 15 seconds after the start of charge accumulation in the other block 202. In other words, the image sensor 100 can set a different exposure time (charge accumulation time, so-called shutter speed) for each block 202 in one image capture.

[0027] In addition to the above-mentioned exposure time, the image sensor 100 can vary the amplification rate of the image signal (so-called ISO sensitivity) for each block 202. The image sensor 100 can change the timing for starting charge accumulation and the timing for reading out the light receiving signal for each block 202. In other words, the image sensor 100 can change the frame rate for capturing a moving image for each block 202.

[0028] In summary, the image sensor 100 is configured to be able to vary imaging conditions such as exposure time, amplification factor, and frame rate for each block 202. For example, if a readout line (not shown) for reading out an imaging signal from a photoelectric conversion unit (not shown) of the pixel 201 is provided for each block 202 and the imaging signal can be read out independently for each block 202, it is possible to vary the exposure time (shutter speed) for each block 202.

[0029] In addition, if an amplifier circuit (not shown) that amplifies the image signal generated from the photoelectrically converted charge is provided independently for each block 202 and the amplification rate of the amplifier circuit is configured to be controllable independently for each amplifier circuit, the signal amplification rate (ISO sensitivity) can be made different for each block 202.

[0030] Furthermore, imaging conditions that can be made different for each block 202 include, in addition to the imaging conditions described above, the frame rate, gain, resolution (thinning rate), the number of rows or columns for adding pixel signals, the charge accumulation time or number of accumulations, the number of digitization bits, etc. Furthermore, the control parameters may be parameters for image processing after image signals are obtained from the pixels.

[0031] In addition, the imaging conditions can be controlled by, for example, providing the image sensor 100 with a liquid crystal panel having independently controllable sections (each section corresponding to one block 202) for each block 202, and using this as a neutral density filter that can be turned on and off, thereby making it possible to control the brightness (aperture value) for each block 202.

[0032] The number of pixels 201 constituting the block 202 does not have to be the above-mentioned 2×2=4 pixels. The block 202 only needs to have at least two pixels 201, and conversely, the block 202 may have more than four pixels 201.

[0033] 3 is a circuit diagram of the imaging chip 113. In FIG. 3, a rectangle surrounded by a dotted line representatively represents a circuit corresponding to one pixel 201. A rectangle surrounded by a dashed line corresponds to one block 202 (202-1 to 202-4). At least a part of the transistors described below corresponds to the transistor 105 in FIG. 1.

[0034] As described above, the reset transistors 303 of the pixels 201 are turned on / off in units of blocks 202. The transfer transistors 302 of the pixels 201 are also turned on / off in units of blocks 202. In the example shown in Fig. 3, a reset wiring 300-1 is provided for turning on / off the four reset transistors 303 corresponding to the upper left block 202-1, and a TX wiring 307-1 is also provided for supplying a transfer pulse to the four transfer transistors 302 corresponding to the same block 202-1.

[0035] Similarly, a reset wiring 300-3 for turning on / off the four reset transistors 303 corresponding to the lower left block 202-3 is provided separately from the reset wiring 300-1. Also, a TX wiring 307-3 for supplying transfer pulses to the four transfer transistors 302 corresponding to the same block 202-3 is provided separately from the TX wiring 307-1.

[0036] Similarly, in the upper right block 202-2 and the lower right block 202-4, a reset line 300-2 and a TX line 307-2, and a reset line 300-4 and a TX line 307-4 are provided in each block 202, respectively.

[0037] The 16 PDs 104 corresponding to each pixel 201 are connected to a corresponding transfer transistor 302. A transfer pulse is supplied to the gate of each transfer transistor 302 via the TX wiring for each block 202. The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD between the drain of the transfer transistor 302 and the source of the reset transistor 303 is connected to the gate of the corresponding amplification transistor 304.

[0038] The drains of the reset transistors 303 are commonly connected to a Vdd wiring 310 to which a power supply voltage is supplied. A reset pulse is supplied to the gates of the reset transistors 303 via the reset wiring for each block 202.

[0039] The drains of the amplification transistors 304 are commonly connected to a Vdd wiring 310 to which a power supply voltage is supplied. In addition, the source of each amplification transistor 304 is connected to the drain of the corresponding selection transistor 305. The gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. The decoder wiring 308 is provided independently for each of the 16 selection transistors 305.

[0040] The sources of the selection transistors 305 are connected to a common output wiring 309. The load current source 311 supplies a current to the output wiring 309. That is, the output wiring 309 for the selection transistors 305 is formed by a source follower. The load current source 311 may be provided on the imaging chip 113 side or on the signal processing chip 111 side.

[0041] Here, the flow from the start of charge accumulation to pixel output after accumulation is completed will be described. When a reset pulse is applied to the reset transistor 303 through the reset wiring for each block 202, and at the same time a transfer pulse is applied to the transfer transistor 302 through the TX wiring for each block 202 (202-1 to 202-4), the potentials of the PD 104 and the floating diffusion FD are reset for each block 202.

[0042] When the transfer pulse is released, each PD 104 converts the incident light it receives into electric charges and accumulates them. After that, when the transfer pulse is applied again without the reset pulse being applied, the accumulated electric charges are transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after the electric charges are accumulated.

[0043] When a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, a fluctuation in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 through the amplification transistor 304 and the selection transistor 305. As a result, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

[0044] As described above, the reset wiring and TX wiring are common to the four pixels forming a block 202. That is, the reset pulse and the transfer pulse are each applied simultaneously to the four pixels in the same block 202. Therefore, all the pixels 201 forming a certain block 202 start and end charge accumulation at the same timing. However, pixel signals corresponding to the accumulated charges are selectively output from the output wiring 309 by sequentially applying selection pulses to the respective selection transistors 305.

[0045] In this manner, the charge accumulation start timing can be controlled for each block 202. In other words, imaging can be performed at different timings between different blocks 202.

[0046] 4 is a block diagram showing an example of the configuration of the image sensor 100. An analog multiplexer 411 sequentially selects 16 PDs 104 forming a block 202 and outputs the respective pixel signals to output wiring 309 provided corresponding to the block 202. The multiplexer 411 is formed in the image sensor chip 113 together with the PDs 104.

[0047] The pixel signals output via the multiplexer 411 undergo correlated double sampling (CDS) and analog-to-digital (A / D) conversion by a signal processing circuit 412 that performs CDS and analog-to-digital (A / D) conversion and is formed in the signal processing chip 111. The A / D converted pixel signals are passed to a demultiplexer 413 and stored in pixel memories 414 corresponding to the respective pixels. The demultiplexer 413 and pixel memories 414 are formed in the memory chip 112.

[0048] The arithmetic circuit 415 processes the pixel signals stored in the pixel memory 414 and passes them to a downstream image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or in the memory chip 112. Note that although the connections for four blocks 202 are shown in Fig. 4, in reality, these exist for every four blocks 202 and operate in parallel.

[0049] However, it is not necessary for there to be a calculation circuit 415 for every four blocks 202. For example, one calculation circuit 415 may perform sequential processing by referring to the values ​​of the pixel memories 414 corresponding to each of the four blocks 202 in order.

[0050] As described above, output wiring 309 is provided corresponding to each block 202. Since the imaging element 100 has the imaging chip 113, the signal processing chip 111, and the memory chip 112 stacked on top of each other, by using the bumps 109 for electrical connection between the chips for these output wiring 309, it is possible to route the wiring without increasing the size of each chip in the planar direction.

[0051] <Example of electronic device block configuration> Fig. 5 is an explanatory diagram showing an example of a block configuration of an electronic device. Details of the example block configuration will be described later (Fig. 9). The electronic device 500 is, for example, a lens-integrated camera. The electronic device 500 includes an imaging optical system 501, an imaging element 100, a control unit 502, a liquid crystal monitor 503, a memory card 504, an operation unit 505, a DRAM 506, a flash memory 507, and an audio recording unit 508. The control unit 502 includes a detection unit that detects camera shake and subject blur, as described later.

[0052] The imaging optical system 501 is made up of a plurality of lenses, and forms a subject image on the imaging surface 200 of the image sensor 100. For convenience, the imaging optical system 501 is illustrated as a single lens in FIG.

[0053] The imaging element 100 is, for example, an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device), and captures an object image formed by an imaging optical system 501 and outputs an imaging signal. The control unit 502 is an electronic circuit that controls each unit of the electronic device 500, and includes a processor that executes a program, peripheral circuits such as an image processing circuit, and various sensors such as an acceleration sensor.

[0054] A predetermined control program is written in advance in flash memory 507, which is a non-volatile storage medium. The processor of control unit 502 controls each unit by reading and executing the control program from flash memory 507. This control program uses DRAM 506, which is a volatile storage medium, as a working area.

[0055] The liquid crystal monitor 503 is a display device that uses a liquid crystal panel. The control unit 502 causes the image sensor 100 to repeatedly capture an image of a subject at predetermined intervals (for example, 1 / 60th of a second). Then, various types of image processing are performed on the image signal output from the image sensor 100 to create a so-called through image, which is displayed on the liquid crystal monitor 503. In addition to the through image, the liquid crystal monitor 503 also displays, for example, a setting screen for setting image capturing conditions.

[0056] The control unit 502 creates an image file (described later) based on the imaging signal output from the imaging element 100, and records the image file on a memory card 504, which is a portable recording medium. The operation unit 505 has various operation members such as push buttons, and outputs operation signals to the control unit 502 in response to the operation of these operation members.

[0057] Recording unit 508 is, for example, configured with a microphone, and converts environmental sounds into audio signals and inputs them to control unit 502. Note that control unit 502 may record the video file on a recording medium (not shown), such as a solid state drive (SSD) or hard disk built into electronic device 500, instead of recording the video file on memory card 504, which is a portable recording medium.

[0058] <Example of image file configuration> 6 is an explanatory diagram showing an example of the configuration of an image file generated by imaging by electronic device 500. Image file 600 is generated by an image processing circuit or an image processing program in control unit 502, and stored in memory card 504, DRAM 506, or flash memory 507. Image file 600 is composed of two blocks, a header section 601 and a data section 602. Header section 601 is a block located at the beginning of image file 600. In header section 601, file basic information area 611, mask area 612, and imaging information area 613 are stored in the order described above.

[0059] In the file basic information area 611, for example, the size and offset of each section (header section 601, data section 602, mask area 612, imaging information area 613, etc.) in the image file 600 are recorded. In the mask area 612, imaging condition information and mask information, which will be described later, are recorded. In the imaging information area 613, information related to imaging, such as the model name of the electronic device 500 and information about the imaging optical system 501 (for example, information about optical characteristics such as aberration), are recorded. The data section 602 is a block located after the header section 601, and image information is recorded therein.

[0060] <Relationship between the imaging surface 200 and the subject image> 7 is an explanatory diagram showing the relationship between the imaging plane 200 and a subject image. (a) shows an example of an image of a subject on the imaging plane 200 (imaging range) of the imaging element 100. (b) shows mask information set when imaging (a). (c) shows an example of an image of a subject on the imaging plane 200 (imaging range) of the imaging element 100 a predetermined time after (a). (d) shows mask information set when imaging (c).

[0061] In (a), the control unit 502 captures an image of a subject. The image capture in (a) may be, for example, an image capture performed to create a live view image (a so-called through image). The control unit 502 executes a predetermined image analysis process on the subject image obtained by the image capture in (a). The image analysis process detects a main subject, for example, by a well-known subject detection technique (a technique for calculating a feature amount and detecting an area in which a predetermined subject exists). In the first embodiment, everything other than the main subject is considered to be background.

[0062] A block 202 that shows only an image of the detected main subject (main subject image MI) becomes a main subject block 202. A collection of main subject blocks 202 becomes a main subject region 702. A block 202 that shows only an image of the background becomes a background block 202. A collection of background blocks 202 becomes a background region 701. A block 202 that shows both a main subject image MI and an image of the background is called an indefinite block 202. A collection of indefinite blocks 202 is called a coexistence region 703.

[0063] Specifically, in (a), for example, the main subject region 702 is set to a shape that follows the outline of the main subject image MI. The control unit 502 sets different imaging conditions for each block 202 in the main subject region 702 and each block 202 in the background region 701. For example, a faster shutter speed is set for each of the former blocks 202 compared to each of the latter blocks 202. In this way, image blurring is less likely to occur in the main subject region 702 in the imaging of (c) that is captured following the imaging of (a).

[0064] Furthermore, when the main subject region 702 is backlit due to the influence of a light source such as the sun present in the background region 701, the control unit 502 sets a relatively high ISO sensitivity and a slow shutter speed for each of the former blocks 202. Furthermore, the control unit 502 sets a relatively low ISO sensitivity and a fast shutter speed for each of the latter blocks 202. In this way, in the image capture of (c), it is possible to prevent crushed black in the main subject region 702 that is backlit and blown white in the background region 701 with a large amount of light. The settings for the blocks 202 that constitute the coexistence region 703 will be described later.

[0065] The image analysis process may be a process different from the process of detecting the background region 701 and the main subject region 702 described above. For example, the image analysis process may be a process of detecting a portion of the entire imaging surface 200 whose brightness is equal to or higher than a certain level (a portion that is too bright) and a portion of the entire imaging surface 200 whose brightness is lower than a certain level (a portion that is too dark). When the image analysis process is such a process, the control unit 502 sets the shutter speed and the ISO sensitivity for the block 202 included in the former region so that the exposure value (Ev value) is lower than that for the block 202 included in the other region.

[0066] Furthermore, the control unit 502 sets the shutter speed and ISO sensitivity for the blocks 202 included in the latter region so that the exposure value (Ev value) is higher than that for the blocks 202 included in the other regions. In this way, the dynamic range of the image obtained by capturing the image of (c) can be made wider than the inherent dynamic range of the image sensor 100.

[0067] 7B shows an example of mask information 704 corresponding to the imaging plane 200 shown in (a). A "1" is stored in the position of the block 202 belonging to the background region 701, a "2" is stored in the position of the block 202 belonging to the main subject region 702, and a "0" is stored in the position of the block 202 belonging to the coexistence region 703.

[0068] However, the position of the block 202 belonging to the coexistence region 703 may be updated to "2" when the control unit 502 determines that the block 202 should belong to the main subject region 702, or to "1" when the control unit 502 determines that the block 202 should belong to the background region 701. The control unit 502 may also store the value "0" before the update and the value "1" or "2" after the update.

[0069] The control unit 502 executes image analysis processing on the image data of the first frame to detect a background region 701, a main subject region 702, and a coexistence region 703. As a result, the frame captured in (a) is divided into a background region 701, a main subject region 702, and a coexistence region 703 as shown in (b). The control unit 502 sets different imaging conditions for each block 202 in the background region 701 and each block 202 in the main subject region 702, captures the image in (c), and creates image data. An example of mask information 704 at this time is shown in (d).

[0070] Mask information 704 (b) corresponding to the result of imaging (a) and mask information 704 (d) corresponding to the result of imaging (c) are captured at different times (there is a time difference), so that, for example, when the subject moves or when the user moves electronic device 500, these two pieces of mask information 704 have different contents. In other words, mask information 704 is dynamic information that changes with the passage of time. Therefore, in a certain block 202, different imaging conditions are set for each frame.

[0071] <Examples of image files> 8 is an explanatory diagram showing a specific example of the configuration of the image file 600. In the mask area 612, the identification information 801, the image capturing condition information 802, and the mask information 704 are recorded in the order described above.

[0072] The identification information 801 indicates that this image file 600 was created by a multi-imaging condition still image capturing function. The multi-imaging condition still image capturing function is a function for capturing still images using the image sensor 100 with a plurality of different imaging conditions (for example, a long exposure time and a short exposure time) set. For example, in backlit conditions, the main subject is exposed at a long exposure time and the background at a short exposure time. The long and short exposure times are set values ​​that are determined relatively.

[0073] The imaging condition information 802 is information that indicates what kind of use (purpose, role) exists in the block 202. For example, as described above, when the imaging plane 200 (FIG. 7(a)) is divided into a background region 701, a main subject region 702, and a coexistence region 703, if the coexistence region 703 remains undefined, each block 202 belongs to either the background region 701, the main subject region 702, or the coexistence region 703. On the other hand, if the coexistence region 703 is determined to be either the background region 701 or the main subject region 702, each block 202 belongs to either the background region 701 or the main subject region 702.

[0074] That is, when creating this image file 600, the imaging condition information 802 includes, in block 202, two types of imaging conditions, for example, "capture a still image of the main subject region at 1 / 60 [sec]" (long second) and "capture a still image of the background region at 1 / 1000 [sec]" (short second), and information indicating a unique number assigned to each of these imaging conditions. For example, the number "1" indicates the imaging condition "capture a still image of the background region at 1 / 1000 [sec]". The number "2" indicates the imaging condition "capture a still image of the main subject region at 1 / 60 [sec]". The number "0" indicates the imaging condition "the exposure time for capturing a still image of the coexistence region is indefinite".

[0075] Here, for ease of explanation, the unique numbers assigned to each imaging condition are "1" and "2", but three or more types of exposure time may be assigned. Therefore, for example, a number corresponding to the number of stages indicating the exposure time may be assigned.

[0076] 7, the mask information 704 is "information in which the numbers assigned to the imaging condition information 802 are expressed in the form of a two-dimensional map in accordance with the positions of the blocks 202." In other words, when the two-dimensionally arranged blocks 202 are specified by two-dimensional coordinates (x, y) defined by two integers x and y, the use of the block 202 at the position (x, y) is expressed by the imaging number at the position (x, y) of the mask information 704.

[0077] Imaging number "1" is the number assigned to block 202 in which a background image exists, imaging number "2" is the number assigned to block 202 in which a main subject image exists, and imaging number "0" is the number assigned to block 202 in which a background image and a main subject image exist. For example, if imaging number "2" is entered at coordinate (3,5) in mask information 704, it can be seen that the imaging condition "capture a still image of the main subject region in 1 / 60 [sec]" has been given to block 202 located at coordinate (3,5). In other words, it can be seen that block 202 located at coordinate (3,5) belongs to main subject region 702.

[0078] The data section 602 includes image information 811 , a Tv value map 812 , an Sv value map 813 , a Bv value map 814 , and Av value information 815 .

[0079] Image information 811 is information in which the imaging signal output from the imaging element 100 by the imaging of FIG. 7B is recorded in a form before various image processing is performed, and is so-called RAW image data.

[0080] The Tv value map 812 is information in which the Tv values, which indicate the shutter speeds set for the blocks 202, are expressed in the form of a two-dimensional map according to the positions of the blocks 202. For example, the shutter speed set for the block 202 located at the coordinates (x, y) can be determined by checking the Tv value stored at the coordinates (x, y) of the Tv value map 812.

[0081] The Sv value map 813 is information in which the Sv value, which indicates the ISO sensitivity set for each block 202, is expressed in the form of a two-dimensional map, similar to the Tv value map 812.

[0082] The Bv value map 814 is information that represents the subject brightness measured for each block 202 during imaging in FIG. 7(b), i.e., the Bv value representing the brightness of the subject light incident on each block 202, in the form of a two-dimensional map, similar to the Tv value map 812.

[0083] The Av value information 815 is information that indicates the aperture value at the time of capturing the image in (b) of Fig. 7. Unlike the Tv, Sv, and Bv values, the Av value is not a value that exists for each block 202. Therefore, unlike the Tv, Sv, and Bv values, the Av value stores only a single value, and is not information in which multiple values ​​are mapped two-dimensionally.

[0084] As described above, the control unit 502 performs imaging using the multi-imaging condition still image imaging function, and records on the memory card 504 an image file 600 in which image information 811 generated by the imaging element 100, in which different imaging conditions can be set for each block 202, and data relating to the imaging conditions for each block 202 (imaging condition information 802, mask information 704, Tv value map 812, Sv value map 813, Bv value map 814, etc.) and Av value information 815 are associated.

[0085] <Example of detailed block configuration of electronic device> 9 is an explanatory diagram showing a detailed block configuration example of an electronic device. Electronic device 500 has image sensor 100, control unit 502, display unit 901, and recording unit 902. Display unit 901 specifically corresponds to liquid crystal monitor 503 and EVF (Electronic View Finder) shown in FIG. 5, for example. Recording unit 902 specifically corresponds to memory card 504, DRAM 506, and flash memory 507 shown in FIG. 5, for example.

[0086] As described above, the image sensor 100 can set different imaging conditions for each of a plurality of imaging regions that capture an image of a subject, and has the imaging chip 113 and the signal processing chip 111. The imaging regions are the blocks 202 that exist on the imaging surface 200. The signal processing chip 111 includes a setting circuit 911. The setting circuit 911 sets the imaging conditions for each block 202 in accordance with setting information from the control unit 502.

[0087] The control unit 502 has an image processing unit 912 and a setting unit 913. The image processing unit 912 is realized by an image processing circuit or an image processing program executed by a processor. The image processing unit 912 executes image analysis processing as described in Fig. 7. Specifically, for example, the image processing unit 912 executes subject detection and area discrimination to identify a background area 701, a main subject area 702, and a coexistence area 703 by discriminating between a subject image and a background image.

[0088] The setting unit 913 sets a first imaging condition in a first imaging area among the multiple imaging areas in which a first subject image exists, sets a second imaging condition in a second imaging area among the multiple imaging areas in which a second subject image exists, and sets a third imaging condition in a third imaging area among the multiple imaging areas in which the first subject image and the second subject image exist, based on at least one of the first imaging condition and the second imaging condition.

[0089] Specifically, for example, the setting unit 913 sets an exposure time of short seconds (e.g., 1 / 60 seconds) as the first imaging condition for a block 202 (mask information 704 having a value of "1") in which a background image exists among the group of blocks constituting the imaging plane 200. The block 202 with the mask information 704 having a value of "1" is referred to as a "background block 202."

[0090] Furthermore, the setting unit 913 sets exposure time: long seconds (for example, 1 / 1000 seconds) as the second imaging condition for block 202 (mask information 704 value "2") in which the main subject image MI exists among the group of blocks constituting the imaging plane 200. The block 202 with mask information 704 value "2" is referred to as the "main subject block 202."

[0091] Furthermore, setting unit 913 sets the third imaging condition for block 202 (where the value of mask information 704 is "0") in which the main subject image MI and the background image exist among the group of blocks constituting imaging plane 200, as shown in [1] and [2] below, based on the luminance value of background block 202 and the luminance value of main subject block 202. Note that block 202 with the value of mask information 704 being "0" is referred to as an "indefinite block 202."

[0092] [1. Either the background block 202 (short time) or the main subject block 202 (long time)] The setting unit 913 sets the third imaging condition of the indefinite block 202 to either the first imaging condition of the background block 202 or the second imaging condition of the main subject block 202. For example, if the background block 202 has whiteout and the main subject block 202 does not have blackout, and if setting the second imaging condition to the indefinite block 202 results in whiteout of image data of the background image portion in the indefinite block 202, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the first imaging condition of the background block 202 (short second).

[0093] Furthermore, if the first imaging condition is set for the indefinite block 202 and the main subject portion in the indefinite block 202 becomes blocked up, the setting unit 913 sets the third imaging condition for the indefinite block 202 to the second imaging condition (long seconds) for the main subject block 202. Furthermore, if neither blown-out highlights nor blocked up shadows occur in the indefinite block regardless of whether the first imaging condition or the second imaging condition is set for the indefinite block 202, then in order to prioritize the main subject, the setting unit 913 sets the third imaging condition for the indefinite block 202 to the second imaging condition (long seconds) for the main subject block 202.

[0094] Here, blown-out highlights occur when the luminance value of the indefinite block 202 is equal to or greater than a first luminance value. The first luminance value is a luminance value at which the group of pixels (a collection of photoelectric conversion units) constituting the indefinite block 202 is saturated. Crushed black occurs when the luminance value of the indefinite block 202 is equal to or less than a second luminance value lower than the first luminance value. The second luminance value is a state in which the group of pixels (a collection of photoelectric conversion units) constituting the indefinite block 202 has insufficient luminance. Specifically, this occurs when the luminance value is 0. Note that the second luminance value does not have to be 0, and a value greater than 0 may be set as the second luminance value.

[0095] It is not necessary for the background image portions of all the indefinite blocks 202 to be blown out. For example, when the background image portions of some of the indefinite blocks 202 are blown out, the setting unit 913 may set the third imaging condition of the indefinite blocks 202 to the first imaging condition of the background blocks 202 (short second). Similarly, it is not necessary for the main subject image MI portions of all the indefinite blocks 202 to be crushed black. For example, when the main subject image MI portions of some of the indefinite blocks 202 are crushed black, the setting unit 913 may set the third imaging condition of the indefinite blocks 202 to the second imaging condition of the main subject block 202 (long second).

[0096] [2. Between background block 202 (short second) and main subject block 202 (long second)] The setting unit 913 sets the third imaging condition of the indefinite block 202 to an imaging condition between the first imaging condition of the background block 202 and the second imaging condition of the main subject block 202. For example, if the exposure time of the first imaging condition of the background block 202 is 1 / 1000 seconds (short second) and the exposure time of the second imaging condition of the main subject block 202 is 1 / 60 seconds (long second), there are three exposure times (1 / 125 seconds, 1 / 250 seconds, 1 / 500 seconds) between the first imaging condition and the second imaging condition.

[0097] Therefore, the setting unit 913 sets the third imaging condition to one of these exposure times. The setting unit 913 may set the third imaging condition to an exposure time of 1 / 250 seconds that is intermediate between the first imaging condition and the second imaging condition. Furthermore, when a setting is made to suppress the brightness of the background, the setting unit 913 may set the third imaging condition to an exposure time of 1 / 500 seconds that is closer to the first imaging condition. Furthermore, when a setting is made to prioritize the main subject, the setting unit 913 may set the third imaging condition to an exposure time of 1 / 125 seconds that is closer to the second imaging condition.

[0098] After that, the setting unit 913 transmits setting information that is the setting result to the setting circuit 911 in the signal processing chip 111. The setting information includes coordinate values ​​that identify the block 202 and values ​​that indicate the imaging conditions set for each block 202. The value that indicates the imaging conditions is, for example, a number that indicates the exposure time.

[0099] Upon receiving the setting information, the setting circuit 911 in the signal processing chip 111 sets the charge accumulation time from the start to the end of charge accumulation, i.e., the exposure time, in units of blocks 202 in accordance with the setting information. This enables the image sensor 100 to acquire image data of the next frame in accordance with the setting information.

[0100] Moreover, the image processing unit 912 in the control unit 502 has an adjustment unit 921. The adjustment unit 921 adjusts the signal level of the image data output from the indefinite block 202 so as to amplify or attenuate it according to the setting information from the setting unit 913 (so-called digital gain adjustment). Specifically, for example, the adjustment unit 921 adjusts the gain as shown in the following [3] and [4].

[0101] [3. Gain adjustment using the settings in [1] above] When the setting unit 913 sets the third imaging condition of the indefinite block 202 to the first imaging condition (short second), the image data relating to the main subject among the image data from the indefinite block 202 becomes darker than the image data from the main subject block 202 to which the second imaging condition (long second) is set. This makes the boundary between the main subject block 202 and the indefinite block 202 stand out in the main subject region 702. For this reason, the image processing unit 912 identifies the image data relating to the main subject among the image data from the indefinite block 202 by region discrimination, and the adjustment unit 921 adjusts the signal level of the image data relating to the main subject so as to amplify it. In other words, the adjustment unit 921 uniformly increases the luminance value of the pixel group constituting the image data relating to the main subject by a predetermined amount.

[0102] Furthermore, when the setting unit 913 sets the third imaging condition of the indefinite block 202 to the second imaging condition (long seconds), the adjustment unit 921 causes the image data relating to the background among the image data from the indefinite block 202 to be brighter than the image data from the background block 202 to which the first imaging condition (short seconds) is set. This makes the boundary between the background block 202 and the indefinite block 202 stand out in the background region 701. For this reason, the image processing unit 912 identifies the image data relating to the background among the image data from the indefinite block 202 by region discrimination, and the adjustment unit 921 adjusts the signal level of the image data relating to the background to be reduced. In other words, the adjustment unit 921 reduces the luminance value of the pixel group constituting the image data relating to the background by a predetermined amount uniformly.

[0103] [4. Gain adjustment using the settings in [2] above] When the setting unit 913 sets the third imaging condition of the indefinite block 202 closer to the first imaging condition (short second) (for example, 1 / 500 second), the adjustment unit 921 causes the image data relating to the main subject among the image data from the indefinite block 202 to become darker than the image data from the main subject block 202 to which the second imaging condition (long second) is set. This causes the boundary between the main subject block 202 and the indefinite block 202 to stand out within the main subject region 702.

[0104] For this reason, the image processing unit 912 identifies image data related to the main subject from the image data from the indefinite block 202 by area discrimination, and the adjustment unit 921 adjusts the signal level of the image data related to the main subject so as to amplify it. That is, the adjustment unit 921 uniformly increases the luminance values ​​of the pixel groups constituting the image data related to the main subject by a predetermined amount. In this case, the third imaging condition has a longer exposure time than the first imaging condition, so the amount of increase in the luminance values ​​may be lower than the amount of increase in the case of [3].

[0105] Furthermore, when the setting unit 913 sets the third imaging condition of the indefinite block 202 closer to the second imaging condition (long second) (for example, 1 / 125 second), the adjustment unit 921 causes image data relating to the background among the image data from the indefinite block 202 to be brighter than image data from the background block 202 to which the first imaging condition (short second) is set. This causes the boundary between the background block 202 and the indefinite block 202 to stand out in the background region 701.

[0106] For this reason, the image processing unit 912 identifies image data related to the background from the image data from the indefinite block 202 by region discrimination, and the adjustment unit 921 adjusts the signal level of the image data related to the background so as to decrease it. That is, the adjustment unit 921 uniformly decreases the luminance values ​​of the pixel groups constituting the image data related to the background by a predetermined amount. In this case, the third imaging condition has a shorter exposure time than the second imaging condition, so the amount of decrease in the luminance values ​​may be lower than the amount of decrease in the case of [3].

[0107] Furthermore, when the setting unit 913 sets the third imaging condition of the indefinite block 202 to an intermediate value between the first and second imaging conditions (for example, 1 / 250 seconds), the adjustment unit 921 may amplify the signal level of image data relating to the main subject and attenuate the signal level of image data relating to the background for the indefinite block 202, as in the cases of the first imaging condition (shorter second) and the second imaging condition (longer second) described above. Also, since the image structure in the indefinite block 202 is complex, such as having a lot of texture, there are cases in which region division into the main subject image MI portion and the background image portion in the indefinite block 202 is not performed appropriately. For such indefinite block 202, the adjustment unit 921 may not perform gain adjustment.

[0108] Moreover, the generation unit 922 in the image processing unit 912 generates the above-mentioned image file 600. For example, when a subject image is captured by pressing the release button, the generation unit 922 generates the image file 600. In the image file 600, the background region 701, image data from the background region 701, and the first imaging condition (short second) are associated with each other.

[0109] In addition, in the image file 600, the main subject region 702, image data from the main subject region 702, and the second imaging condition (long time) are associated with each other. In the image file 600, the coexistence region 703, image data from the coexistence region 703, and the third imaging condition are associated with each other.

[0110] Specifically, for example, the type of subject for each block 202 is recorded in the mask information 704. Furthermore, the imaging conditions set for each block 202 are recorded in the above-mentioned Tv value map 812. Furthermore, the luminance value for each block 202 is recorded in the above-mentioned Bv value map 814.

[0111] When sequentially displaying images (through images) generated sequentially by the output from the imaging element 100, the image processing unit 912 outputs thinned image data to the display unit to sequentially display the through images on the display unit 901. When the release button is pressed, the image processing unit 912 outputs image data in which all pixels have been output to the display unit 901. The generating unit 922 also records the generated image file 600 in the recording unit 902. Note that the predicting unit 923 and the detecting unit 924 are functions of the second embodiment described later, and therefore will not be described in the first embodiment.

[0112] <Example of setting and adjusting the third imaging condition> 10 to 12 are explanatory diagrams showing setting examples 1 to 3 of the third imaging condition. In Fig. 10 to 12, (A) shows the imaging plane 200 before the third imaging condition is set, and (B) shows the imaging plane 200 after the third imaging condition is set. The main subject image MI is the woman in the center, and the background image BI is anything other than the main subject image MI. Here, an example is taken in which the subject is imaged against the light.

[0113] Moreover, the thin-line grid represents blocks 202, and in particular, a collection of background blocks 202 containing only background images BI represents background region 701, to which a first imaging condition (short second) is set. A collection of main subject blocks 202 containing only main subject images MI represents main subject region 702, to which a second imaging condition (long second) is set. A collection of indefinite blocks 202 containing main subject images MI and background images BI surrounded by thick lines represents coexistence region 703, to which a third imaging condition is set by setting section 913, and gain adjustment is performed by adjustment section 921.

[0114] 10 shows an example in which the indefinite block 202 is set to the same first imaging condition (short second) as the background image BI when setting the indefinite block 202 to the same second imaging condition (long second) as the main subject region 702 causes whiteout in the background image BI portion in the indefinite block 202. In this case, the luminance value of the main subject image MI portion in the coexistence region 703 becomes lower than that of the main subject region 702. This makes the boundary between the main subject block 202 and the indefinite block 202 stand out in the main subject region. Therefore, the adjustment unit 921 gains up the luminance value of the main subject image MI portion for each indefinite block 202 in the coexistence region 703.

[0115] 11 shows an example in which the indefinite block 202 is set to the same second imaging condition (long second) as the main subject image MI when the main subject image MI portion in the indefinite block 202 is subject to crushed blacks due to setting the indefinite block 202 to the same first imaging condition (short second) as the background region 701. Note that if the background image BI is not blown out and the main subject image MI is not subject to crushed blacks, setting unit 913 prioritizes the main subject, and similarly sets indefinite block 202 to the same second imaging condition (long second) as the main subject image MI.

[0116] In these cases, as shown in (B), the coexistence region 703 also has a long time period, so that the luminance value of the background image BI portion in the coexistence region 703 is higher than that of the background region 701. This causes the boundary portion between the background block 202 and the indefinite block 202 to stand out in the background region. Therefore, the adjustment unit 921 gain-down the luminance value of the background image BI portion for each indefinite block 202 in the coexistence region 703.

[0117] 12 shows an example in which, when the first imaging condition (short second) and the second imaging condition (long second) are set for the indefinite block 202 due to overexposure in the background image BI and crushed shadows in the main subject image MI, the third imaging condition for the indefinite block 202 is set to an imaging condition between the first imaging condition (short second) and the second imaging condition (long second). In this case, as shown in (B), if the coexistence region 703 is closer to the short second, the luminance value of the main subject image MI portion in the coexistence region 703 will be lower than that of the main subject region 702. This makes the boundary between the main subject block 202 and the indefinite block 202 conspicuous in the main subject region 702.

[0118] Therefore, the adjustment unit 921 increases the gain of the luminance value of the main subject image MI portion for each indefinite block 202 in the coexistence region 703. On the other hand, if the coexistence region 703 is on the long side of the exposure time, the luminance value of the background image BI portion in the coexistence region 703 will be higher than that of the background region 701. This makes the boundary between the background block 202 and the indefinite block 202 stand out in the background region. Therefore, the adjustment unit 921 decreases the gain of the luminance value of the background image BI portion for each indefinite block 202 in the coexistence region 703.

[0119] 10 to 12, an example of backlighting has been described, but in the case of frontlighting, the first imaging condition is a long second and the second imaging condition is a short second, and the setting unit 913 sets the first imaging condition and the adjustment unit 921 adjusts the first imaging condition. Also, in the first embodiment, the exposure time has been described as an example of an imaging condition, but the imaging condition may be the ISO sensitivity or the exposure value.

[0120] In this way, according to the first embodiment, the third imaging condition of the third imaging region that is the boundary between the first subject image (background image BI) and the second subject image (main subject image MI) is not an imaging condition unrelated to the first imaging condition or the second imaging condition. Therefore, a natural subject image can be displayed. EXAMPLES

[0121] Next, a second embodiment will be described. In the second embodiment, the contents related to the second embodiment will be mainly described, and the same contents as those in the first embodiment will be described with the same reference numerals and will not be described. In the second embodiment, as in the first embodiment, an example of backlit photography will be described (the first imaging condition is a short second, and the second imaging condition is a long second), but it can also be applied to a case of front lighting (the first imaging condition is a long second, and the second imaging condition is a short second). In the second embodiment, as in the first embodiment, an example of an exposure time will be described as an imaging condition, but the imaging condition may be the ISO sensitivity or the exposure value.

[0122] <Impact on Coexistence Zone 703> The second embodiment shows a setting example in which a third imaging condition of the coexistence area 703 constituted by the indefinite block 202 described in the first embodiment is set based on the degree of influence on at least one of the imaging areas of the background area 701 and the main subject area 702 existing within a predetermined range from the coexistence area 703.

[0123] 13 is an explanatory diagram showing the degree of influence on the coexistence region 703. The background region 701 existing within a predetermined range is a block set in which a predetermined number of background blocks 202 are linked to an indefinite block 202. In addition, all background blocks 202 existing on the imaging plane 200 can also be included in the background region 701 existing within the predetermined range.

[0124] Similarly, a main subject region 702 existing within a predetermined range is a block set formed by connecting a predetermined number of main subject blocks 202 from an indefinite block 202. All main subject blocks 202 existing on the imaging surface 200 can also be main subject regions 702 existing within a predetermined range. The extent of the predetermined range (the value of the predetermined number) is preset in electronic device 500.

[0125] The degree of influence is an index value indicating the influence that the coexistence region 703 has on an image from the background region 701 and main subject region 702 within a predetermined range. Specifically, the degree of influence is, for example, the difference in evaluation value regarding the area between the background image BI portion and the main subject image MI portion in the indefinite block 202, or the difference in evaluation value regarding the image complexity between the background image BI portion and the main subject image MI portion in the indefinite block 202. The evaluation value regarding the image complexity is, for example, the integral value of the frequency components of the image. Details will be described later.

[0126] An evaluation value for the area of ​​the background image BI portion in an indefinite block 202 (hereinafter, area value S1) is, for example, the number of pixels in the background image BI portion. An evaluation value for the area of ​​the main subject image MI portion in an indefinite block 202 (hereinafter, area value S2) is, for example, the number of pixels in the main subject image MI portion. If there are many blocks 202 containing images with larger areas around the indefinite block 202, the burden of image processing is reduced.

[0127] For an indefinite block 202 in the coexistence region 703, when S1>S2, the area of ​​the background image BI portion is larger than the area of ​​the main subject image MI portion in the indefinite block 202. Therefore, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the first imaging condition (short second) of the background region 701 in which the background image BI exists. This causes the indefinite block 202 and the adjacent background block 202 to have the same imaging condition, reducing the burden of image processing.

[0128] On the other hand, when S2>S1, the area of ​​the main subject image MI portion is larger than that of the background image BI portion in that indefinite block 202. Therefore, the setting unit 913 sets the third imaging condition of that indefinite block 202 to the second imaging condition (long second) of the main subject region 702 in which the main subject image MI exists. This causes the indefinite block 202 and the adjacent main subject block 202 to have the same imaging condition, reducing the burden of image processing.

[0129] An evaluation value for image complexity of the background image BI portion in indefinite block 202 (hereinafter, complexity f1) is, for example, an integral value of the frequency components of the background image BI portion. An evaluation value for image complexity of the main subject image MI portion in indefinite block 202 (hereinafter, complexity f2) is, for example, an integral value of the frequency components of the main subject image MI portion. The larger the integral value, in other words, the higher the frequency, the more complex the structure of the image obtained from that subject image.

[0130] For an indefinite block 202 in the coexistence region 703, when f1>f2, the integral value of the background image BI portion in that indefinite block 202 is larger than that of the main subject image MI portion. This indicates that the image structure of the background image BI portion is complex. Therefore, the setting unit 913 sets the third imaging condition of the indefinite block 202 to the first imaging condition (short second) of the background region 701 in which the background image BI exists. As a result, the adjustment target in the adjustment unit 921 is the main subject image MI portion, which has a simpler image structure than the background image BI portion, so that the adjustment unit 921 can perform the adjustment process on the simpler image structure, and the adjustment process can be facilitated.

[0131] On the other hand, if f2>f1, the integral value of the main subject image MI portion is larger than that of the background image BI portion in that indefinite block 202. This indicates that the image structure of the main subject image MI portion is complex. Therefore, setting unit 913 sets the third imaging condition of that indefinite block 202 to the second imaging condition (long second) of main subject region 702 in which the main subject image MI exists. As a result, the adjustment target in adjustment unit 921 is the background image BI portion, which has a simpler image structure than the main subject image MI portion, so adjustment unit 921 can perform adjustment processing on a simpler image structure, making it possible to facilitate the adjustment processing.

[0132] In this way, by taking the area into consideration, the target region of image processing can be made smaller. Furthermore, if image processing is not performed, the boundary between the coexistence region 703 and the background region 701, or the boundary between the coexistence region 703 and the main subject region 702, becomes discontinuous, and the length of the vertical or horizontal boundary line becomes shorter. As a result, the boundary line becomes less noticeable in the image. Furthermore, by taking the complexity of the image into consideration, adjustment by the adjustment unit 921 becomes easier, and image quality can be improved.

[0133] In the above description, the imaging condition with higher image complexity is applied to the third imaging condition, but the imaging condition with lower image complexity, i.e., with lower frequency components, may be applied to the third imaging condition. In this case, as in the case of considering the area, the boundary between different imaging conditions is an imaging area where a subject image with a more complex image structure exists. For example, when the main subject image MI has a higher complexity than the background image BI, the third imaging condition of the indefinite block 202 is the first imaging condition (long second) of the background image BI. Therefore, a boundary between different imaging conditions exists between the coexistence area 703 and the main subject area 702. However, since the image structure of the main subject area 702 is complex, even if a boundary line is formed in the main subject area 702, the boundary line is less noticeable in the image.

[0134] <Setting unit example> FIG. 14 is an explanatory diagram showing an example of a setting unit for an indefinite block 202. The setting unit is the number of indefinite blocks 202 in a group that handles the influence shown in FIG. 13. In FIG. 14, the blocks 202 or block sets indicated by thick black lines are the setting units. (A) sets one block as the setting unit. (B) sets a block set connected in the horizontal and vertical directions as the setting unit. The setting unit 913 determines the setting unit by, for example, raster scanning from left to right in each row starting from the upper left block 202. Therefore, blocks 202 that overlap in the horizontal and vertical directions are preferentially included in the horizontal block set.

[0135] The setting unit 913 can set the third imaging condition based on the above-mentioned degree of influence for each setting unit. This allows for consideration of the continuity of the background image BI portion and the main subject image MI portion in the indefinite block 202. For example, even if the area of ​​the background image BI portion is smaller than the main subject image MI portion in one indefinite block 202, the area of ​​the background image BI portion may be larger than the main subject image MI portion in a setting unit consisting of multiple indefinite blocks 202. This allows for the third imaging condition of the indefinite block 202 to be set to an appropriate imaging condition.

[0136] <Boundary length of coexistence area 703> Fig. 15 is an explanatory diagram showing the boundary length of the coexistence region 703. In Fig. 15, the thick black line indicates the boundary between the background region 701 and the coexistence region 703, and the thick white line indicates the boundary between the main subject region 702 and the coexistence region 703. When the imaging conditions of the background block 202 and the indefinite block 202 on both sides of the boundary, where the region between the thick black line and the thick white line is the coexistence region 703, are different, the boundary between the blocks 202 becomes conspicuous. Similarly, when the imaging conditions of the main subject block 202 and the indefinite block 202 on both sides of the boundary are different, the boundary between the blocks 202 becomes conspicuous. It is preferable that the length of the conspicuous boundary is short.

[0137] Therefore, the setting unit 913 compares the length of the boundary (thick black line) between the background region 701 and the coexistence region 703 with the length of the boundary (thick white line) between the main subject region 702 and the coexistence region 703. Then, the setting unit 913 sets the third imaging condition to the first imaging condition or the second imaging condition so that the imaging conditions are the same on both sides of the longer boundary (so that the imaging conditions are different on both sides of the shorter boundary). This makes it possible to make the boundary where the imaging conditions are different on both sides less noticeable.

[0138] <Halo Countermeasures> FIG. 16 is an explanatory diagram showing the reduction of the effect of halos. A halo (phenomenon) is an aberration that occurs as a result of the path of subject light being distorted as it passes through a lens. For example, in backlit photography, halos are likely to occur at the boundary between a main subject that is in shadow due to the backlight and a bright background. (A) shows a main subject image MI in which a halo 1600 has occurred, and (B) and (C) are enlarged views of a certain indefinite block 202a. Although the halo 1600 exists in the background block 202, in subject detection, the area up to the outer edge (dashed line) of the halo 1600 that encompasses the main subject image MI is detected as the main subject.

[0139] In (B), it is assumed that the third imaging condition of the indefinite block 202a is set to the first imaging condition (short second). In this case, the adjustment unit 921 does not perform gain adjustment on the image data from the background region 701 (not including the halo 1600). The adjustment unit 921 performs gain-up adjustment on the image data from the main subject region 702 (including the halo 1600). In this case, when the halo 1600 is predicted, the adjustment unit 921 does not perform gain adjustment on the image data from the region of the halo 1600 (the gain may be reduced), as with the image data from the background region 701. Therefore, when the subject image is displayed, an image in which the halo 1600 is suppressed can be displayed.

[0140] In (C), it is assumed that the third imaging condition of the indefinite block 202a is set to the first imaging condition (long seconds). In this case, the adjustment unit 921 performs gain down adjustment on the image data from the background region 701 (not including the halo 1600). The adjustment unit 921 does not perform gain adjustment on the image data from the main subject region 702 (including the halo 1600). In this case, when the halo 1600 is predicted, the adjustment unit 921 performs gain down adjustment on the image data from the region of the halo 1600, similar to the image data from the background region 701. Therefore, when the subject image is displayed, it is possible to display an image in which the halo 1600 is suppressed.

[0141] The prediction of the halo is performed by the prediction unit 923 shown in FIG. 9. Specifically, for example, the prediction unit 923 predicts that a halo will occur if the step between the first imaging condition (short second) and the second imaging condition (long second) is equal to or greater than a predetermined step. For example, if the exposure time of the first imaging condition of the background block 202 is 1 / 1000 seconds (short second) and the exposure time of the second imaging condition of the main subject block 202 is 1 / 60 seconds (long second), there are three steps of exposure time (1 / 125 seconds, 1 / 250 seconds, 1 / 500 seconds) between the first imaging condition and the second imaging condition. If the threshold value for the prediction of the halo occurrence is two steps or more, there is a difference of three steps, so the prediction unit 923 predicts that a halo will occur. The prediction unit 923 may also predict the occurrence of a halo based on image data from the image sensor 100.

[0142] 17 is a graph showing an example of a prediction of halo occurrence using a gamma function. The horizontal axis is the output value of RAW image data from image sensor 100, and the vertical axis is the output value after gamma correction in image processing unit 912. x1 is the output value of image data from the area where main subject image MI (area other than halo) exists, and y1 is the output value of x1 after gamma correction. x2 is the output value of image data from the area where the outer edge of main subject image MI (halo 1600) exists, and y2 is the output value of x2 after gamma correction.

[0143] If the luminance difference between y1 and y2 is equal to or greater than a predetermined value, the prediction unit 923 predicts that a halo 1600 occurs. As a result, the adjustment unit 921 performs gain adjustment as shown in FIG.

[0144] When capturing an image of a subject using electronic device 500, the user can apply at least one of the following settings: setting the image capturing conditions based on the degree of impact on coexistence area 703 described above; setting the image capturing conditions based on the boundary length of coexistence area 703; and settings for preventing halos.

[0145] In addition, when applying two or more of these settings, for example, when applying a setting of imaging conditions based on the degree of impact on the coexistence area 703 and a setting of imaging conditions based on the boundary length of the coexistence area 703, the electronic device 500 prioritizes the setting of imaging conditions based on the degree of impact on the coexistence area 703.

[0146] Furthermore, when applying the setting of the imaging conditions based on the degree of influence on the coexistence region 703 and the setting for the halo countermeasure, the electronic device 500 preferentially executes the setting of the imaging conditions based on the degree of influence on the coexistence region 703. Furthermore, when applying the setting of the imaging conditions based on the degree of influence on the coexistence region 703, the setting of the imaging conditions based on the boundary length of the coexistence region 703, and the setting for the halo countermeasure, the electronic device 500 executes in this order: the setting of the imaging conditions based on the degree of influence on the coexistence region 703, the setting of the imaging conditions based on the boundary length of the coexistence region 703, and the setting for the halo countermeasure.

[0147] Furthermore, when applying both the setting of the imaging condition based on the boundary length of the coexistence area 703 and the setting for preventing halos, the electronic device 500 gives priority to the setting of the imaging condition based on the boundary length of the coexistence area 703.

[0148] However, when the electronic device 500 detects camera shake or subject blurring by the detection unit 924, it sets the imaging conditions giving priority to the main subject. Setting the imaging conditions giving priority to the main subject means setting the block 202 including the main subject image MI to the same imaging conditions, and specifically, for example, setting the coexistence region 703 to the same imaging conditions as the main subject region 702.

[0149] 15, the setting unit 913 sets the coexistence region 703 to the same imaging conditions (long seconds) as the main subject region 702. For camera shake, for example, the detection unit 924 detects a predetermined amount or more of shake using a gyro sensor or an acceleration sensor built into the electronic device 500. For subject shake, for example, the detection unit 924 detects a motion vector indicating the direction and amount of movement of the main subject, and if the amount of movement in the direction in which the main subject moves between successive frames is a predetermined amount or more, it is detected as subject shake.

[0150] In addition, either the area or the image complexity is selected for setting the imaging conditions based on the degree of influence on the coexistence region 703. When the area is selected, it is possible to speed up the setting of the imaging conditions based on the degree of influence on the coexistence region 703. On the other hand, when the image complexity is selected, it is possible to increase the accuracy of the setting of the imaging conditions based on the degree of influence on the coexistence region 703.

[0151] <Example of setting procedure> 18 is a flowchart showing a first example of a process procedure for setting imaging conditions in the electronic device 500. The electronic device 500 executes initial settings in response to a user's operational input (step S1801). In the initial settings (step S1801), settings are made regarding the degree of influence on the coexistence region 703 (area or complexity), the boundary length of the coexistence region 703, halo countermeasures, the number of blocks included in the setting unit, and whether or not to execute a setting process for avoiding highlight and shadow blockage.

[0152] The highlight / shadow block avoidance setting process is a process in which the setting unit 913 sets the third imaging condition of the indefinite block 202 to either the first imaging condition (short second) or the second imaging condition (long second) as shown in Fig. 10. Specifically, the setting unit 913 sets the imaging condition in which highlight / shadow block does not occur to the indefinite block 202 from among the first imaging condition and the second imaging condition.

[0153] Next, the electronic device 500 starts acquiring a through image from the image sensor 100 (step S1802). The electronic device 500 executes image analysis on the acquired image data of the through image by the image processing unit 912 (step S1803). Specifically, for example, the image processing unit 912 detects a main subject by subject detection, and determines the background other than the main subject, and distinguishes the area type of each block 202 (background area 701, main subject area 702, coexistence area 703) by area discrimination.

[0154] Next, electronic device 500 judges whether the degree of influence (area or image complexity) on coexistence region 703 is set by the initial setting (step S1801) (step S1804). If the degree of influence on coexistence region 703 is not set (step S1804: No), the mode is set to main subject priority mode, and electronic device 500 executes highlight and shadow blockage avoidance setting process or main subject priority setting process by setting unit 913 (step S1805) and proceeds to step S1810. Specifically, for example, if the initial setting (step S1801) is set to execute highlight and shadow blockage avoidance setting process, highlight and shadow blockage avoidance setting process is executed, and if not, main subject priority setting process is executed. Note that if highlight and shadow blockage do not occur regardless of whether the first imaging condition or the second imaging condition is set, main subject priority setting process may be used.

[0155] The main subject priority setting process (step S1805) is a process in which the setting unit 913 sets the third imaging condition of the indefinite block 202 to the second imaging condition (long second) which is the same as the main subject image MI, as shown in Fig. 11. In this case, as shown in Fig. 11(B), the coexistence region 703 also has a long second, so that the luminance value of the background image BI portion in the coexistence region 703 is higher than that of the background region 701. Therefore, in the gain adjustment process (step S1810), the electronic device 500 uses the adjustment unit 921 to reduce the gain of the luminance value of the background image BI portion for each indefinite block 202 in the coexistence region 703.

[0156] In step S1804, if the degree of influence on coexistence region 703 has been set (step S1804: Yes), it is determined whether the detection flag for camera shake or subject detection blur is ON or OFF (step S1806). If the detection flag is ON (step S1806: ON), it indicates that camera shake or subject detection blur is being detected, and the process proceeds to main subject priority setting processing (step S1805). If the detection flag is OFF (step S1806: OFF), it indicates that camera shake or subject detection blur is not being detected, and the process proceeds to step S1807.

[0157] In step S1807, electronic device 500 determines whether the degree of influence on coexistence region 703 initially set (step S1801) is area or image complexity (step S1807). If it is area (step S1807: area), electronic device 500 executes area priority setting processing (step S1808) and proceeds to gain adjustment processing (step S1810). Details of area priority setting processing (step S1808) will be described later with reference to FIG. 19 or FIG. 20.

[0158] On the other hand, if the complexity is image complexity (step S1807: complexity), electronic device 500 executes complexity priority setting processing (step S1809) and proceeds to gain adjustment processing (step S1810). Details of the complexity priority setting processing (step S1809) will be described later with reference to FIG. 25 or FIG. 26.

[0159] In step S1810, electronic device 500 executes gain adjustment processing by adjustment unit 921 (step S1810). For example, when the third imaging condition of indefinite block 202 is set to the first imaging condition of background block 202 (short second), adjustment unit 921 increases the gain of image data from the main subject image MI portion in indefinite block 202. Also, when the third imaging condition of indefinite block 202 is set to the second imaging condition of main subject block 202 (long second), adjustment unit 921 decreases the gain of image data from the background image BI portion in indefinite block 202.

[0160] After that, electronic device 500 detects camera shake or subject blurring using detection unit 924 (step S1811). If camera shake or subject blurring is detected (step S1811: Yes), the detection flag is set to ON (step S1812) and the process proceeds to step S1814. On the other hand, if camera shake or subject blurring is not detected (step S1811: No), the detection flag is set to OFF and the process proceeds to step S1814.

[0161] The electronic device 500 determines whether or not to end the through image acquisition (step S1811), and if not (step S1814: No), returns to step S1803. On the other hand, if the through image acquisition is to be ended (step S1814: Yes), the electronic device 500 ends the process of setting the imaging conditions. Note that while the process of setting the imaging conditions is being executed, the user can press the release button at any time to capture an image of the subject. In this case, image data is generated by the output of all pixels from the imaging element 100 at the timing of capturing the image, and the image file 600 is recorded in the recording section.

[0162] <Area Priority Setting Process (Step S1808)> Fig. 19 is a flowchart showing a detailed processing procedure example 1 of the area priority setting process (step S1808) shown in Fig. 18. The area priority setting process (step S1808) in Fig. 19 is a process for setting a third imaging condition based on the size of the areas of the background image BI portion and the main subject image MI portion in the indefinite block 202.

[0163] The electronic device 500 determines whether or not there is an unselected setting unit by the setting unit 913 (step S1901). If there is an unselected setting unit (step S1901: Yes), the electronic device 500 selects one of the unselected setting units by the setting unit 913 (step S1902). Then, the electronic device 500 calculates, by the setting unit 913, an area value S1 of the background image BI portion and an area value S2 of the main subject image MI portion in the selected setting unit (step S1903).

[0164] Then, the electronic device 500 compares the area values ​​S1 and S2 by the setting unit 913 (step S1904). If the area value S1 is larger (step S1904: Yes), the electronic device 500 sets the third imaging condition of the selected setting unit to the first imaging condition of the background region 701 by the setting unit 913 (step S1905), and returns to step S1901.

[0165] If the area value S1 is not larger (step S1904: No), the electronic device 500 sets the third imaging condition of the selected setting unit as the second imaging condition of the main subject region 702 (step S1906) by the setting unit 913, and returns to step S1901. If there is no unselected setting unit in step S1901 (step S1901: No), the electronic device 500 ends the area priority setting process (step S1808) and proceeds to gain adjustment process (step S1810).

[0166] Fig. 20 is a flowchart showing a detailed processing procedure example 2 of the area priority setting process (step S1808) shown in Fig. 18. The area priority setting process (step S1808) in Fig. 20 sets a third imaging condition based on the size of the areas of the background image BI portion and the main subject image MI portion in the indefinite block 202, and sets the third imaging condition based on the boundary length of the coexistence region 703 and measures against halos, since the areas do not determine superiority or inferiority if the area difference is within an allowable range. The same processes as those in Fig. 19 are assigned the same step numbers, and their explanations will be omitted.

[0167] The electronic device 500 calculates the area value S1 of the background image BI portion and the area value S2 of the main subject image MI portion in the selected setting unit (step S1903), and judges whether the area difference between the area values ​​S1 and S2 is within the allowable range (step S2001). If it is within the allowable range (step S2001: Yes), the electronic device 500 increments the indefinite count C by 1 (step S2002) using the setting unit 913, and proceeds to step S1904. The indefinite count C is the number of times the area difference falls within the allowable range. The initial value of the indefinite count C is 0. On the other hand, if the area difference is not within the allowable range (step S2001: No), proceeds to step S1904.

[0168] If the area value S1 is larger (step S1904: Yes), the electronic device 500 causes the setting unit 913 to provisionally set the third imaging condition of the selected setting unit to the first imaging condition of the background region 701 (step S2003), and returns to step S1901.

[0169] Also, if the area value S1 is not larger (step S1904: No), the electronic device 500 provisionally sets the third imaging condition of the selected setting unit as the second imaging condition of the main subject region 702 by the setting unit 913 (step S2004), and returns to step S1901. If there is no unselected setting unit in step S1901 (step S1901: No), the electronic device 500 determines, by the setting unit 913, whether the ratio of the indefinite count C is equal to or greater than a predetermined ratio (step S2005). For example, assuming that the number of setting units is N and the predetermined ratio is 0.4, if C / N is equal to or less than 0.4, the process proceeds to step S2006.

[0170] If it is equal to or greater than the predetermined ratio (step S2005: Yes), the electronic device 500 causes the setting unit 913 to finalize the third imaging condition for each setting unit provisionally set in steps S2003 and S2004 (step S2006). That is, the smaller the indefinite count number C, the more the area difference is outside the allowable range when viewed as a whole image, so the provisionally determined third imaging condition is reflected. Then, the area priority setting process (step S1808) is terminated, and the process proceeds to the gain adjustment process (step S1810).

[0171] On the other hand, in step S2005, if the ratio of the indefinite count C is not equal to or less than the predetermined ratio (step S2005: No), the process will be described with reference to FIGS.

[0172] Fig. 21 is a flowchart showing a detailed processing procedure example 2-1 of the area priority setting process (step S1808) shown in Fig. 18. Fig. 21 shows a process that branches from step S2005: No in Fig. 20 and moves to gain adjustment process (step S1810) when the boundary length setting of the coexistence area 703 is ON in the initial setting (step S1801).

[0173] The electronic device 500 calculates the background side boundary length L1 and the main subject side boundary length L2 of the coexistence region 703 by the setting unit 913 (step S2101). For example, in FIG. 15, the background side boundary length L1 is the total length of the thick black line, and the main subject side boundary length L2 is the total length. Then, the electronic device 500 compares the boundary lengths L1 and L2 by the setting unit 913 (step S1904). If the background side boundary length L1 is longer (step S2102: Yes), the electronic device 500 sets the third imaging condition of the coexistence region to the first imaging condition of the main subject region 702 by the setting unit 913 (step S2103), and proceeds to the gain adjustment process (step S1810).

[0174] If the background-side boundary length L1 is not larger (step S2102: No), the electronic device 500 sets the third imaging condition of the coexistence region 703 to the second imaging condition of the background region 701 by the setting unit 913 (step S1906), and proceeds to the gain adjustment process (step S1810). This allows the imaging condition to be set to the imaging region with the shorter boundary length of the coexistence region 703, while giving priority to the degree of influence (area) on the coexistence region 703, if the area difference is not important.

[0175] Fig. 22 is a flowchart showing a detailed processing procedure example 2-2 of the area priority setting process (step S1808) shown in Fig. 18. Fig. 22 shows a process that branches from step S2005: No in Fig. 20 and moves to gain adjustment process (step S1810) when the boundary length and halo countermeasure settings of the coexistence area 703 are ON in the initial setting (step S1801). The same processes as those in Fig. 21 are given the same step numbers and their explanations are omitted.

[0176] After step S2101, the electronic device 500 determines whether the difference between the boundary lengths L1 and L2 (boundary length difference) is within the allowable range by the setting unit 913 (step S2201). If it is not within the allowable range (step S2201: No), the process proceeds to step S2102. On the other hand, if the boundary length is within the allowable range (step S2201: Yes), the electronic device 500 executes halo countermeasure processing by the setting unit 913 (step S2202) and proceeds to gain adjustment processing (step S1810).

[0177] Fig. 23 is a flowchart showing a detailed processing procedure example 1 of the halo countermeasure processing (step S2202) shown in Fig. 22. The electronic device 500 predicts the occurrence of a halo by the prediction unit 923 (step S2301). For example, if the step between the first imaging condition and the second imaging condition is equal to or greater than a predetermined step, the prediction unit 923 predicts that a halo will occur, and proceeds to step S2302. Also, as shown in Fig. 17, if the difference between the gamma correction values ​​y1 and y2 of the output value x1 of the image data from the inside of the main subject image MI and the output value x2 of the image data from the outer edge is equal to or greater than a predetermined value, the prediction unit 923 predicts that a halo will occur, and proceeds to step S2302.

[0178] If it is predicted that no halo will occur (step S2301: No), the provisional settings of steps S2003 and S2004 are finalized (step S2302), and the process proceeds to gain adjustment processing (step S1810). On the other hand, if it is predicted that a halo will occur (step S2301: Yes), the electronic device 500 sets the imaging condition of the coexistence region 703 to the first imaging condition of the background image by the setting unit 913 (step S2303), and the process proceeds to gain adjustment processing (step S1810). As a result, the coexistence region 703 corresponding to the outer edge of the main subject image MI is matched with the first imaging condition (short second) inside the background image BI, thereby reducing the influence of the halo.

[0179] In this way, while giving priority to the degree of influence (area) on the coexistence region 703, if the area difference is not determinative, the imaging condition is set to the imaging area with the shorter boundary length of the coexistence region 703, and if the boundary length difference is not determinative, a third imaging condition for the coexistence region 703 can be set by halo countermeasure processing (step S2202).

[0180] Fig. 24 is a flowchart showing a detailed processing procedure example 2-3 of the area priority setting process (step S1808) shown in Fig. 18. Fig. 24 shows a process that branches from step S2005: No in Fig. 20 and moves to gain adjustment process (step S1810) when the boundary length setting of the coexistence area 703 is OFF and the setting for halo countermeasures is ON in the initial setting (step S1801). The same processes as those in Fig. 23 are given the same step numbers and their explanations are omitted.

[0181] 24, the coexistence region 703 corresponding to the outer edge of the main subject image MI is matched with the first imaging condition (short second) inside the background image BI, thereby reducing the effect of the halo. In this way, while giving priority to the degree of effect (area) on the coexistence region 703, when the difference in area cannot be determined, the third imaging condition for the coexistence region 703 can be set by halo countermeasure processing (step S2202).

[0182] <Complexity Priority Setting Process (Step S1809)> Fig. 25 is a flowchart showing a detailed processing procedure example 1 of the complexity priority setting process (step S1809) shown in Fig. 18. The complexity priority setting process (step S1809) in Fig. 25 is a process for setting a third imaging condition based on the degree of image complexity of the background image BI portion and the main subject image MI portion in the indefinite block 202.

[0183] The electronic device 500 determines whether or not there is an unselected setting unit by the setting unit 913 (step S2501). If there is an unselected setting unit (step S2501: Yes), the electronic device 500 selects one unselected setting unit by the setting unit 913 (step S2502). Then, the electronic device 500 calculates the complexity f1 of the background image BI portion and the complexity f2 of the main subject image MI portion in the selected setting unit by the setting unit 913 (step S2503).

[0184] Then, the electronic device 500 compares the complexities f1 and f2 through the setting unit 913 (step S2504). If the complexity f1 is greater (step S2504: Yes), the electronic device 500 sets the third imaging condition of the selected setting unit to the first imaging condition of the background region 701 through the setting unit 913 (step S2505), and returns to step S2501.

[0185] If the complexity f1 is not greater (step S2504: No), the electronic device 500 sets the third imaging condition of the selected setting unit to the second imaging condition of the main subject region 702 (step S2506) by the setting unit 913, and returns to step S2501. If there is no unselected setting unit in step S2501 (step S2501: No), the electronic device 500 ends the area priority setting process (step S1808) and proceeds to gain adjustment process (step S1810).

[0186] Fig. 26 is a flowchart showing a detailed processing procedure example 2 of the complexity priority setting process (step S1809) shown in Fig. 18. The complexity priority setting process (step S1809) in Fig. 26 sets a third imaging condition based on the degree of image complexity of the background image BI portion and the main subject image MI portion in the indefinite block 202, and sets the third imaging condition based on the boundary length of the coexistence region 703 and halo countermeasures since image complexity does not determine superiority or inferiority if the difference in complexity is within an allowable range. The same processes as those in Fig. 25 are assigned the same step numbers, and their explanations will be omitted.

[0187] The electronic device 500 calculates the complexity f1 of the background image BI portion and the complexity f2 of the main subject image MI portion in the selection setting unit (step S2503) and judges whether the difference between the complexities f1 and f2 is within the allowable range (step S2601). If it is within the allowable range (step S2601: Yes), the electronic device 500 increments the indefinite count C by 1 (step S2602) by the setting unit 913 and proceeds to step S2504. The indefinite count C is the number of times the difference in complexity falls within the allowable range. The initial value of the indefinite count C is 0. On the other hand, if the difference in complexity is not within the allowable range (step S2601: No), the electronic device 500 proceeds to step S2504.

[0188] If the complexity f1 is greater (step S2504: Yes), the electronic device 500 causes the setting unit 913 to provisionally set the third imaging condition of the selected setting unit to the first imaging condition of the background region 701 (step S2603), and returns to step S2501.

[0189] Also, if the complexity f1 is not larger (step S2504: No), the electronic device 500 provisionally sets the third imaging condition of the selected setting unit to the second imaging condition of the main subject region 702 by the setting unit 913 (step S2604), and returns to step S2501. If there is no unselected setting unit in step S2501 (step S2501: No), the electronic device 500 determines, by the setting unit 913, whether the ratio of the indefinite count C is equal to or greater than a predetermined ratio (step S2605). For example, assuming that the number of setting units is N and the predetermined ratio is 0.4, if C / N is equal to or less than 0.4, the process proceeds to step S2606.

[0190] If it is equal to or greater than the predetermined ratio (step S2605: Yes), the electronic device 500 causes the setting unit 913 to finalize the third imaging condition for each setting unit provisionally set in steps S2603 and S2604 (step S2606). That is, the smaller the indefinite count number C, the more the difference in complexity is outside the allowable range when viewed as a whole image, and therefore the provisionally determined third imaging condition is reflected. Then, the complexity priority setting process (step S1809) is terminated, and the process proceeds to the gain adjustment process (step S1810).

[0191] On the other hand, in step S2605, if the ratio of the indefinite count C is not equal to or less than the predetermined ratio (step S2605: No), the processes shown in Fig. 21 to Fig. 24 are applied. By applying the process in Fig. 21, while giving priority to the degree of influence on the coexistence region 703 (complexity of the image), if the difference in complexity cannot be determined, the imaging condition can be set to the imaging region with the shorter boundary length of the coexistence region 703.

[0192] In addition, by applying the processing of Figures 22 and 23, while giving priority to the degree of influence on the coexistence region 703 (complexity of the images), if the difference in complexity is not determinative, the imaging condition is set to the imaging condition of the imaging region with the shorter boundary length of the coexistence region 703, and if the difference in boundary length is not determinative, a third imaging condition for the coexistence region 703 can be set by halo countermeasure processing (step S2202).

[0193] 24, the coexistence region 703 corresponding to the outer edge of the main subject image is matched with the first imaging condition (short second) inside the background image, thereby reducing the effect of the halo. In this way, while giving priority to the degree of effect (complexity) on the coexistence region 703, when the difference in complexity is not important, the third imaging condition for the coexistence region 703 can be set by halo countermeasure processing (step S2202).

[0194] Fig. 27 is a flowchart showing an example 2 of the process procedure for setting the imaging conditions in the electronic device 500. Unlike Fig. 18, Fig. 27 shows an example of the process procedure for setting the imaging conditions when the setting of the degree of influence on the coexistence area 703 is OFF. Therefore, in Fig. 27, the area priority setting process (step S1808) and the complexity priority setting process (step S1809) are not executed. Also, the same processes as Fig. 18 are given the same step numbers, and their explanations are omitted.

[0195] In FIG. 27, when the detection flag for camera shake or subject shake is OFF, any one of the processes for setting image capturing conditions based on the boundary length of the coexistence area 703 shown in FIG. 21, the process for setting image capturing conditions based on the boundary length of the coexistence area 703 and halo countermeasures shown in FIG. 22, and the process for setting image capturing conditions based on halo countermeasures is executed, and then the process proceeds to gain adjustment processing (step S1810).

[0196] In addition, in the initial setting (step S1801), if the boundary length setting of the coexistence area 703 is ON and the halo countermeasure setting is OFF, Fig. 21 is applied to the processing in Fig. 27. Also, if the boundary length setting of the coexistence area 703 is ON and the halo countermeasure setting is ON, Fig. 22 is applied to the processing in Fig. 27. However, for details of the halo countermeasure processing (step S2202) in Fig. 22, Fig. 28 described later is applied. Also, if the boundary length setting of the coexistence area 703 is OFF and the setting for the halo countermeasure is ON, Fig. 29 described later is applied to the processing in Fig. 27.

[0197] Fig. 28 is a flowchart showing a detailed processing procedure example 2 of the halo countermeasure processing (step S2202) shown in Fig. 22. The same processing as in Fig. 23 is given the same step number, and the description thereof will be omitted. In the imaging condition setting processing in Fig. 27, the area priority setting processing (step S1808) and the complexity priority setting processing (step S1809) are not executed, so that step S2302 in Fig. 23 is unnecessary in Fig. 28. Therefore, if it is predicted that no halo will occur (step S2301: No), the process proceeds to the gain adjustment processing (step S1810).

[0198] This allows the coexistence region 703 corresponding to the outer edge of the main subject image to be matched with the first imaging condition (short second) inside the background image, thereby reducing the effect of the halo. In this way, while giving priority to the boundary length of the coexistence region 703, if the boundary length difference is not important, the third imaging condition for the coexistence region 703 can be set by halo countermeasure processing (step S2202).

[0199] Fig. 29 is a flowchart showing a detailed example of a processing procedure for the halo countermeasure processing. As in Fig. 28, in the setting processing of the imaging conditions in Fig. 27, the area priority setting processing (step S1808) and the complexity priority setting processing (step S1809) are not executed in Fig. 29, so step S2302 in Fig. 23 is unnecessary in Fig. 28. Therefore, if it is predicted that no halo will occur (step S2301: No), the process proceeds to the gain adjustment processing (step S1810).

[0200] This allows the effect of the halo to be reduced by matching the coexistence region 703 corresponding to the outer edge of the main subject image with the first imaging condition (short second) inside the background image. In this way, the third imaging condition for the coexistence region 703 can be set solely as a measure against the halo.

[0201] In this way, according to the second embodiment, the third imaging condition of the third imaging region, which is the boundary between the first subject image (background image BI) and the second subject image (main subject image MI), is an imaging condition that takes into consideration the balance of the entire image. Therefore, a natural subject image can be displayed.

[0202] The present invention is not limited to the above, and may be implemented in any combination thereof. In addition, other embodiments that are conceivable within the scope of the technical concept of the present invention are also included in the scope of the present invention. [Explanation of symbols]

[0203] 100 imaging element, 111 signal processing chip, 112 memory chip, 113 imaging chip, 200 imaging surface, 202 block, 500 electronic device, 501 imaging optical system, 502 control unit, 600 image file, 901 display unit, 902 recording unit, 911 setting circuit, 912 image processing unit, 913 setting unit, 921 adjustment unit, 922 generation unit, 923 prediction unit, 924 detection unit

Claims

[Claim 1] an imaging element capable of setting different imaging conditions for each of a plurality of imaging regions for imaging an object; a setting unit that sets a first imaging condition in a first imaging area among the plurality of imaging areas in which a first subject image exists, sets a second imaging condition different from the first imaging condition in a second imaging area among the plurality of imaging areas in which a second subject image exists, and sets a third imaging condition in a third imaging area among the plurality of imaging areas in which the first subject image and the second subject image exist, based on at least one of the first imaging condition and the second imaging condition and a luminance value based on an output from the third imaging area; An electronic device having the

Images