Camera element and camera device

By introducing a display control unit and a selection unit in the electronic device, users can set different shooting conditions for multiple shooting areas, solving the problem that it is difficult to flexibly set the shooting conditions in the prior art, and improving the personalization of the shooting effect and precision.

CN113890993BActive Publication Date: 2024-11-26NIKON CORP
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202111144079.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2014-09-30
Filing Date
2015-09-30
Publication Date
2024-11-26
Estimated Expiration
2035-09-30

AI Technical Summary

Technical Problem

In the prior art, it is difficult to set different shooting conditions for each area, resulting in poor shooting results.

Method used

By introducing a display control unit and a selection unit in the electronic device, images of a plurality of shooting areas are displayed, and shooting conditions are set according to the area selected by the user, so as to ensure the independence of shooting conditions of each area.

Benefits of technology

It realizes flexible setting and control of different shooting areas, improving the personalization and accuracy of shooting effects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113890993B_ABST
    Figure CN113890993B_ABST
Patent Text Reader

Abstract

An electronic device, comprising: a display control unit that causes a display unit to display a first image and a second image, wherein the first image is generated by capturing light incident on a first area of ​​a capturing element, and the second image is generated by capturing light incident on a second area of ​​the capturing element that is different from the first area; and a selection unit that selects an area of ​​the first area and the second area for which shooting conditions are to be changed based on the first image and the second image displayed on the display unit.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This invention application is a divisional application of an invention application with an international application date of September 30, 2015, an international application number of PCT / JP2015 / 077787, a national application number of 201580064891.X entering the Chinese national phase, and an invention name of “electronic device, shooting device and shooting system”. Technical Field

[0002] The invention relates to electronic equipment, a photographing device and a photographing system. Background Art

[0003] An imaging element including an imaging chip and a signal processing chip has been proposed (see Patent Document 1).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2006-49361 Summary of the invention

[0007] In the prior art, there is a problem that it is difficult to set shooting conditions for each area.

[0008] According to a first embodiment of the present invention, an electronic device includes: a display control unit, which causes a display unit to display a first image and a second image, wherein the first image is an image generated by photographing light incident on a first area of ​​a photographing element, and the second image is an image generated by photographing light incident on a second area of ​​the photographing element that is different from the first area; and a selection unit, which selects an area of ​​the first area and the second area for which shooting conditions are to be changed based on the first image and the second image displayed on the display unit.

[0009] According to a second aspect of the present invention, the electronic device according to the first aspect preferably includes a setting unit that sets a photographing condition of the area selected by the selection unit, of the first area and the second area.

[0010] According to a third aspect of the present invention, in the electronic device of the second aspect, preferably, the setting unit sets the shooting condition of the area not selected by the selection unit by setting the shooting condition of the area selected by the selection unit among the first area and the second area.

[0011] According to the fourth aspect of the present invention, in an electronic device of the third aspect, preferably, the display control unit causes the display unit to display a third image and a fourth image, the third image being an image generated by photographing light incident on the first area under the shooting conditions set by the setting unit, and the fourth image being an image generated by photographing light incident on the second area under the shooting conditions set by the setting unit.

[0012] According to the fifth aspect of the present invention, in the electronic device of the third or fourth aspect, preferably, the setting unit sets the shooting conditions of the area selected by the selection unit among the first area and the second area to be different from the shooting conditions of the area not selected by the selection unit.

[0013] According to the sixth embodiment of the present invention, in an electronic device of any one of the third to fifth embodiments, preferably, the display control unit causes the display unit to display a setting image, which is used to set shooting conditions for the areas selected by the selection unit in the first area and the second area, and the setting unit sets the shooting conditions based on the setting image displayed on the display unit.

[0014] According to the seventh embodiment of the present invention, an electronic device includes: a display control unit, which causes the display unit to display a subject image captured by a capturing unit, wherein the capturing unit is controlled under prescribed capturing conditions for each of a plurality of capturing areas; and a segmentation unit, which segments the subject image into at least one partial area, the display control unit causing the display unit to display a partial image and the subject image, the partial image being an image captured by the capturing unit of the capturing area corresponding to the partial area.

[0015] According to the eighth embodiment of the present invention, in the electronic device of the seventh embodiment, it is preferred that it further includes a setting unit for setting shooting conditions of the shooting area corresponding to the partial area, the display control unit displays the partial image overlapping with the subject image, and the setting unit selects the partial area on the subject image and sets the shooting conditions related to the corresponding shooting area.

[0016] According to the ninth aspect of the present invention, in the electronic device of the seventh aspect, it is preferred that it further includes a setting unit for setting shooting conditions of the shooting area corresponding to the partial area, the display control unit displays the partial image on an area of ​​the display unit that is different from an area where the subject image is displayed, and the setting unit selects the partial image displayed on the display unit and sets the shooting conditions of the shooting area corresponding to the selected partial image.

[0017] According to a tenth aspect of the present invention, in the electronic device according to the ninth aspect, preferably, the display control unit displays the partial area on the subject image displayed by the display unit.

[0018] According to the eleventh aspect of the present invention, in the electronic device of the ninth or tenth aspect, preferably, the segmentation unit segments the subject image into a plurality of partial areas, and the display control unit displays a plurality of partial images corresponding to the plurality of partial areas in an enlarged manner on the display unit.

[0019] According to the twelfth mode of the present invention, in the electronic device of the ninth or tenth mode, preferably, the segmentation unit segments the subject image into a plurality of partial areas, and the display control unit displays a plurality of partial images corresponding to the plurality of partial areas on the display unit at a glance.

[0020] According to the thirteenth aspect of the present invention, in the electronic device of any one of the eighth to twelfth aspects, it is preferred that the display control unit displays the subject image captured by the capturing unit of the capturing area corresponding to the partial area on the display unit based on the capturing conditions set by the setting unit.

[0021] According to the fourteenth embodiment of the present invention, in the electronic device of any one of the seventh to thirteenth embodiments, preferably, the segmentation unit has a subject detection unit that detects a specified subject from the subject image, and divides the area corresponding to the specified subject detected by the subject detection unit into the partial area.

[0022] According to the fifteenth embodiment of the present invention, in the electronic device of any one of the seventh to thirteenth embodiments, preferably, the segmentation unit has an external operation unit for selecting a desired area from the subject image displayed by the display control unit, and segments the desired area selected by the external operation unit into the partial areas.

[0023] According to a sixteenth aspect of the present invention, in the electronic device of any one of the seventh to fifteenth aspects, preferably, the imaging condition includes at least one of a charge accumulation time of the imaging region and a magnification factor of the imaging region.

[0024] According to the seventeenth embodiment of the present invention, in the electronic device of any one of the eighth to thirteenth embodiments, it is preferred that the electronic device includes: a photographing unit, which controls the photographing unit according to a prescribed photographing condition for each of a plurality of photographing areas; a photographing control unit, which controls each of the plurality of photographing areas according to the prescribed photographing condition; and a recording unit that records the subject image photographed by the photographing unit in a recording medium, the photographing control unit controls the corresponding photographing area according to the photographing condition set by the setting unit and generates the subject image, and the recording unit records the subject image in the recording medium.

[0025] According to the eighteenth embodiment of the present invention, a photographing system includes an electronic device and a photographing device of any one of the eighth to thirteenth embodiments, the electronic device also includes a first communication unit capable of communicating with the photographing device, the photographing device including: a photographing unit that controls the photographing unit with prescribed photographing conditions for each of a plurality of photographing areas; a photographing control unit that controls each of the plurality of photographing areas with the prescribed photographing conditions and generates a subject image; and a second communication unit capable of communicating with the electronic device, the second communication unit sending the subject image to the electronic device via the first communication unit, and the first communication unit sending the shooting conditions set by the setting unit for each partial area in the subject image displayed by the display control unit to the photographing device.

[0026] According to the nineteenth embodiment of the present invention, an electronic device includes: a shooting element, which has a first area and a second area different from the first area, and shoots incident light in the first area under a first shooting condition to generate a first signal, and shoots incident light in the second area under a second shooting condition to generate a second signal; and a changing unit, which changes the second shooting condition by changing the first shooting condition.

[0027] According to a twentieth aspect of the present invention, in the electronic device according to the nineteenth aspect, preferably, the changing unit changes the second imaging condition according to an amount of change in the first imaging condition.

[0028] According to a twenty-first aspect of the present invention, in the electronic device of the nineteenth or twentieth aspect, preferably, an amount of change in the second imaging condition is different from an amount of change in the first imaging condition.

[0029] According to the twenty-second aspect of the present invention, in the electronic device according to the nineteenth aspect, preferably, the changing unit changes the second imaging condition according to the light incident on the second area.

[0030] According to the twenty-third aspect of the present invention, in the electronic device of the twenty-second aspect, it is preferred that the electronic device includes a detection unit that detects the second signal output from the imaging element, and the changing unit changes the second imaging condition according to the detection result of the detection unit.

[0031] According to a twenty-fourth aspect of the present invention, in the electronic device according to the twenty-third aspect, preferably, the changing unit changes the second imaging condition according to the subject detected by the detecting unit.

[0032] According to a twenty-fifth aspect of the present invention, in the electronic device according to any one of the nineteenth to twenty-fourth aspects, preferably, the changing unit changes the second imaging condition within a range in which the second signal does not exceed a predetermined brightness threshold.

[0033] According to the twenty-sixth aspect of the present invention, in the electronic device of any one of the nineteenth to twenty-fifth aspects, it is preferred that the electronic device includes an operation unit used to change the first shooting condition, and the changing unit changes the second shooting condition by operating the operation unit.

[0034] According to the twenty-seventh embodiment of the present invention, in the electronic device of any one of the nineteenth to twenty-sixth embodiments, it is preferred that it includes a display unit that displays an image used to change the first shooting condition, and the changing unit changes the second shooting condition according to the image displayed on the display unit.

[0035] According to the twenty-eighth embodiment of the present invention, the electronic device includes: a setting unit, which sets a first shooting condition for shooting light incident on a first area of ​​a shooting element and a second shooting condition for shooting light incident on a second area of ​​the shooting element, wherein the second area is an area different from the first area; and a changing unit, which changes the second shooting condition set by the setting unit by changing the first shooting condition.

[0036] According to the twenty-ninth aspect of the present invention, in the electronic device of the twenty-eighth aspect, preferably, the electronic device includes an operation unit used to change the first shooting condition, and the changing unit changes the second shooting condition by operating the operation unit.

[0037] According to the 30th aspect of the present invention, in the electronic device of the 28th or 29th aspect, it is preferred that it includes a display unit that displays an image used to change the first shooting condition, and the changing unit changes the second shooting condition according to the image displayed on the display unit.

[0038] According to the thirty-first embodiment of the present invention, in the electronic device of any one of the nineteenth to thirtieth embodiments, it is preferred that it further includes a prohibition unit that prohibits the change unit from changing the second shooting condition so as to cause the shooting signal of the second area to deviate from a prescribed range.

[0039] According to the thirty-second embodiment of the present invention, in the electronic device of the thirty-first embodiment, preferably, the second area corresponds to at least one of the darkest part and the brightest part of the subject, and the prohibition unit prohibits the change of the second shooting condition such as exchanging the size relationship between the shooting signal of the first area and the shooting signal of the second area.

[0040] According to a thirty-third aspect of the present invention, a photographing device includes the electronic device according to any one of the nineteenth to thirty-second aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a block diagram showing the configuration of an electronic device according to the first embodiment.

[0042] Figure 2 (a) is a top view schematically showing the imaging surface of the imaging element. Figure 2 (b) is a top view in which a part of the imaging surface is enlarged.

[0043] Figure 3 (a) is a schematic diagram showing a state in which a subject image is displayed on a liquid crystal display in an information terminal device that determines that the posture of the information terminal device is in the vertical position. Figure 3 (b) is a diagram showing an example of a display mode in which a main subject area is superimposed on a subject image to be clearly displayed, and a partial image is displayed. Figure 3 (c) indicates that Figure 3 FIG. 1 is a diagram showing an example of a display form of an enlarged image of a main subject in a partial image in (a). Figure 3 (d) indicates that Figure 3 FIG. 1 is a diagram showing an example of a display form of an enlarged image of a main subject in a partial image in (a).

[0044] Figure 4 (a) is a diagram showing an example of a display form in which a person is displayed larger on a liquid crystal display in an information terminal device whose posture is determined by the posture detection unit to be in a vertical position. Figure 4 (b) is a diagram showing an example of a display mode in which the outline of the main subject region is displayed on the subject image.

[0045] Figure 5(a) is a diagram showing an example of a display form when a person displayed on a liquid crystal display is small in an information terminal device whose posture is determined by the posture detection unit to be in a vertical position. Figure 5 (b) is a diagram showing an example of a display mode in which an enlarged image of a main subject in a subject image is displayed.

[0046] Figure 6 (a) is a schematic diagram showing a state in which a subject image is displayed on a liquid crystal display in an information terminal device whose posture is determined by the posture detection unit to be in the horizontal position. Figure 6 (b) is a diagram showing an example of a display mode in which a main subject area is superimposed on a subject image to be clearly displayed, and a partial image is displayed. Figure 6 (c) indicates that Figure 6 (b) is a diagram showing an example of a display form of an enlarged image of a main subject in the partial image. Figure 6 (d) indicates that Figure 6 (b) is a diagram showing an example of a display form of an enlarged image of a main subject in the partial image.

[0047] Figure 7 This is a flowchart regarding the process of setting shooting conditions for each main subject and shooting a subject image.

[0048] Figure 8 It is a block diagram showing the configuration of an information terminal device according to the second embodiment.

[0049] Fig. 9 (a) is a diagram schematically showing a state in which a user touches a person in a subject image displayed on a liquid crystal display with a finger. Fig. 9 (b) is a diagram showing an example of a display mode in which a main subject region is superimposed on a subject image to be clearly displayed, and an enlarged image is displayed.

[0050] Fig.10 (a) is a diagram schematically showing a state in which a user touches a person in a subject image displayed on a liquid crystal display with two fingers. Fig.10 (b) is a diagram illustrating the operation of the frame line displayed on the liquid crystal display. Fig.10 (c) is a diagram illustrating the operation of the frame line displayed on the liquid crystal display. Fig.10 (d) is a diagram for explaining the operation of the frame line displayed on the liquid crystal display. Fig.10 (e) is a diagram for explaining the operation of the frame line displayed on the liquid crystal display.

[0051] Fig.11(a) is a diagram for explaining a case where a line along a trajectory drawn with a finger is displayed on a liquid crystal display. Fig.11 (b) is a diagram for explaining a case where a line along a trajectory drawn with a finger is displayed on a liquid crystal display.

[0052] Fig.12 It is a figure which shows a modification example.

[0053] Fig.13 It is a figure which shows a modification example.

[0054] Fig.14 It is a figure which shows a modification example.

[0055] Fig.15 It is a figure which shows a modification example.

[0056] Fig.16 It is a figure which shows a modification example.

[0057] Fig.17 It is a figure which shows a modification example.

[0058] Fig.18 It is a figure which shows a modification example.

[0059] Fig.19 It is a figure which shows a modification example.

[0060] Fig. 20 It is a block diagram showing the configuration of an imaging device according to a fourth embodiment of the present invention.

[0061] Fig.21 Schematically shows a subject and its brightness distribution.

[0062] Fig. 22 (a) is a histogram of the brightness of each unit group in the case where neither underexposure nor overexposure exists. Fig. 22 (b) is a histogram of the brightness of each unit group in a case where underexposure exists but overexposure does not exist. Fig. 22 (c) is a histogram of the brightness of each unit group in the case where overexposure exists but underexposure does not exist. Fig. 22 (d) is a histogram of the brightness of each unit group in the case where both underexposure and overexposure exist.

[0063] Fig.23 This is an explanatory diagram of Modification Example 7.

[0064] Fig.24 This is an explanatory diagram of modification example 8.

[0065] Fig.25 It is a cross-sectional view of a stacked imaging element.

[0066] Fig.26 This is a diagram illustrating the pixel arrangement and blocks of an imaging chip.

[0067] Fig. 27 This is a circuit diagram corresponding to the unit of the imaging chip.

[0068] Fig.28 This is a block diagram showing the functional structure of the imaging element. DETAILED DESCRIPTION

[0069] ---First Embodiment---

[0070] First, the imaging element 22 mounted in the electronic device (for example, the imaging device 10) according to one embodiment of the present invention will be described. The imaging element 22 is the element described in Japanese Patent Application No. 2012-139026 filed by the applicant of the present application. Fig.25 2 is a cross-sectional view of the imaging element 22. The imaging element 22 includes an imaging chip 2113 that outputs a pixel signal corresponding to incident light, a signal processing chip 2111 that processes the pixel signal, and a memory chip 2112 that stores the pixel signal. The imaging chip 2113, the signal processing chip 2111, and the memory chip 2112 are stacked and electrically connected to each other using conductive bumps 2109 such as Cu.

[0071] Furthermore, as shown in the figure, the incident light is mainly incident in the positive direction of the Z axis indicated by the hollow arrow. In this embodiment, in the imaging chip 2113, the side on which the incident light is incident is called the back side. In addition, as shown in the coordinate axis, the left direction of the paper plane perpendicular to the Z axis is taken as the positive direction of the X axis, and the front direction of the paper plane perpendicular to the Z axis and the X axis is taken as the positive direction of the Y axis. In the following figures, Fig.25 The coordinate axes are used as the reference and the coordinate axes are displayed in a way that the orientation of each graph is clear.

[0072] An example of the imaging chip 2113 is a CMOS image sensor. Specifically, it is a back-illuminated CMOS image sensor. The PD (PhotoDiode) layer 2106 is arranged on the back side of the wiring layer 2108. The PD layer 2106 has a plurality of photodiodes 2104 arranged two-dimensionally and storing charges corresponding to incident light, and transistors 2105 provided corresponding to the photodiodes 2104. That is, the PD layer 2106 includes a plurality of photodiodes 2104 having a photoelectric conversion function of converting light into charges, and a plurality of transistors 2105 for reading out signals generated by charges stored in the photodiodes 2104 to the wiring layer 2108.

[0073] A color filter 2102 is provided on the incident side of the PD layer 2106 via a passivation film 2103. The color filter 2102 has a plurality of types that transmit mutually different wavelength bands, and has a specific arrangement corresponding to each photodiode 2104. That is, the color filter 2102 has a plurality of filters with different spectral characteristics, and the photodiode 2104 receives the light transmitted through the filter. The arrangement of the color filter 2102 will be described later. A group of the color filter 2102, the photodiode 2104, and the transistor 2105 forms one pixel.

[0074] A microlens 2101 is provided corresponding to each pixel on the incident light incident side of the color filter 2102. The microlens 2101 focuses the incident light toward the corresponding photodiode 2104.

[0075] The wiring layer 2108 has wiring 2107 for transmitting the pixel signal from the PD layer 2106 to the signal processing chip 2111. The wiring 2107 may be multi-layered, and passive elements and active elements may be provided.

[0076] A plurality of bumps 2109 are arranged on the surface of the wiring layer 2108. The plurality of bumps 2109 are aligned with the plurality of bumps 2109 provided on the opposite surface of the signal processing chip 2111, and the imaging chip 2113 and the signal processing chip 2111 are pressurized, etc., so that the aligned bumps 2109 are bonded to each other and electrically connected.

[0077] Similarly, a plurality of bumps 2109 are arranged on the mutually facing surfaces of the signal processing chip 2111 and the memory chip 2112. The bumps 2109 are aligned with each other and the signal processing chip 2111 and the memory chip 2112 are pressed, etc., so that the aligned bumps 2109 are bonded to each other and electrically connected.

[0078] Furthermore, the connection between the bumps 2109 is not limited to the Cu bump connection by solid phase diffusion, and micro bump connection by molten solder can also be used. In addition, for example, one bump 2109 can be set for one block described later. Therefore, the size of the bump 2109 can be larger than the pitch of the photodiode 2104. In addition, a bump larger than the bump 2109 corresponding to the pixel area can also be set in the peripheral area outside the pixel area where pixels are arranged.

[0079] The signal processing chip 2111 has TSVs (Through Silicon Vias) 2110 that connect circuits disposed on the front and back surfaces. The TSVs 2110 are preferably disposed in the peripheral region. Alternatively, the TSVs 2110 may be disposed in the peripheral region of the imaging chip 2113 and the memory chip 2112.

[0080] Fig.26 This is a diagram illustrating the pixel arrangement of the imaging chip 2113. In particular, it shows the state of observing the imaging chip 2113 from the back side. In the pixel area, more than 8 million pixels are arranged, for example, in a matrix. In the present embodiment, for example, one block 2131 is formed by 4 adjacent pixels of 2 pixels × 2 pixels. Moreover, one unit group 32 is formed by 4 adjacent blocks of 2 blocks × 2 blocks. The grid lines in the figure represent the concept of forming a block 2131 and a unit group 32 by concentrating adjacent pixels. The number of pixels forming the block 2131 and the number of blocks 2131 forming the unit group 32 are not limited to the above examples, and can be more or less than them.

[0081] As shown in the partial enlarged view of the pixel area, block 2131 includes four so-called Bayer arrays in the top, bottom, left and right directions, and the Bayer array is formed by four pixels, namely, green pixels Gb, Gr, blue pixels B, and red pixels R. The green pixel is a pixel having a green filter as the color filter 2102, and receives light in the green band of the incident light. Similarly, the blue pixel is a pixel having a blue filter as the color filter 2102, and receives light in the blue band, and the red pixel is a pixel having a red filter as the color filter 2102, and receives light in the red band.

[0082] In the present embodiment, a plurality of blocks 2131 are defined for one block 2131 in such a manner that at least one group of four pixels, Gb, Gr, B and R, is included. Each block 2131 can control the four pixels in the block 2131 by the control parameters determined for each block 2131. That is, it is possible to obtain shooting signals with different shooting conditions through the pixel groups included in a certain block 2131 and the pixel groups included in other blocks 2131. That is, for the shooting conditions of the pixel groups included in a certain block 2131, the shooting action can be performed under shooting conditions different from those of the pixel groups included in other blocks 2131. Examples of control parameters are frame rate, gain, thinning rate, the number of rows or columns for adding pixel signals, the accumulation time or number of accumulation times of charge, the number of bits for digitization, etc. Moreover, the control parameters can also be parameters in image processing after the image signal is acquired from the pixel. That is, for the frame rate, gain, thinning-out rate, the number of addition rows or addition columns for adding pixel signals, the charge accumulation time, the number of charge accumulation times and the number of digitized bits of the pixel group included in a certain block 2131, the shooting action can be performed with a frame rate, gain, thinning-out rate, the number of addition rows or addition columns for adding pixel signals, the charge accumulation time, the number of charge accumulation times and the number of digitized bits that are different from those of the pixel group included in other blocks 2131.

[0083] Fig. 27 This is a circuit diagram corresponding to one unit group 32 of the imaging chip 2113 . Fig. 27 In the diagram, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel. In addition, a rectangle surrounded by a dashed line corresponds to one block 2131. In addition, at least a portion of each transistor described below corresponds to Fig.25 corresponds to transistor 2105.

[0084] As described above, the unit group 32 is formed of four blocks 2131. The reset transistors 2303 of the pixels included in the unit group 32 are turned on / off in units of the blocks 2131. In addition, the transfer transistors 2302 of the pixels included in the unit group 32 are also turned on / off in units of the blocks 2131. Fig. 27 In the example shown, a reset wiring 2300 - 1 for turning on / off four reset transistors 2303 corresponding to the upper left block 2131 - 1 is provided, and a TX wiring 2307 - 1 for supplying a transmission pulse to four transmission transistors 2302 corresponding to the block 2131 - 1 is provided.

[0085] Similarly, a reset wiring 2300-3 for turning on / off four reset transistors 2303 corresponding to the lower left block 2131-3 is provided separately from the reset wiring 2300-1. In addition, a TX wiring 2307-3 for supplying a transmission pulse to four transmission transistors 2302 corresponding to the block 2131-3 is provided separately from the TX wiring 2307-1.

[0086] Similarly, in the upper right block 2131 - 2 and the lower right block 2131 - 4 , the reset wiring 2300 - 2 and the TX wiring 2307 - 2 , and the reset wiring 2300 - 4 and the TX wiring 2307 - 4 are provided in each block 2131 , respectively.

[0087] The 16 photodiodes 2104 corresponding to each pixel are respectively connected to the corresponding transfer transistor 2302. A transfer pulse is supplied to the gate of each transfer transistor 2302 via the TX wiring of each of the above blocks 2131. The drain of each transfer transistor 2302 is connected to the source of the corresponding reset transistor 2303, and the so-called floating diffusion FD between the drain of the transfer transistor 2302 and the source of the reset transistor 2303 is connected to the gate of the corresponding amplifier transistor 2304.

[0088] The drains of the reset transistors 2303 are commonly connected to a Vdd wiring 2310 to which a power supply voltage is supplied. A reset pulse is supplied to the gates of the reset transistors 2303 via the reset wiring of each of the blocks 2131 .

[0089] The drain of each amplifier transistor 2304 is commonly connected to a Vdd wiring 2310 to which a power supply voltage is supplied. In addition, the source of each amplifier transistor 2304 is connected to the drain of the corresponding selection transistor 2305. A decoder wiring 2308 to which a selection pulse is supplied is connected to the gate of each selection transistor 2305. In this embodiment, the decoder wiring 2308 is independently provided for each of the 16 selection transistors 2305. Moreover, the source of each selection transistor 2305 is connected to a common output wiring 2309. The load current source 2311 supplies current to the output wiring 2309. That is, the output wiring 2309 provided for the selection transistor 2305 is formed by a source follower. In addition, the load current source 2311 can be provided on the imaging chip 2113 side or on the signal processing chip 2111 side. In addition, when the number of pixels included in the unit group 32 is one, the selection transistor 2305 can be omitted.

[0090] Here, the process from the start of charge accumulation to pixel output after the accumulation is completed is described. When a reset pulse is applied to the reset transistor 2303 through the reset wiring of each block 2131 and a transfer pulse is applied to the transfer transistor 2302 through the TX wiring of each block 2131, the potential of the photodiode 2104 and the floating diffusion FD are reset for each block 2131.

[0091] Each photodiode 2104 converts the received light into electric charge and accumulates it when the application of the transfer pulse is released. Thereafter, when the transfer pulse is applied again without applying the reset pulse, the electric charge accumulated in the photodiode 2104 is transferred to the floating diffusion FD. The electric potential of the floating diffusion FD changes from the reset potential to the signal electric potential after the electric charge is accumulated. That is, the electric potential of the floating diffusion FD becomes the electric potential corresponding to the electric charge accumulated in the photodiode 2104. Moreover, when the selection pulse is applied to the selection transistor 2305 through the decoder wiring 2308, the change of the signal electric potential of the floating diffusion FD is transmitted to the output wiring 2309 via the amplifier transistor 2304 and the selection transistor 2305. That is, when the selection pulse is applied to the selection transistor 2305 through the decoder wiring 2308, the signal generated by the electric potential of the floating diffusion FD is transmitted to the output wiring 2309. Thus, the pixel signal corresponding to the reset electric potential and the signal electric potential is output from the unit pixel to the output wiring 2309.

[0092] As described above, in this embodiment, the reset wiring and the TX wiring are common to the four pixels forming the block 2131. That is, the reset pulse and the transmission pulse are applied to the four pixels in the block 2131 at the same time. Therefore, all the pixels forming a certain block 2131 start charge accumulation at the same timing and end charge accumulation at the same timing. However, by sequentially applying the selection pulse to each selection transistor 2305, the pixel signal corresponding to the accumulated charge is selectively output from the output wiring 2309.

[0093] In this way, in the present embodiment, the charge accumulation start timing can be controlled for each block 2131. In other words, imaging can be performed at different timings between different blocks 2131.

[0094] Fig.28 2 is a block diagram showing the functional structure of the imaging element 22. The analog multiplexer 2411 sequentially selects the 16 photodiodes 2104 forming the unit group 32, and outputs the respective pixel signals to the output wiring 2309 provided corresponding to the unit group 32. The multiplexer 2411 is formed in the imaging chip 2113 together with the photodiodes 2104.

[0095] The pixel signal outputted from the multiplexer 2411 is subjected to CDS and A / D conversion by a signal processing circuit 2412 formed in the signal processing chip 2111 and performing CDS (Correlated Double Sampling) and analog / digital (A / D) conversion. The pixel signal after A / D conversion is transmitted to the demultiplexer 2413 and stored in the pixel memory 2414 corresponding to each pixel. The demultiplexer 2413 and the pixel memory 2414 are formed in the memory chip 2112.

[0096] The operation circuit 2415 processes the pixel signal stored in the pixel memory 2414 and transmits it to the subsequent image processing unit. The operation circuit 2415 can be set in the signal processing chip 2111 or the memory chip 2112. Fig.28 , but in reality, these exist for each unit group 32 and operate in parallel. However, the operation circuit 2415 may not be provided for each unit group 32. For example, one operation circuit 2415 may sequentially perform processing while referring to the values ​​of the pixel memory 2414 corresponding to each unit group 32 in sequence.

[0097] As described above, output wirings 2309 are provided corresponding to the respective unit groups 32. Since the imaging element 22 is stacked with the imaging chip 2113, the signal processing chip 2111, and the memory chip 2112, by using the electrical connection between the chips using the bumps 2109 for these output wirings 2309, the wirings can be arranged without enlarging the size of each chip in the surface direction.

[0098] Figure 1 1 is a block diagram showing the structure of an electronic device according to the first embodiment of the present invention. In this embodiment, as an example of an electronic device, an information terminal device 10 having a shooting function is described as an example. The information terminal device 10 is a device that is thin, has a large liquid crystal display on the front surface, and has a shooting optical system on the back side, such as a tablet-type information terminal device. The information terminal device 10 includes a shooting optical system 21, a shooting element 22, a control unit 23, a liquid crystal display 24, a memory card 25, an operating unit 26, a DRAM 27, a flash memory 28, and a posture detection unit 29. In addition, in the following description, among the various functions and structures of the information terminal device 10, the functions and structures related to shooting are described, and the description of other functions and structures is omitted.

[0099] The imaging optical system 21 is composed of a plurality of lenses, and forms a subject image on the imaging surface of the imaging element 22. Figure 1 In FIG. 1 , the photographing optical system 21 is illustrated as one lens.

[0100] The imaging unit, i.e., the imaging element 22, captures the subject image formed by the imaging optical system 21 and outputs an imaging signal. The imaging element 22 has an imaging surface 30 on which a plurality of imaging pixels are arranged. The control unit 23 is an electronic circuit that controls each part of the information terminal device 10, and is composed of a CPU and its peripheral circuits. A predetermined control program is pre-written in the flash memory 28 as a non-volatile recording medium. The control unit 23 controls each part by reading the control program from the flash memory 28 and executing it. The control program uses the DRAM 27 as a volatile recording medium as an operation area.

[0101] The liquid crystal display 24 as a display unit is a display device using a liquid crystal panel. The control unit 23 repeatedly operates the imaging element 22 to capture the subject image at a predetermined period (for example, one sixtieth of a second). Then, various image processing is performed on the shooting signal output from the imaging element 22 to generate a subject image, and the subject image is displayed on the liquid crystal display 24. In addition to the above-mentioned subject image, the liquid crystal display 24 also displays, for example, the captured image data described later and a setting screen for setting the shooting conditions. In addition, the display device may be a display device using, for example, an organic EL panel other than the liquid crystal display 24.

[0102] The posture detector 29 is, for example, an acceleration sensor, and detects the posture of the information terminal 10. The control unit 23 determines based on the signal output from the posture detector 29 whether the information terminal 10 is in a vertical position for capturing a vertically long image or in a horizontal position for capturing a horizontally long image.

[0103] The control unit 23 performs various image processing on the shooting signal output from the imaging element 22 to generate the captured image data, and records the image data in the memory card 25 which is a portable recording medium. The operation unit 26 has various operation members such as a push button, and outputs an operation signal to the control unit 23 according to the operation of these operation members. In addition, the liquid crystal display 24 is a touch screen display device, and constitutes a part of the operation unit 26. The control unit 23 performs display control so that the liquid crystal display 24 displays the subject image captured by the imaging element 22.

[0104] The control unit 23 includes an object recognition unit 23a, a weight setting unit 23b, an area division unit 23c, a shooting condition setting unit 23d, an image processing unit 23e, and a shooting control unit 23f by software. These parts are implemented by software by the control unit 23 executing a predetermined control program stored in the flash memory 28. In addition, these parts can also be configured as electronic circuits.

[0105] The object recognition unit 23a recognizes or detects people, their facial parts, animals such as dogs and cats, vehicles such as bicycles, cars, and electric vehicles, food on a plate, toys, and other pre-specified specific subject objects, i.e., subject elements, from the subject image captured by the camera element 22 through methods such as the well-known so-called template matching processing.

[0106] The weight setting unit 23b weights the subject objects recognized by the object recognition unit 23a. Specifically, the weight setting unit 23b assigns a predetermined weight to each subject object recognized by the object recognition unit 23a, such as a person, a face, an animal, a vehicle, food, a toy, etc. For example, the weight setting unit 23b assigns the largest weight to a person and a face, then assigns the second largest weight to an animal, assigns the third largest weight to a vehicle, and assigns a small weight to food and toys, etc. In this way, the weight setting unit 23b ranks the subject objects by assigning predetermined weights to the subject objects recognized by the object recognition unit 23a.

[0107] The area division unit 23c determines whether the number of the subject objects recognized by the object recognition unit 23a and weighted by the weight setting unit 23b exceeds a predetermined number. If the number of the subject objects is within the predetermined number, all the subject objects recognized by the object recognition unit 23a are determined as the main subjects. If the number of the subject objects exceeds the predetermined number, the subject objects are selected from the objects with a relatively high ranking until the predetermined number is reached, and the predetermined number of subject objects are determined as the main subjects. In addition, the user can set the predetermined number in advance through the operation unit 26.

[0108] The area division unit 23c further determines at which position, that is, in which area, the main subject determined in the above manner exists in the subject image, and divides (partitions) the area of ​​the main subject in the subject image into partial areas. As described later, when the liquid crystal display 24 displays the subject image, the area of ​​the main subject divided by the area division unit 23c, specifically, the frame line indicating the outer shape or contour of the main subject, is displayed in a manner overlapping with the subject image. In addition, the area division unit 23c divides the area of ​​the main subject on the subject image, which means that the shooting area formed by shooting the main subject is determined, that is, divided, on the shooting surface of the shooting element 22.

[0109] The shooting condition setting unit 23d sets the shooting conditions of the imaging element 22, and sets the image processing conditions for the main subject as described in detail later. The image processing unit 23e performs multiple image processing on the subject image. The shooting control unit 23f controls each of the multiple shooting areas (the multiple shooting areas each include at least one shooting pixel of the imaging element 22) independently of each other. That is, the shooting control unit 23f controls the shooting conditions such as exposure time, magnification, frame rate, etc. for each unit group 32 of the imaging element 22 as described above. In addition, the function of the shooting control unit 23f can also be set in the imaging element 22.

[0110] Figure 2 (a) is a top view schematically showing an imaging surface 30 of the imaging element 22. Figure 2 (b) is a top view of a part of the imaging surface 30a which is enlarged. Figure 2 As shown in (b), a plurality of imaging pixels 31 are arranged two-dimensionally on the imaging plane 30. Each of the imaging pixels 31 has a color filter (not shown). The color filters include three types: red (R), green (G), and blue (B). Figure 2 The symbols "R", "G" and "B" in (b) indicate the types of color filters that the image capturing pixels 31 have. Figure 2As shown in FIG. 2( b ), imaging pixels 31 having such color filters are arranged in a so-called Bayer array on an imaging plane 30 of the imaging element 22 .

[0111] The imaging pixel 31 with a red filter performs photoelectric conversion on the light in the red wavelength band of the incident light and outputs a light receiving signal (photoelectric conversion signal). Similarly, the imaging pixel 31 with a green filter performs photoelectric conversion on the light in the green wavelength band of the incident light and outputs a light receiving signal. In addition, the imaging pixel 31 with a blue filter performs photoelectric conversion on the light in the blue wavelength band of the incident light and outputs a light receiving signal.

[0112] The imaging element 22 of this embodiment is configured to be able to control each imaging area (composed of a total of 4 imaging pixels 31 of adjacent 2 pixels × 2 pixels), that is, a unit group 32. For example, when charge accumulation is started simultaneously for two different unit groups 32, the charge can be read out 1 / 30 second after the charge accumulation starts for one unit group 32, that is, the light receiving signal can be read out, and the charge can be read out 1 / 15 second after the charge accumulation starts for the other unit group 32. In addition, the charge accumulation start timing of one unit group 32 can be different from the charge accumulation start timing of another unit group 32. In other words, the imaging element 22 can set a different exposure time (so-called shutter speed) for each unit group 32 in one shooting.

[0113] In addition to the above-mentioned exposure time, the imaging element 22 can also make the amplification factor (so-called ISO sensitivity) of the light-receiving signal different for each unit group 32. The imaging element 22 can change the timing of starting charge accumulation and the timing of reading the light-receiving signal for each unit group 32. The imaging element 22 can change the frame rate when shooting a moving image for each unit group 32. Figure 1 The imaging control unit 23 f of the control unit 23 shown controls imaging conditions such as exposure time, magnification, and frame rate for each unit group 32 .

[0114] As described above, the imaging element 22 is configured so that imaging conditions such as exposure time, magnification, and frame rate can be made different for each unit group 32 .

[0115] The information terminal device 10 thus configured is configured to manually set the imaging conditions for the main subject, which is an important subject object detected by the object recognition unit 23a, separately from the imaging conditions for other subject parts such as the subject background.

[0116] Figure 3(a) is a schematic diagram showing a state in which a subject image 110 is displayed on the liquid crystal display 24 in the information terminal device 10 in which the posture detection unit 29 determines that the posture of the information terminal device is in the vertical position. In the following description, as an example of a subject, two people 101, 102, a small animal 103, a table 105, and a cake 104 placed on a plate as food exist within the shooting range of the information terminal device 10. Moreover, when viewed from the information terminal device 10 side, the two people 101, 102 are located on the other side of the table 105, the small animal 103 is under the table 105, and the cake 104 is placed on the plate on the table 105. Moreover, for the convenience of description, it is assumed that there is no subject that can be extracted as other subject elements in the background part 108 of the subject image 110. The subject image 110 is captured under the same shooting conditions (exposure time, magnification, frame rate) for all unit groups 32, i.e., the entire shooting surface 30, based on exposure conditions obtained based on the metering value of the subject brightness or exposure conditions manually set by the user.

[0117] The object recognition unit 23a recognizes specific subject objects, namely, two persons 101, 102, a small animal 103, a cake 104 on a plate, and a table 105, from the subject image 110 displayed on the liquid crystal display 24. The object recognition unit 23a performs subject detection using image data representing the subject image 110 output from the imaging element 22. As a result of the subject detection, the object recognition unit 23a recognizes specific subject objects, namely, two persons 101, 102, a small animal 103, a cake 104 on a plate, and a table 105.

[0118] When the object recognition unit 23a detects a plurality of subject objects, the weight setting unit 23b weights each of the plurality of subject objects detected by the object recognition unit 23a. Here, the weight setting unit 23b assigns the largest weight to the two persons 101 and 102, the second largest weight to the small animal 103, the third largest weight to the cake 104 on the plate, and the fourth largest weight to the table 105.

[0119] The region division unit 23c determines the top four objects, for example, among the five subject objects detected by the object recognition unit 23a and weighted by the weight setting unit 23b, as the main subjects, and determines or divides the positions of the four main subjects in the subject image, that is, the regions of the four main subjects, into main subject regions. In the following description, determination or division of the main subject region is also referred to as region division.

[0120] When the main subject area is determined or divided by the area division unit 23c, as shown in FIG. Figure 3As shown in (b) of FIG. 1 , the control unit 23 causes the four main subject regions to overlap with the subject image 110A displayed in the display area 111 to be clearly displayed. For example, the control unit 23 causes the outlines 101a, 102a, 103a, and 104a of the four main subject regions to be displayed as dashed lines on the subject image 110A. Instead of displaying the outlines as dashed lines as shown in the figure, the outline display 101a, 102a, 103a, and 104a may be displayed so that each main subject region is brighter and highlighted than other regions as a whole, or is colored and highlighted.

[0121] The control unit 23 generates a partial image 181 of the main subjects 101 to 104 based on the subject image 110A. The control unit 23 displays the outlines 101a, 102a, 103a, and 104a of the main subject regions on the subject image 110A. Figure 3 As shown in (b) of FIG. 2 , a partial image display area 190 is further provided below the subject image 110A, specifically, at the lower part of the liquid crystal display 24, so that partial images 181 of the main subjects 101 to 104 are displayed. That is, the control unit 23 displays the partial areas determined by the area division unit 23c as the outlines 101a, 102a, 103a, and 104a of the main subject areas, and displays the partial images 181 corresponding to the areas of the main subjects in the subject image. In addition, the control unit 23 displays the setting screen 130 of the shooting conditions of the main subject in the upper right display area 114 of the display screen of the liquid crystal display 24. In this way, the partial images 181 are displayed at the lower side of the display screen so that the user can easily select the partial images with his fingers. The setting screen 130 is displayed on the side of the user's dominant hand so that the user can easily set the shooting conditions. Figure 3 In the embodiment shown in (b), it is assumed that the user's dominant hand is the right hand. If the user's dominant hand is the left hand, the image is displayed on the left side of the display screen. In addition, even if the setting screen 130 and the partial image display area 190 are set in the subject image 110A, the control unit 23 controls the display in a manner that the aspect ratio of the subject image 110A (the ratio of the vertical length to the horizontal length of the subject image 110A) does not change. In addition, the control unit 23 makes it difficult for the subject image 110A to be hidden by the user's fingers by displaying the partial image 181 on the lower side of the display screen, thereby improving the visibility of the subject image 110A.

[0122] The setting screen 130 of the shooting condition of the main subject includes, as an example of the shooting condition, that is, an example of the setting item 131 of the shooting condition, for example, "shutter speed", "ISO sensitivity", and "image processing effect" in order from the top. The image processing effect refers to a setting item set when performing image processing for obtaining, for example, skin correction taking whitening into consideration, black and white images, and nostalgic images. Thus, the setting item 131 of the setting screen 130 includes "shutter speed" and "ISO sensitivity" as shooting conditions, and "image processing effect" as image processing conditions, but in the following description, it is also expressed only as "shooting condition". Moreover, in the case of performing dynamic image shooting, the setting item 131 of the shooting condition is also displayed in the setting screen 130. "Frame rate" is additionally displayed. In this way, the setting change of the frame rate for the main subject can be performed.

[0123] As each setting item 131, an item display 131a, a setting status display 131b, an up button display 131c, and a down button display 131d are displayed respectively. For example, when "shutter speed" is used as an example, the text "shutter speed" indicating the shutter speed as a setting item is displayed in the item display 131a, and the text "1 / 125 seconds" indicating the current setting state of the shutter speed is displayed in the setting state display 131b. The up button display 131c and the down button display 131d are display buttons for changing the current setting state. When the up button display 131c or the down button display 131d is touched, the setting state is changed in sequence. In addition, a decision button 132 for determining the set shooting conditions is displayed at the lower right of the display area 114. The same is true for "ISO sensitivity". Furthermore, until the user selects a main subject area and changes the imaging conditions of the main subject area as described later, the setting state is set to the same “shutter speed” and “ISO sensitivity” for all unit groups 32 , that is, the entire imaging surface 30 .

[0124] When a user selects a certain partial image 181, Figure 3 As shown in (c) of FIG. 1 , the control unit 23 displays the enlarged image of the main subject of the selected partial image 181 on the lower side of the liquid crystal display 24. That is, the control unit 23 displays the partial image corresponding to the area of ​​the main subject in the subject image as an enlarged image. The control unit 23 displays the image of the partial image display area 190 in an enlarged manner compared with the image representing the main subject of the subject image 110A displayed in the display area 111. Figure 3 (c) means Figure 3 In (b) , the partial image 181 of the person 101 is selected, and the enlarged image 121 of the person 101 is displayed in the partial image display area 190 . Figure 3(d) means Figure 3 In (b), the partial image 181 of the cake 104 is selected, and the enlarged image 124 of the cake 104 is displayed in the partial image display area 190. That is, the image of the cake 104 displayed in the partial image display area 190 is enlarged and displayed in comparison with the image representing the main subject of the subject image 110A displayed in the display area 111, thereby indicating that the cake 104 has been selected. In this way, by causing the liquid crystal display 24 to display the enlarged image of the main subject, even if the display of the main subject in the subject image 110A is small, the user can easily make the desired setting because the enlarged image of the main subject is visible.

[0125] The control unit 23 displays the enlarged image in the partial image display area 190 and displays the return button 182 at the lower right of the liquid crystal display 24. When the user presses the return button 182, the control unit 23 returns the display mode of the partial image display area 190 to the previous state. Figure 3 The display form shown in (b).

[0126] like Figure 3 As shown in (c), in a state where the partial image display area 190 displays an enlarged image of the main subject 101, for various setting items 131a of the main subject setting 130, the user touches the up button display 131c and the down button display 131d to set shooting conditions such as shutter speed.

[0127] For example, Figure 3 (c) shows the following example, that is, since the person 101 may move when photographing, the shutter speed setting value is changed from 1 / 125 second to 1 / 250 second, and the image processing effect is set to the whitening mode in order to make the photo appear white.

[0128] When the decision button 132 at the lower right of the display area 114 is operated, the shooting condition setting unit 23d sets the shooting condition displayed in the display area 114 as the shooting condition for the person 101. Specifically, the shooting condition setting unit 23d sets the exposure time (charge accumulation time) and magnification of the plurality of unit groups 32 of the imaging element 22 corresponding to the person 101 as the main subject obtained by the area division unit 23c to the imaging control unit 23f based on the set shutter speed and ISO sensitivity. More specifically, the shooting condition setting unit 23d sets the exposure time and magnification of the plurality of unit groups 32 (the plurality of unit groups 32 output image signals related to the person 101 as the main subject) in the imaging plane 30 of the imaging element 22 to the imaging control unit 23f. At the same time, the shooting condition setting unit 23d sets the whitening mode for the main subject to the image processing unit 23e.

[0129] The imaging element 22 generates a subject image captured under the imaging conditions set by the imaging condition setting unit 23d for the main subject. The image processing unit 23e performs the whitening process set by the imaging condition setting unit 23d on the main subject. The control unit 23 causes the subject image subjected to the whitening process to be displayed on the liquid crystal display 24. The user can check the effects of the imaging conditions and image processing conditions set by the user through the subject image.

[0130] Thereafter, when a release button (not shown) or a button indicating shooting is operated, the control unit 23 generates subject image data obtained by shooting and processing each main subject under the shooting conditions and image processing conditions set respectively by the shooting condition setting unit 23d, and records it in the memory card 25.

[0131] In the main subject setting 130, when the user sets the shooting condition of the main subject, a partial image is selected from the plurality of partial images 181, and the control unit 23 enlarges and displays the selected partial image and sets the shooting condition of the main subject. Alternatively, when the user selects a partial image, the shooting condition of the main subject can be set without displaying the enlarged image. Furthermore, when the user selects a partial image, the shooting condition of the main subject can be set without displaying the plurality of partial images. Figure 3 The main subject highlighted by the dotted line in the live view image of (b) is selected and the shooting conditions are set.

[0132] In the above description, the case where there are multiple main subjects is described. However, when there are only one or two main subjects, the control unit 23 controls each part as follows regarding the setting of the shooting conditions for the main subjects. Figure 4 (a) Figure 5 (a) is a schematic diagram showing a state in which a subject image of a person 106 is displayed on the liquid crystal display 24 in the information terminal device 10 determined by the posture detection unit 29 as being in the vertical position. Figure 4 In (a), the person 106 is displayed in a larger size on the liquid crystal display 24, that is, the proportion of the person 106 occupying the display area of ​​the liquid crystal display 24 is larger than a predetermined proportion. Figure 5 In (a), the person 106 displayed on the liquid crystal display 24 is displayed in a small size, that is, the proportion of the person 106 occupying the display area of ​​the liquid crystal display 24 is smaller than a predetermined proportion.

[0133] exist Figure 4 (a) Figure 5In the state shown in (a) of FIG. 1 , the object recognition unit 23a extracts the person 106 as the subject object, and the weight setting unit 23b gives the person 106 the greatest weight. The area segmentation unit 23c determines the subject object 106 as the main subject. Fig.18 ) is displayed on the liquid crystal display 24, the persons 106 and 107 are determined as the main subjects respectively.

[0134] like Figure 4 As shown in (a) of FIG. 1 , when there is only one main subject and the proportion of the main subject occupying the shooting range is greater than a predetermined proportion, the control unit 23 performs the above-mentioned area division, such as Figure 4 As shown in (b) of FIG. 1 , the control unit 23 causes the outline 106a of the main subject area to be displayed on the subject image 110B. In addition, the control unit 23 causes the liquid crystal display 24 to display a setting screen 130 for the shooting conditions for the person 106 as the main subject. For example, the control unit 23 causes the setting screen 130 for the shooting conditions for the person 106 as the main subject to be displayed on the upper right of the liquid crystal display 24. That is, the control unit 23 does not display the outline 106a of the main subject area on the subject image 110B. Figure 3 The setting screen 130 and the partial image display area 190 shown in (b) and the like.

[0135] When the control unit 23 changes the setting state display 131 b of each setting item 131 and the determination button 132 is touched, the imaging condition setting unit 23 d sets the changed imaging condition as the imaging condition for the person 106 .

[0136] As a result, the person 106 in the subject image 110B is displayed according to the set shooting conditions and image processing conditions. Although not shown in the figure, the same applies to the case where there are two main subjects and the proportion of the main subjects occupying the shooting range is larger than the specified proportion. In this case, when the image of the person 106 as the main subject is selected, the shooting conditions for the person 106 can be set, and when the image of the person 107 is selected, the shooting conditions for the person 107 can be set.

[0137] When Figure 4 When a release button not shown in the figure or a button indicating shooting is operated in the state shown in (b), the control unit 23 generates subject image data obtained by shooting and processing the main subject under the shooting conditions and image processing conditions set by the shooting condition setting unit 23d, and records it in the memory card 25.

[0138] like Figure 5As shown in (a) of FIG. 1 , the control unit 23 performs the above-mentioned area division when there is only one main subject and the proportion of the main subject occupying the shooting range is less than a predetermined proportion, as shown in FIG. Figure 5 As shown in (b) of FIG. 1 , the control unit 23 displays the outline 106a of the main subject area on the subject image 110C and displays an enlarged image 125 of the person 106 as the main subject in the partial image display area 190 below the subject image 110C, specifically, in the lower part of the liquid crystal display 24. A setting screen 130 for the shooting conditions for the person 106 as the main subject is displayed in the upper right display area 114 of the screen.

[0139] When the control unit 23 changes the setting state display 131 b of each setting item 131 and the determination button 132 is operated, the imaging condition setting unit 23 d sets the changed imaging condition as the imaging condition for the person 106 .

[0140] As a result, the control unit 23 displays the person 106 in the subject image 110C and the person 106 in the enlarged image 125 in accordance with the set shooting conditions and image processing conditions.

[0141] When Figure 5 When a release button (not shown) or a button for instructing shooting is operated in the state shown in (b) of FIG. 1 , the control unit 23 generates subject image data obtained by shooting and processing the main subject under the shooting conditions and image processing conditions set by the shooting condition setting unit 23d, and records the data in the memory card 25. Although not shown, the same applies to the case where there are two main subjects and the proportion of the main subjects occupying the shooting range is smaller than the prescribed proportion.

[0142] -----In the case where the information terminal device 10 is in the horizontal position-----

[0143] When the posture detection unit 29 determines that the posture of the information terminal device 10 is in the horizontal position, the control unit 23 causes the liquid crystal display 24 to display, for example, Figure 6In the following description, as an example of a subject, there are two persons 101, 102, a small animal 103, a table 105A, 105B, 105C, and cakes 104A, 104B placed on a plate as food in the shooting range of the information terminal device 10. The object recognition unit 23a recognizes the two persons 101, 102, the small animal 103, the table 105A, 105B, 105C, and the cakes 104A, 104B as subject objects, and the weight setting unit 23b performs a predetermined weighting on the eight subject objects and sorts the subject objects. The region segmentation unit 23c determines five subject objects, namely, the persons 101, 102, the small animal 103, and the cakes 104A, 104B, from the sorted eight subject objects as the main subjects.

[0144] Figure 6 (b) is a display screen of the liquid crystal display 24 including a subject image in which five subject objects 101, 102, 103, 104A, and 104B determined as main subjects are highlighted by dotted lines. Figure 6 (b) is the case where the posture detection unit 29 determines that the information terminal device 10 is in the vertical position. Figure 3 (b) is equivalent to FIG. When the posture detection unit 29 determines that the posture of the information terminal device 10 is in the horizontal position, three partial images 126 to 128 of the five main subjects are displayed in the display areas 146 to 148 on the left side of the liquid crystal display 24, respectively. On the right side of the screen, a setting screen 130 for the shooting conditions of the main subjects is displayed. In addition, when the operation marks 115 and 116 of the display areas 146 and 148 are operated, the control unit 23 replaces the images of the three main subjects displayed in the display areas 146 to 148 with the partial images of the remaining two main subjects in sequence.

[0145] Here, for example, Figure 6 When the image 128 of the cake 104A displayed in the display area 148 in the state shown in (b) is selected, as shown in FIG. Figure 6 As shown in (c) of FIG. 1 , the control unit 23 displays the enlarged image 129 of the cake 104A in the partial image display area 149 on the left side of the liquid crystal display 24. Thus, the user can appropriately change the shooting conditions for the cake 104A as the main subject from the setting screen 130. Since the enlarged image 129 of the cake 104A is displayed, the visibility of the enlarged image 129 is good, and the setting desired by the user becomes easy.

[0146] Alternatively, the control unit 23 may be configured such that the subject image 110E of the cake 104A, which is the setting target of the shooting condition, is not hidden by the partial image display area 149 or the setting screen 130. Figure 6 As shown in (d) of FIG. 1 , the control unit 23 appropriately changes the display range of the subject image 110E. Figure 6 As shown in (d) of FIG. 1 , an example is shown in which the control unit 23 changes the display range of the subject image 110E on the liquid crystal display 24 so that the subject image of the cake 104A is displayed in the display area 152 for displaying the subject image between the partial image display area 149 and the setting screen 130. Figure 6 As shown in (d) of FIG. 1 , the visibility of the setting screen 130 may be improved by not displaying the subject image 110E in the display area of ​​the setting screen 130 for shooting conditions.

[0147] When the decision button 132 at the lower right of the display area 151 is operated, the shooting condition setting unit 23d sets the shooting condition displayed on the display area 151 to the shooting condition for the cake 104A. As a result, the cake 104A in the subject image 110E and the cake 104A in the enlarged image 129 of the display area 149 are displayed according to the set shooting conditions and image processing conditions. The same applies to the setting of the shooting conditions and image processing conditions related to other main subjects. The display and release actions of the subject image after the shooting conditions and image processing conditions related to the main subject are the same as in the above example. Furthermore, it is also possible to Figure 6 When the enlarged image 129 of the display area 149 in (c) is operated twice within a short period of time, the display mode of the liquid crystal display 24 returns to Figure 6 The state shown in (b).

[0148] Furthermore, the display mode of the liquid crystal display 24 is the same when there is only one main subject and the proportion of the main subject occupying the shooting range is less than the prescribed proportion. In this case, the enlarged image on the left side of the screen is displayed in the center in the vertical direction. In addition, when there are only one or two main subjects, the display mode of the liquid crystal display 24 is the same when the proportion of the main subject occupying the shooting range is greater than the prescribed proportion. Figure 4 (a) Figure 4 (b) The same display form.

[0149] ---flow chart---

[0150] Figure 7 1 is a flowchart of a process for setting shooting conditions for each of the above-mentioned main subjects and shooting the subject image. Figure 7The program indicating the processing is started and executed by the control unit 23. In step S1, the control unit 23 starts the application program for photographing and proceeds to step S3. In step S3, the photographing control unit 23f controls the photographing pixels 31 of the photographing element 22 so that the charge accumulation time and the amplification factor are the same values ​​for all unit groups 32, for example, determined based on the brightness of the subject, and displays the subject image obtained based on the photographing signal from the photographing pixels 31 controlled in this way on the liquid crystal display 24.

[0151] In step S7, the object recognition unit 23a recognizes or extracts the subject object, i.e., the subject element, from the subject image 110 displayed on the liquid crystal display 24 as the subject image 110, and the process proceeds to step S9. The object recognition unit 23a recognizes or extracts the subject object, i.e., the subject element, based on the image data representing the subject image 110 output from the imaging element 22. In step S9, the weight setting unit 23b weights the subject element extracted by the object recognition unit 23a, and the process proceeds to step S11. In step S11, the region segmentation unit 23c determines the predetermined number of elements with the highest importance set by the weight setting unit 23b among the subject elements extracted by the object recognition unit 23a as the main subject, performs region segmentation, and the process proceeds to step S15.

[0152] In step S15, the control unit 23 controls each part so that the main subject is clearly displayed on the subject image, for example, by displaying the main subject with a dotted line or highlighting the outline of the main subject, and then proceeds to step S17. In step S17, the control unit 23 determines whether there are multiple main subjects. In this embodiment, if the number of main subjects is a predetermined number, for example, more than three, it is assumed that there are multiple main subjects. If the determination in step S17 is positive, in step S19, the control unit 23, for example, Figure 3 As shown in (b) of FIG. 1 , the liquid crystal display 24 displays the subject image 110A and the partial image 181 of the main subject, and the process proceeds to step S21. In step S21, the control unit 23 sets the shooting conditions for each main subject according to the user's operation, and the process proceeds to step S23. In addition, the display switching of the liquid crystal display 24 and the setting of the shooting conditions according to the user's operation in step S21 are as described above.

[0153] In step S23, the control unit 23 determines whether there is a shooting instruction from the user, that is, whether a release button (not shown) or a button for instructing shooting is operated. If the determination in step S23 is positive, the process proceeds to step S25, and if the determination in step S23 is negative, the process returns to step S21.

[0154] In step S25, the control unit 23 controls the imaging element 22 in such a manner that each main subject is photographed according to the shooting conditions set for each main subject as described above, and performs prescribed image processing on the main subject of the subject image obtained by photographing as described above, generates captured image data, and records it in the memory card 25.

[0155] In step S27, the control unit 23 determines whether to continue shooting. For example, the control unit 23 determines whether there is an instruction to end the shooting application. If it is determined in step S27 that the shooting is to continue, the process returns to step S7. If it is determined that the shooting is to end, the shooting application is ended.

[0156] If the judgment in step S17 is negative, that is, it is judged that the number of main subjects is 1 or 2, the process proceeds to step S31. In step S31, the control unit 23 determines whether the proportion of the main subject in the shooting range is larger than the specified proportion, that is, it determines whether the image of the main subject is larger than the specified size. If the judgment in step S31 is negative, that is, the proportion of the main subject in the shooting range is less than the specified proportion, the process proceeds to step S19.

[0157] If the judgment in step S31 is affirmative, that is, if the proportion of the main subject occupying the shooting range is greater than the prescribed proportion, the process proceeds to step S35. In step S35, the control unit, for example, Figure 4 As shown in (b) of FIG. 1 , the liquid crystal display 24 displays the subject image 110B and the setting screen 130 for the shooting conditions, and the process proceeds to step S21 .

[0158] In the above-mentioned first embodiment, the following effects are achieved.

[0159] (1) The main subject region in the subject image is divided into partial regions, and the partial image and / or enlarged image of the main subject and the subject image are displayed on the liquid crystal display 24. This allows the user to easily recognize the main subject, thereby improving convenience and providing an electronic device with good usability.

[0160] (2) A configuration is provided in which a partial image and / or an enlarged image of a main subject and a subject image are displayed on the liquid crystal display 24, and a shooting condition of a shooting area corresponding to the partial image can be manually set. Then, the shooting conditions of the plurality of unit groups 32 of the imaging element 22 corresponding to the main subject are set to the manually set shooting conditions. Thus, a user interface can be provided in which the user can appropriately set the shooting conditions for each shooting area.

[0161] (3) The control unit 23 is configured to clearly display the main subject area in the subject image. This allows the user to easily recognize the main subject even if the main subject is displayed small on the liquid crystal display 24, which improves convenience.

[0162] (4) The main subject image is displayed as the partial image 181 independently of the subject image on the liquid crystal display 24. This improves the overview of the main subject image and makes it easier for the user to select the main subject for which the shooting conditions are to be changed.

[0163] (5) The image of the main subject is displayed as an enlarged image independently of the subject image on the liquid crystal display 24. This improves the visibility of the main subject, makes it easier to change the shooting conditions, and makes it easier to confirm the effects of the shooting conditions and image processing conditions set by the user.

[0164] (6) The object recognition unit 23a is configured to recognize a specific subject object from the subject image displayed as the subject image 110 on the liquid crystal display 24. Among the subject objects detected by the object recognition unit 23a, the object ranked highest after weighting by the weight setting unit 23b is determined as the main subject and becomes the subject of manual setting of the shooting conditions. Thus, even if the object recognition unit automatically recognizes a plurality of subject objects, the number of main subjects can be limited to a predetermined number.

[0165] (7) The image processing effect can also be changed for the main subject. This makes it easy to obtain a subject image with the subject image quality desired by the user.

[0166] ---Second Embodiment---

[0167] Reference Figure 8-Figure 9 (b) of the present invention describes a second embodiment of the present invention. In the following description, the same figure marks are added to the same components as the first embodiment, and the differences are mainly described. The points not specifically described are the same as the first embodiment. The object recognition unit 23a of the first embodiment automatically recognizes a specified subject object as a main subject from a subject image, or recognizes a specified subject object as a main subject from a subject image. That is, it is an object recognition unit that detects the area or range of the subject object. When the user specifies a point or a part in the subject image as a part of the main subject, the object recognition unit 23a of the second embodiment automatically detects the area or range of the main subject based on the color information of the subject image of the specified part, or detects the area or range of the main subject based on the color information of the subject image of the specified part.

[0168] Figure 82 is a block diagram showing the configuration of the information terminal device 10 of the second embodiment. The information terminal device 10 of the second embodiment is different from the information terminal device 10 of the first embodiment in that it does not include the weight setting unit 23b. Other aspects are the same as the information terminal device 10 of the first embodiment.

[0169] Fig. 9 (a) schematically shows a state in which a user touches a person 101 in a subject image 110 displayed on a liquid crystal display 24 with a finger 161. Fig. 9 As shown in (a) of FIG. 1 , when the user selects a point or a part in the subject image 110 as a part of the main subject, the object recognition unit 23a automatically detects a region or range of the image having color information substantially the same as the color information based on the color information of the subject image 110 of the selected part. Fig. 9 When a part of the face of the person 101 in the subject image 110 is touched as shown in (a), the object recognition unit 23a detects the extension of the skin color part based on the skin color information of the face, thereby detecting the face area of ​​the person 101. When a part of the area of ​​the subject image 110 displayed on the liquid crystal display 24 is specified by the user, the object recognition unit 23a recognizes a specific area including the specified area as the main subject. For example, when a part of the area of ​​the person displayed on the subject image 110 is specified by the user, the object recognition unit 23a recognizes the person area as the specific area. In addition, when a plurality of persons are displayed on the liquid crystal display 24 as the subject image 110, when a part of a person area is specified by the user, the object recognition unit 23a may recognize a plurality of person areas as the specific area. That is, even when a single area is specified by the user, the object recognition unit 23a may recognize a plurality of areas as the specific area based on, for example, the color information or brightness information of the specified area and the subject information (person, face, etc.) of the specified area.

[0170] When the object recognition unit 23a detects the region of the main subject, the region division unit 23c divides the main subject region in the subject image 110A. Fig. 9 As shown in (b), the face, i.e., the main subject, is highlighted on the subject image 110A as indicated by the dotted line. Similarly, by the user touching other parts on the subject image 110A, the area of ​​other main subjects is detected based on their color information and highlighted on the subject image 110A.

[0171] The subsequent change of the photographing condition from the setting screen 130 of the photographing condition of the main subject is the same as that of the first embodiment, and thus detailed description thereof will be omitted.

[0172] In the above-mentioned second embodiment, in addition to the effects of the first embodiment, the following effects are achieved.

[0173] (1) When the user touches a point or a portion of the subject image that is a part of the main subject, the main subject is automatically detected. Thus, the user can arbitrarily specify the main subject, so it is easy to obtain a subject image with the subject image quality desired by the user.

[0174] ---Third Embodiment---

[0175] Reference Fig.10 (a)~ Fig.11 (b) of the present invention describes a third embodiment of the present invention. In the following description, the same reference numerals are added to the same components as the first embodiment, and the differences are mainly described. Points not specifically described are the same as the first and second embodiments. The third embodiment is different from the first and second embodiments mainly in that when the user indicates the range of the subject, the subject within the indicated range is set as the main subject.

[0176] The configuration of the information terminal device 10A according to the third embodiment Figure 8 The configuration of the information terminal device 10A of the second embodiment shown is the same.

[0177] Fig.10 (a) schematically shows a state in which a user touches a person 101 in a subject image 110 displayed on the liquid crystal display 24 with two fingers 161. In this embodiment, for example Fig.10 When the user indicates any two positions of the liquid crystal display 24 while the subject image 110 is displayed on the liquid crystal display 24, as shown in (a) of FIG. Fig.10 As shown in (b) of FIG. 1 , the control unit 23 causes the liquid crystal display 24 to display a rectangular frame line 171 with the two indicated positions as diagonal lines.

[0178] The control unit 23 Fig.10 When the zoom operation is performed as shown in (c) of FIG. 171, the frame line 171 is enlarged. Fig.10 When the frame line 171 is reduced in size as shown in (d) of FIG. 1 , the frame line 171 is reduced in size. In addition, when the relative positions of the two fingers 161 change, the aspect ratio of the frame line 171 is changed in accordance with the position touched by the finger 161. In addition, when the frame line 171 or the area inside the frame line 171 is long-pressed by the finger 161, the control unit 23 selects the frame line 171. In this state, when the finger 161 moves on the display surface of the liquid crystal display 24, as shown in FIG. Fig.10 As shown in (e), the position of the frame line 171 is moved according to the movement of the finger 161.

[0179] In addition, in judging Fig.11 When the display surface of the liquid crystal display 24 is touched by a finger 161 and the finger 161 draws on the display surface in the direction of arrow A in this state, the control unit 23 causes the liquid crystal display 24 to display a line 172 along the drawn trajectory. Fig.11 As shown by arrow B in (b), when the display surface of the liquid crystal display 24 is traced with the finger 161, the control unit 23 displays the line 172 along the trajectory of the finger 161. Moreover, when the starting point of the line 172 overlaps with the trajectory of the finger 161, the control unit 23 sets the closed line 172 as the frame line 171 (not shown). Furthermore, even if the finger 161 leaves the display surface of the liquid crystal display 24 before the starting point of the line 172 overlaps with the trajectory of the finger 161, as long as the distance between the starting point and the position where the finger 161 leaves is short, the control unit 23 automatically closes the line 172 and sets it as the frame line 171.

[0180] When the frame line 171 is designated in this manner, the object recognition unit 23a detects the subject object within the frame line 171 from the subject image 110 displayed on the liquid crystal display 24. Then, the area division unit 23c determines the subject object detected by the object recognition unit 23a as the main subject, and determines or divides the position of the main subject in the subject image 110, that is, the area of ​​the main subject, into the main subject area. The subsequent change of the shooting condition from the setting screen 130 of the shooting condition of the main subject is the same as that of the first embodiment, and therefore a detailed description thereof will be omitted.

[0181] In the third embodiment described above, in addition to the effects of the first and second embodiments, the following effects are achieved.

[0182] (1) The frame line 171 or the line 172 is drawn by touching the display surface of the liquid crystal display 24. Then, the subject object is detected in the area specified by the frame line 171 or the line 172. Thus, the user can specify the main subject as desired, so it is easy to obtain the subject image the user wants.

[0183] ---Modification---

[0184] (1) In the above description, the object recognition unit 23a recognizes a specific subject object, and the shooting conditions can be changed for the area inside the outline of the main subject. However, the present invention is not limited to this. For example, it is also possible to change the shooting conditions of the entire area within the frame line 171 of the third embodiment.

[0185] (2) The display mode may be changed according to the screen size of the liquid crystal display 24. For example, in the case of a tablet terminal device with a large screen size, the main subject is displayed larger than in a terminal device such as a smartphone, so unless the main subject is very small, for example, the enlarged image may not be displayed. In other words, the control unit 23 may determine whether the size of the displayed main subject is a predetermined size, and control the display mode according to the determination result.

[0186] (3) The display order of the setting items 131 of the shooting conditions may be changed according to the type of the subject. For example, if the main subject is a person, the possibility of changing the setting of the brightness of the person is high, so the control unit 23 may preferentially display the setting items related to the change of brightness as the setting items 131. This improves the operability of the setting change.

[0187] (4) In the above description, the liquid crystal display 24 is a touch panel type display device, which constitutes a part of the operating unit 26, but the present invention is not limited to this. For example, the operating unit 26 may be provided with an operating device such as an operating key or an operating dial or a mouse, and an arbitrary position on the display screen of the liquid crystal display 24 can be indicated by moving a cursor or a pointer on the display screen to a desired position.

[0188] (5) When displaying an enlarged image of a main subject, the aspect ratio of the area displaying the subject image may be different from that of the original subject image. In this case, the entire subject image may be displayed in a reduced size or the image may be appropriately cropped from the subject image and displayed.

[0189] (6) In the above description, "shutter speed", "ISO sensitivity", "frame rate", and "image processing effect" are listed as examples of the setting items 131 of the shooting conditions, but these are only examples and the present invention is not limited thereto. For example, at least one of the "shutter speed", "ISO sensitivity", and "frame rate" as shooting conditions may be included in the setting items 131. In addition, for example, the "image processing effect" as an image processing condition may not be included in the setting items 131. Other items other than the above may be included as the setting items 131.

[0190] (7) In the above description, the information terminal device 10 is a device such as a tablet-type information terminal device that is thin and has a large liquid crystal display on the front surface and a photographing optical system on the back side, but the present invention is not limited to this. For example, the information terminal device 10 may be a camera with an integrated lens or a lens-interchangeable camera in which a lens can be detachably mounted and replaced on the camera body.

[0191] (8) In the above description, a single electronic device, namely, the information terminal device 10, having the image sensor 22 and the control unit 23 is described, but the present invention is not limited to such an embodiment. For example, the present invention can also be applied to an electronic device that controls the image sensor 22 provided externally. The following describes in detail the form of providing the image sensor 22 externally.

[0192] Fig.12 It is a block diagram schematically showing the configuration of the imaging system according to this modification. Fig.12 The illustrated photographing system 1000 includes a photographing unit 1001 and an electronic device 1002. The photographing unit 1001 includes the imaging optical system 21 and the photographing element 22 described in the first embodiment, and also includes a first communication unit 1003. In addition, the electronic device 1002 includes the control unit 23, the liquid crystal display 24, the memory card 25, the operation unit 26, the DRAM 27, the flash memory 28, and the posture detection unit 29 described in the first embodiment, and also includes a second communication unit 1004. The first communication unit 1003 and the second communication unit 1004 can perform bidirectional data communication through, for example, a well-known wireless communication technology or an optical communication technology. In addition, it is also possible to have a structure in which the first communication unit 1003 and the second communication unit 1004 perform bidirectional data communication by connecting the photographing unit 1001 and the electronic device 1002 by a wired connection using a wired cable or the like.

[0193] In the imaging system 1000 of this modification, the control unit 23 controls the imaging element 22 through data communication via the second communication unit 1004 and the first communication unit 1003. For example, by sending and receiving predetermined control data with the imaging unit 1001, different imaging conditions are set for each unit group 32, or imaging signals are read from each unit group 32.

[0194] As described above, in the imaging system 1000, it is the control unit 23 that performs control for each unit group 32. The electronic device 1002 does not have an imaging element 22, but the same control as in the first embodiment can be performed by controlling the imaging element 22 (imaging unit 1001) provided outside the electronic device 1002. That is, the present invention can be applied to electronic devices that do not have an imaging element 22.

[0195] (9) The display form of the above-mentioned enlarged image is an example, and the present invention is not limited to this. Fig.13 As shown in (a) of FIG. 1 , a mark 201 may be displayed next to the main subject (person 101), and an enlarged image of the main subject may be displayed in the mark 201. Furthermore, the position of the mark is not limited to next to the main subject, but may be above or below the main subject, or may be diagonally opposite to the main subject. In addition, the shape of the mark may not be Fig.13The rectangle shown in (a) may be a circle or a polygon. In addition, the marked outline may not be displayed. In addition, an image showing the shooting conditions currently set for the main subject may be displayed together with or instead of the enlarged image of the main subject.

[0196] (10) As a slide bar for changing the setting of each setting item 131, for example Fig.13 As shown in (b) of FIG. 1 , a slide bar for moving a slider (knob) 201 a along the outline of the annotation 201 for enlarging and displaying the main subject may be displayed. Fig.13 In the example shown in (b), a scale line 201b is provided along the outline of the circular mark 201. Furthermore, it is also possible to configure such that the user can change the setting value of the predetermined setting item 131 by moving the slider 201a along the outline of the mark 201. Furthermore, Fig.13 In (b) of FIG. 2 , the main subject is omitted from illustration. In addition, a slide bar or a slider 201 a may be displayed together with the annotation 201 or in place of the annotation 201 .

[0197] (11) As another example of a display mode in which the main subject is clearly displayed in the subject image, the main subject may be displayed so as to appear to float from the screen. Fig.14 (a) is a diagram showing an example in which a subject image 110 is displayed on the liquid crystal display 24. For example, the image of the person 101 as the main subject in the subject image 110 may be displayed as follows: Fig.14 The image 101b of the person 101 shown in (b) is slightly (for example, about 1.2 times) enlarged and displayed so that the person 101 appears to float from the screen.

[0198] (12) In the above description, the display area of ​​the partial image 181 or the enlarged image and the display area of ​​the subject image are separated by a straight line, but the present invention is not limited to this, and they may be separated by a curved line, for example.

[0199] (13) For example Fig.15 As shown in (a) of FIG. 1 , when the image of the person 101 as the main subject in the subject image 110 is touched, an icon 202 for selecting a setting item of the shooting condition may be displayed around the image of the person 101. Fig.15As shown in (b), when the image of the person 101 as the main subject in the subject image 110 is touched, a slide bar 203 for changing the setting items of the shooting conditions may be displayed around the image of the person 101. By doing so, the setting of the shooting conditions can be changed directly for the main subject. In this case, the shooting conditions currently set for the main subject may also be displayed using the slide bar 203.

[0200] (14) For example Fig.16 As shown in FIG. 20, a diagram 204 showing where the main subject region for which the shooting conditions are to be changed is located within the subject image 110 may be displayed together with the subject image 110. Fig.16 As shown, a diagram (region display diagram) 204 showing a main subject region to be changed in the photographing conditions may be displayed in, for example, the upper right region of the liquid crystal display 24 . Fig.16 In the example shown, for example, a table 105, a cake 104 placed on a plate, and a small box 109 are present in the shooting range of the information terminal device 10. The cake 104 placed on the plate and the small box 109 are placed on the table 105. Furthermore, it is assumed that the table 105 is the main subject area for which the shooting conditions are to be changed, that is, the main subject for which the shooting conditions are to be changed.

[0201] In the area display diagram 204, the area corresponding to the background portion 108 in the subject image 110 and the area corresponding to the area other than the table 105 in the subject image 110 are, for example, blocked or displayed in a darkened manner. Thus, in the area display diagram 204, the table 105, which is the object of the change in the shooting condition, is clearly displayed compared with other areas. Therefore, by viewing the area display diagram 204, the user can easily recognize that the main subject for which the shooting condition is to be changed is the table 105. By operating the slide bar 203 displayed below the area display diagram 204, the shooting condition for the table 105 can be changed. In this case, the slide bar 203 can also be used to display the shooting condition currently set for the main subject.

[0202] (15) For example Fig.17 As shown, it is also possible to configure such that an image of the main subject (person 101) before the setting of the shooting condition is changed is displayed in one of the two display areas 194 and 195 (for example, display area 194), and an image of the main subject (person 101) after the setting of the shooting condition is changed is displayed in the other (for example, display area 195). By adopting such a configuration, it is possible to confirm the subtle changes in the image caused by the change in settings such as shutter speed and ISO sensitivity through the images before and after the setting change.

[0203] (16) For example Fig.18As shown, the liquid crystal display 24 may display the subject image 110 together with an icon for setting a main subject for each theme, and when the icon is selected, a subject matching the theme corresponding to the icon is set as the main subject. Fig.18 In the embodiment, icons 205a, 205b, and 205c for setting a main subject for each theme are displayed on the lower side of the screen of the liquid crystal display 24.

[0204] For example, when the icon 205a is touched, the control unit 23 controls each part so that the people 106 and 107 in the subject image 110 are set as the main subject. For example, when the icon 205b is touched, the control unit 23 controls each part so that the vehicle 206 in the subject image 110 is set as the main subject. For example, when the icon 205c is touched, the control unit 23 controls each part so that the background part 108 in the subject image 110 is set as the main subject. With such a configuration, it becomes easy to set the main subject for each theme and change the shooting conditions.

[0205] (17) For example Fig.19 As shown, the liquid crystal display 24 may also display the subject image 110 and icons for setting the persons in the subject image 110 as the main subjects. When the icon is selected, the person corresponding to the icon is set as the main subject. Fig.19 In the figure, icons 221 to 223 for setting the main subject according to the individual characters are displayed on the lower side of the screen of the liquid crystal display 24.

[0206] That is, the control unit 23 controls each part in such a way as to detect a face from the subject image displayed as the subject image. Then, when a face is detected, an icon for setting the person as the main subject is generated based on the image of the detected face and displayed on the liquid crystal display 24. Fig.19 In the example shown, the control unit 23 detects faces of three persons 211 to 213 from the subject image displayed as the subject image. Then, based on the face images of the persons 211 to 213 , icons 221 to 223 are generated and displayed on the liquid crystal display 24 .

[0207] For example, the control unit 23 controls each part in such a manner that the person 211 in the subject image 110 is set as the main subject when the icon 221 corresponding to the person 211 is touched. For example, the control unit 23 controls each part in such a manner that the person 212 in the subject image 110 is set as the main subject when the icon 222 corresponding to the person 212 is touched. For example, the control unit 23 controls each part in such a manner that the person 213 in the subject image 110 is set as the main subject when the icon 223 corresponding to the person 213 is touched. With such a configuration, it becomes easy to select whether to set the person in the subject image 110 as the main subject for each person.

[0208] (18) In the above description, the posture of the information terminal device 10 detected by the posture detection unit 29 is the same as the display direction of the subject image. In other words, if the posture detection unit 29 detects that the information terminal device 10 is in the horizontal position, the display direction of the subject image is described as the horizontal position, but the present invention is not limited to this. Even if the posture detection unit 29 detects that the information terminal device 10 is in the horizontal position, if the user has set the display direction of the subject image to the vertical position in advance, the control unit 23 sets the display direction of the subject image to the vertical position and performs subsequent control.

[0209] (19) In the above description, no particular mention is made about the minimum unit for dividing a partial region. However, for example, the minimum unit for dividing a partial region may be one unit group 32, such as one pixel.

[0210] (20) The above-mentioned embodiments and modifications may be combined respectively.

[0211] ---Fourth Embodiment---

[0212] Fig. 20 3010 is a block diagram showing the structure of a camera according to a fourth embodiment of the present invention. The camera 3010 is a lens-integrated camera. The camera 3010 includes a camera optical system 21, a camera element 22, a control unit 33, a liquid crystal display 24, a memory card 25, an operation unit 26, a DRAM 27, a flash memory 28, and a recording unit 34.

[0213] The imaging optical system 21 , the imaging element 22 , the liquid crystal display 24 , the memory card 25 , the operation unit 26 , the DRAM 27 , and the flash memory 28 are the same as those in the first embodiment, and thus their description is omitted.

[0214] The control unit 33 generates an image file described below based on the image signal output from the image sensor 22, and records the image file in the memory card 25, which is a removable recording medium. The operation unit 26 has various operation components such as a push button, and outputs an operation signal to the control unit 33 according to the operation of these operation components. The recording unit 34 is composed of, for example, a microphone, etc., and converts the ambient sound into a sound signal and inputs it to the control unit 33.

[0215] Next, the imaging process executed by the control unit 33 will be described in sequence.

[0216] (1) Preparatory shooting

[0217] The control unit 33 performs preliminary shooting before the actual shooting. In the following description, this preliminary shooting is referred to as preliminary shooting. In the preliminary shooting, the control unit 33 sets the shooting conditions of each unit group 32 to the prescribed shooting conditions determined by the known automatic exposure control. One subject image can be obtained by the preliminary shooting.

[0218] (2) Determination of the darkest part, the brightest part, and the main part

[0219] The control unit 33 measures the brightness of the subject, that is, the brightness distribution of the subject, for each unit group 32 based on the subject image obtained by the preliminary shooting. Fig.21 Provide explanation.

[0220] Now, shoot Fig.21 (a) schematically shows a subject. Fig.21 In the captured image 1100 shown in (a) of FIG. 1 , a person 1104 , a building 1102 , the ground 1101 , the sky 1103 , and the sun 1105 are reflected. In addition, a shadow 1106 of the building 1102 exists on the ground 1101 . Fig.21 (b) shows an example of brightness distribution of a subject obtained by photographing the subject through preliminary photographing. Fig.21 One small square in the brightness distribution diagram 67 shown in (b) of FIG. 1 represents one unit group 32 . In addition, the numerical value written in the square represents the brightness value (Bv value) detected in one unit group 32 .

[0221] The control unit 33 Fig.21 The luminance distribution diagram 67 of the subject illustrated in (b) is used to determine the darkest part (darkest part) and the brightest part (brightest part) of the subject. Fig.21 In (b), the minimum brightness value "4" is detected in the area 1109 where the shadow 1106 of the building 1102 is located. In other words, Fig.21The darkest part of the subject illustrated in (a) is the area 1109 (hereinafter referred to as the darkest part 1109) where the shadow 1106 is located. Fig.21 In (b), the maximum brightness value "8" is detected in the area 1108 where the sun 1105 is located. That is, Fig.21 The brightest part of the subject illustrated in (a) is a region 1108 where the sun 1105 is located (hereinafter referred to as the brightest part 1108 ).

[0222] Next, the control unit 33 performs a predetermined main subject detection process on the subject image obtained by the preliminary shooting. The main subject detection process is a process of detecting the face of the subject person using, for example, the so-called face recognition technology and determining the position and range of the subject person. Fig.21 In the subject shown in (a), the face 1107 of the person 1104 is first detected, and based on this, the area 1110 where the person 1104 is located is determined. Here, the area 1110 where the detected person 1104 exists is called the main part of the subject (hereinafter, marked as the main part 1110).

[0223] Furthermore, processing other than the processing of detecting the subject person based on the above-mentioned face recognition technology may be used as the main subject detection processing. For example, the main part of the subject may be determined based on the focus position of the shooting optical system 21 and the color information of the subject. In addition, the main part of the subject may be determined using known feature quantity detection and pattern matching techniques. Alternatively, all parts other than the darkest part 1109 and the brightest part 1108 may be regarded as the main part.

[0224] In addition, in the above-mentioned embodiment, the control unit 33 determines the brightest part 1108, the darkest part 1109, and the main part 1110, respectively. However, depending on the subject image, there is a case where the main part 1110 becomes the brightest part 1108, or the main part 1110 becomes the darkest part 1109. For example, in a backlit scene, the main subject becomes darker, so that the main part 1110 and the darkest part 1109 become the same area. In such a case, the control unit 33 does not need to determine the three parts of the brightest part 1108, the darkest part 1109, and the main part 1110, but only needs to determine the two parts of the main part 1110 (=brightest part 1108) and the darkest part 1109, or the two parts of the main part 1110 (=darkest part 1109) and the brightest part 1108.

[0225] (3) Detection of underexposure and overexposure

[0226] Next, the control unit 33 checks whether there is underexposure in the darkest portion 1109. In other words, the control unit 33 determines whether the brightness value (pixel value) of the subject image obtained by preliminary shooting in any unit group 32 belonging to the darkest portion 1109 is lower than a predetermined lower limit.

[0227] Similarly, the control unit 33 checks whether there is overexposure in the brightest part 1108. In other words, the control unit 33 determines whether the brightness value of the subject image obtained by the preliminary shooting in any unit group 32 within the unit group 32 located in the brightest part 1108 is higher than a predetermined upper limit value. For example, when the brightness value is expressed by an integer from 0 to 255, the lower limit value is set to 1 and the upper limit value is set to 254.

[0228] The detection result of underexposure and overexposure performed by the control unit 33 is one of the following: (A) neither underexposure nor overexposure exists, (B) underexposure exists but overexposure does not exist, (C) overexposure exists but underexposure does not exist, (D) both underexposure and overexposure exist.

[0229] (4) Setting of initial conditions

[0230] Based on the detection results of underexposure and overexposure, the control unit 33 determines: the initial value (initial condition) of the shooting condition set for the unit group 32 belonging to the darkest part 1109 in the actual shooting; the initial value (initial condition) of the shooting condition set for the unit group 32 belonging to the brightest part 1108 in the actual shooting; and the initial value (initial condition) of the shooting condition set for the unit group 32 belonging to the main part 1110 in the actual shooting. The method of determining the initial condition by the control unit 33 will be described in detail later.

[0231] (5) Acceptance of changes to shooting conditions

[0232] Next, the control unit 33 displays the subject image obtained by the preliminary shooting on the liquid crystal display 24. Then, the user is given an opportunity to change the shooting conditions for each unit group 32 from the above-mentioned initial conditions. Specifically, a predetermined change operation is accepted on an operating member (not shown).

[0233] In this embodiment, in order not to complicate the operation, only the shooting condition of the main part 1110 is accepted for change operation. In other words, the user can clearly change only the shooting condition of the main part 1110. The shooting conditions of the darkest part 1109 and the brightest part 1108 are basically automatically changed by the control unit 33 according to the change operation of the shooting condition of the main part 1110. In other words, the shooting conditions of the darkest part 1109 and the brightest part 1108 are linked with the shooting conditions of the main part 1110. The method of changing the shooting conditions by the control unit 33 will be described in detail later.

[0234] The control unit 33 displays the main portion 1110 in an emphasized manner in the subject image displayed on the liquid crystal display 24. For example, a dotted line surrounding the main portion 1110 is displayed, or the main portion 1110 is displayed in a blinking manner. In addition, when at least one of the shooting conditions of the darkest portion 1109, the brightest portion 1108, and the main portion 1110 is changed, the subject image reflecting the change in the shooting condition is displayed on the liquid crystal display 24. In other words, the change in the shooting condition is reflected on the liquid crystal display 24. For example, when the shooting condition of the main portion 1110 is changed to a shooting condition that allows the main portion 1110 to be shot brighter, the main portion 1110 is displayed brighter in the subject image displayed on the liquid crystal display 24.

[0235] (6) Actual shooting

[0236] When the user who has set the shooting condition to a desired value (condition) performs a predetermined shooting operation (e.g., a full-press operation of the release switch) on the operating member (not shown), the control unit 33 causes the image sensor 22 to shoot the subject image and generates image data of the subject (main shooting). At this time, the shooting conditions are independently set for the unit group 32 belonging to the darkest part 1109, the unit group 32 belonging to the brightest part 1108, and the unit group 32 belonging to the main part 1110. The control unit 33 records the image data generated by the main shooting in, for example, the memory card 25 and / or displays it on the liquid crystal display 24.

[0237] The above is the photographing process performed by the control unit 33. The photographing conditions described above may be exposure conditions such as shutter speed, ISO sensitivity, and aperture value, or may be photographing conditions such as a frame rate of a moving image.

[0238] Next, among the above-mentioned photographing processes, (4) setting of initial conditions and (5) acceptance of operations to change photographing conditions will be described case by case according to the results of (3) detection of underexposure and overexposure.

[0239] (A) Neither underexposure nor overexposure exists (normal image)

[0240] Fig. 22 (a) shows a bar graph of the brightness of each unit group 32 in the case where neither underexposure nor overexposure exists. Fig. 22 In (a), the horizontal axis represents brightness (pixel value), and the vertical axis represents the number (frequency) of unit groups 32 having the brightness. Fig. 22 (a) schematically illustrates a set 1121 of unit groups 32 belonging to the darkest portion 1109 , a set 1123 of unit groups 32 belonging to the brightest portion 1108 , and a set 1122 of unit groups 32 belonging to the main portion 1110 .

[0241] Furthermore, the collections 1121, 1122, and 1123 are Fig. 22 In (a), the respective brightness is not overlapped, but in the case where, for example, the brightness values ​​of the respective sets 1121, 1122, and 1123 have a range, or there is almost no brightness difference between the set 1121 of the darkest part 1109 and the set 1122 of the main part brightness, the sets 1121, 1122, and 1123 may overlap. Even in such a case, as described later, the control unit 33 can change the sets 1121, 1122, and 1123 independently in the left-right direction of the brightness axis.

[0242] When neither underexposure nor overexposure exists, that is, the brightness of the entire screen is within a specified range (within the range of the lower limit value to the upper limit value), the above three sets 1121, 1122, and 1123 all converge between the specified lower limit value 1120 and the specified upper limit value 1130. In this case, the control unit 33 sets the initial conditions of the darkest part 1109, the brightest part 1108, and the main part 1110 to the same shooting conditions as the preliminary shooting, that is, the specified shooting conditions determined by the control unit 33. For example, the exposure conditions such as the shutter speed (charge storage time), the aperture value, and the ISO sensitivity (magnification) are set to the specified exposure conditions determined by the automatic exposure control.

[0243] When the user performs a change operation to change the shooting condition of the main part 1110 in such a way that the main part 1110 becomes darker (the brightness of the main part 1110 becomes lower) (for example, an operation to increase the shutter speed of the main part 1110), the control unit 33 changes the shooting condition of the darkest part 1109 in such a way that the brightness difference D1 between the set 1121 of the darkest part 1109 and the set 1122 of the main part 1110 is maintained. That is, the shooting condition of the darkest part 1109 is changed to a shooting condition (for example, a faster shutter speed) such that the darkest part 1109 is shot darker. In other words, Fig. 22 In (a), when the user wants to move the set 1122 of the main part 1110 to the left of the brightness axis, the control unit 33 moves the set 1121 of the darkest part 1109 to the left of the brightness axis in conjunction with the user. That is, when the control unit 33 receives an operation from the user to change the shooting condition of the main part 1110, it changes the shooting condition of the main part 1110 and the shooting condition of the darkest part 1109. The control unit 33 changes the shooting condition of the darkest part 1109 by changing the shooting condition of the main part 1110.

[0244] However, the control unit 33 changes the shooting condition of the darkest portion 1109 in such a way that the brightness of the darkest portion 1109 does not fall below the prescribed lower limit 1120. That is, the control unit 33 stops (prohibits) the change of the shooting condition of the darkest portion 1109 when the brightness of the darkest portion 1109 reaches the prescribed lower limit 1120. In other words, Fig. 22 In (a), when the user attempts to move the set 1122 of the main part 1110 to the left of the brightness axis, which causes the set 1121 of the darkest part 1109 to exceed the prescribed lower limit 1120, the control unit 33 moves only the set 1122 of the main part 1110 to the left of the brightness axis, and stops the set 1121 of the darkest part 1109 at the lower limit 1120 without moving. In this case, the brightness difference D1 between the set 1121 of the darkest part 1109 and the set 1122 of the main part 1110 changes.

[0245] Furthermore, the control unit 33 changes the shooting condition of the main unit 1110 so that the brightness of the main unit 1110 does not fall below the prescribed lower limit 1120. That is, the control unit 33 stops changing the shooting condition of the main unit 1110 when the brightness of the main unit 1110 reaches the prescribed lower limit 1120. In other words, Fig. 22 In (a), when the user attempts to move the set 1122 of the main part 1110 to the left of the brightness axis, which will cause the set 1122 of the main part 1110 to exceed the specified lower limit 1120, the control unit 33 stops the set 1122 of the main part 1110 from moving.

[0246] Furthermore, in the change operation to darken the main portion 1110 (the brightness of the main portion 1110 becomes lower), the shooting condition of the brightest portion 1108 does not change. Fig. 22 In (a), when the user attempts to move the set 1122 of the main part 1110 to the left of the brightness axis, the set 1123 of the brightest part 1108 stops moving.

[0247] Furthermore, when the user performs an operation to change the shooting condition of the main portion 1110 so as to brighten the main portion 1110 (increase the brightness of the main portion 1110) (e.g., an operation to slow down the shutter speed of the main portion 1110), the control unit 33 changes the shooting condition of the brightest portion 1108 so as to maintain the brightness difference between the set 1123 of the brightest portion 1108 and the set 1122 of the main portion 1110. In other words, the shooting condition of the brightest portion 1108 is changed to a shooting condition (e.g., a slower shutter speed) such that the brightest portion 1108 is shot brighter. In other words, Fig. 22In (a), when the user wants to move the set 1122 of the main part 1110 to the right of the brightness axis, the control unit 33 moves the set 1123 of the brightest part 1108 to the right of the brightness axis in conjunction with the user. That is, when the control unit 33 receives an operation from the user to change the shooting condition of the main part 1110, the shooting condition of the main part 1110 and the shooting condition of the brightest part 1108 are changed. The control unit 33 changes the shooting condition of the brightest part 1108 by changing the shooting condition of the main part 1110.

[0248] However, the control unit 33 changes the photographing condition of the brightest portion 1108 so that the brightness of the brightest portion 1108 does not exceed the prescribed upper limit 1130. That is, the control unit 33 stops changing the photographing condition of the brightest portion 1108 when the brightness of the brightest portion 1108 reaches the prescribed upper limit 1130. In other words, Fig. 22 In (a), when the user moves the set 1122 of the main part 1110 to the right of the brightness axis, which causes the set 1123 of the brightest part 1108 to exceed the prescribed upper limit 1130, the control unit 33 moves only the set 1122 of the main part 1110 to the right of the brightness axis, and stops the set 1123 of the brightest part 1108 from moving. In this case, the brightness difference D2 between the set 1123 of the brightest part 1108 and the set 1122 of the main part 1110 changes.

[0249] Furthermore, the control unit 33 changes the shooting condition of the main unit 1110 so that the brightness of the main unit 1110 does not exceed the prescribed upper limit 1130. That is, the control unit 33 stops changing the shooting condition of the main unit 1110 when the brightness of the main unit 1110 reaches the prescribed upper limit 1130. In other words, Fig. 22 In (a), when the user attempts to move the set 1122 of the main part 1110 to the right of the brightness axis, which will cause the set 1122 of the main part 1110 to exceed the specified upper limit value 1130, the control unit 33 stops the set 1122 of the main part 1110 from moving.

[0250] In the change operation to brighten the main portion 1110 (the brightness of the main portion 1110 becomes higher), the shooting condition of the darkest portion 1109 does not change. Fig. 22 In (a), when the user attempts to move the set 1122 of the main part 1110 to the right of the brightness axis, the set 1121 of the darkest part 1109 stops moving.

[0251] Furthermore, the control unit 33 may change the shooting conditions of the darkest portion 1109 according to the change of the shooting conditions of the main portion 1110. For example, when it is considered that a certain amount of high-sensitivity noise is generated in the darkest portion 1109, the shooting conditions of the darkest portion 1109 may be changed so as to eliminate the noise.

[0252] (B) There is underexposure but no overexposure (underexposed image)

[0253] Fig. 22 (b) is a bar graph showing the brightness of each unit group 32 when there is underexposure but no overexposure. When there is underexposure but no overexposure, that is, when the brightness of the brightest part 1108 and the main part 1110 converge within a prescribed range (within the range of a lower limit value to an upper limit value), but the brightness of the darkest part 1109 deviates from the range, the set 1123 of the brightest part 1108 and the set 1122a of the main part 1110 converge between a prescribed lower limit value 1120 and a prescribed upper limit value 1130, and at least a part of the set 1121a of the darkest part 1109 exceeds the prescribed lower limit value 1120 and is located to the left thereof. In this case, the control unit 33 sets the initial condition of the brightest part 1108 to the same shooting condition as the preliminary shooting, that is, the prescribed shooting condition determined by the automatic exposure control.

[0254] The control unit 33 sets the initial condition of the darkest portion 1109 to a shooting condition in which there is no underexposure. In other words, the shooting condition is set to a shooting condition in which the darkest portion 1109 is shot brightly, compared with the prescribed shooting condition determined by the automatic exposure control. Fig. 22 The set 1121a of the darkest part 1109 shown in (b) moves to a position adjacent to the predetermined lower limit value 1120 and to the right of the predetermined lower limit value 1120 ( Fig. 22 The set 1121 represented by the solid line in (b).

[0255] Furthermore, the control unit 33 sets the initial condition of the main part 1110 to a shooting condition corresponding to the initial condition of the darkest part 1109 and the initial condition of the brightest part 1108. Specifically, when the absolute value of the difference between the mode of the brightness of the darkest part 1109 and the mode of the brightness of the main part 1110 in the prescribed shooting condition determined by the automatic exposure control is set to D1, the absolute value of the difference between the mode of the brightness of the main part 1110 and the mode of the brightness of the brightest part 1108 in the shooting condition is set to D2, and the absolute value of the difference between the mode of the brightness of the darkest part 1109 in the prescribed shooting condition determined by the automatic exposure control and the mode of the brightness of the darkest part 1109 in the above initial condition is set to da, the shooting condition is set as the initial condition of the main part 1110 such that the brightness of the main part 1110 becomes larger (brighter) by D3 defined by the following equation (1).

[0256] D3=(da×D2) / (D1+D2)···(1)

[0257] That is, the initial conditions of the darkest part 1109 and the main part 1110 are set in such a manner that underexposure is eliminated while maintaining the relative relationship between the brightness of the darkest part 1109, the brightness of the main part 1110, and the brightness of the brightest part 1108 in the prescribed shooting conditions determined by the automatic exposure control. Fig.21 The set 1122a of the main part 1110 shown is moved to the right of the brightness axis by D3 ( Fig. 22 The set 1122 represented by the solid line in (b).

[0258] When the user performs a change operation to change the shooting condition of the main part 1110 in such a way that the main part 1110 becomes darker (the brightness of the main part 1110 becomes lower), the control unit 33 changes the shooting condition of the brightest part 1108 in such a way that the brightness difference D2' between the set 1123 of the brightest part 1108 and the set 1122 of the main part 1110 is maintained. In other words, the shooting condition of the brightest part 1108 is changed to a shooting condition that makes the brightest part 1108 appear darker. In other words, Fig. 22 In (b), when the user wants to move the set 1122 of the main part 1110 to the left of the brightness axis, the control unit 33 moves the set 1123 of the brightest part 1108 to the left of the brightness axis in conjunction with the user. That is, when the control unit 33 receives an operation from the user to change the shooting condition of the main part 1110, it changes the shooting condition of the main part 1110 and the shooting condition of the brightest part 1108. The control unit 33 changes the shooting condition of the brightest part 1108 by changing the shooting condition of the main part 1110.

[0259] However, the control unit 33 changes the shooting condition of the main unit 1110 in such a way that the brightness of the main unit 1110 does not fall below the prescribed lower limit 1120. That is, the control unit 33 stops (prohibits) the change of the shooting condition of the main unit 1110 when the brightness of the main unit 1110 reaches the prescribed lower limit 1120. In other words, Fig. 22 In (b), when the user attempts to move the set 1122 of the main part 1110 to the left of the brightness axis and causes the set 1122 of the main part 1110 to exceed the specified lower limit 1120, the control unit 33 stops the set 1122 of the main part 1110 from moving.

[0260] Furthermore, when the user performs a change operation to change the shooting condition of the main part 1110 in such a way as to brighten the main part 1110 (the brightness of the main part 1110 becomes higher), the control unit 33 changes the shooting condition of the brightest part 1108 in such a way as to maintain the brightness difference between the set 1123 of the brightest part 1108 and the set 1122 of the main part 1110. In other words, the shooting condition of the brightest part 1108 is changed to a shooting condition that makes the brightest part 1108 appear brighter. In other words, Fig. 22 In (b), when the user wants to move the set 1122 of the main part 1110 to the right of the brightness axis, the control unit 33 moves the set 1123 of the brightest part 1108 to the right of the brightness axis in conjunction with the user. That is, when the control unit 33 receives an operation from the user to change the shooting condition of the main part 1110, it changes the shooting condition of the main part 1110 and the shooting condition of the brightest part 1108. The control unit 33 changes the shooting condition of the brightest part 1108 by changing the shooting condition of the main part 1110.

[0261] However, the control unit 33 changes the photographing condition of the brightest portion 1108 so that the brightness of the brightest portion 1108 does not exceed the prescribed upper limit 1130. That is, the control unit 33 stops changing the photographing condition of the brightest portion 1108 when the brightness of the brightest portion 1108 reaches the prescribed upper limit 1130. In other words, Fig. 22 In (b), when the user moves the set 1122 of the main part 1110 to the right of the brightness axis, which causes the set 1123 of the brightest part 1108 to exceed the prescribed upper limit 1130, the control unit 33 moves only the set 1122 of the main part 1110 to the right of the brightness axis, and stops the set 1123 of the brightest part 1108 from moving. In this case, the brightness difference D2' between the set 1123 of the brightest part 1108 and the set 1122 of the main part 1110 changes.

[0262] Furthermore, the control unit 33 changes the shooting condition of the main unit 1110 so that the brightness of the main unit 1110 does not exceed the prescribed upper limit 1130. That is, the control unit 33 stops changing the shooting condition of the main unit 1110 when the brightness of the main unit 1110 reaches the prescribed upper limit 1130. In other words, Fig. 22 In (b), when the user attempts to move the set 1122 of the main part 1110 to the right of the brightness axis, which will cause the set 1122 of the main part 1110 to exceed the specified upper limit value 1130, the control unit 33 stops the set 1122 of the main part 1110 from moving.

[0263] Furthermore, in the above changing operation, the shooting condition of the darkest portion 1109 does not change. In other words, Fig. 22In (a), even if the user attempts to move the set 1122 of the main part 1110 to either the left or right direction of the brightness axis, the set 1121 of the darkest part 1109 does not move.

[0264] (C) There is overexposure but no underexposure (overexposed image)

[0265] Fig. 22 (c) is a bar graph showing the brightness of each unit group 32 when there is overexposure but no underexposure. When there is overexposure but no underexposure, that is, the brightness of the darkest part 1109 and the main part 1110 converge within a prescribed range (within the range of the lower limit value to the upper limit value), but the brightness of the brightest part 1108 deviates from the range, the set 1121 of the darkest part 1109 and the set 1122a of the main part 1110 converge between the prescribed lower limit value 1120 and the prescribed upper limit value 1130, and at least a part of the set 1123a of the brightest part 1108 exceeds the prescribed upper limit value 1130 and is located on the right. In this case, the control unit 33 sets the initial condition of the darkest part 1109 to the same shooting condition as the preliminary shooting, that is, the prescribed shooting condition determined by the automatic exposure control.

[0266] The control unit 33 sets the initial condition of the brightest portion 1108 to a shooting condition in which there is no overexposure. In other words, the shooting condition is set to shoot the brightest portion 1108 darker than the prescribed shooting condition determined by the automatic exposure control. Fig. 22 The set 1123a of the brightest part 1108 shown in (c) moves to a position adjacent to the prescribed upper limit value 1130 and to the left of the prescribed upper limit value 1130 ( Fig. 22 The set 1123 represented by the solid line in (c).

[0267] Furthermore, the control unit 33 sets the initial condition of the main part 1110 to a shooting condition corresponding to the initial condition of the darkest part 1109 and the initial condition of the brightest part 1108. Specifically, when the absolute value of the difference between the mode of the brightness of the darkest part 1109 and the mode of the brightness of the main part 1110 in the prescribed shooting condition determined by the automatic exposure control is set to D1, the absolute value of the difference between the mode of the brightness of the main part 1110 and the mode of the brightness of the brightest part 1108 in the shooting condition is set to D2, and the absolute value of the difference between the mode of the brightness of the brightest part 1108 in the prescribed shooting condition determined by the automatic exposure control and the mode of the brightness of the brightest part 1108 in the above initial condition is set to db, the shooting condition in which the brightness of the main part 1110 is reduced (darkened) by D4 defined by the following formula (2) is set as the initial condition of the main part 1110.

[0268] D4=(db×D1) / (D1+D2)···(2)

[0269] That is, the initial conditions of the brightest part 1108 and the main part 1110 are set in such a manner that overexposure is eliminated while maintaining the relative relationship between the brightness of the darkest part 1109, the brightness of the main part 1110, and the brightness of the brightest part 1108 in the prescribed shooting conditions determined by the automatic exposure control. Fig. 22 The set 1122a of the main part 1110 shown in (c) is moved to the left of the brightness axis by D4 ( Fig. 22 The set 1122 represented by the solid line in (c).

[0270] When the user performs a change operation to change the shooting condition of the main part 1110 in such a way that the main part 1110 becomes darker (the brightness of the main part 1110 becomes lower), the control unit 33 changes the shooting condition of the darkest part 1109 in such a way that the brightness difference D1' between the set 1121 of the darkest part 1109 and the set 1122 of the main part 1110 is maintained. In other words, the shooting condition of the darkest part 1109 is changed to a shooting condition that makes the darkest part 1109 appear darker. In other words, Fig. 22 In (c), when the user wants to move the set 1122 of the main part 1110 to the left of the brightness axis, the control unit 33 moves the set 1121 of the darkest part 1109 to the left of the brightness axis in conjunction with the user. That is, when the control unit 33 receives an operation from the user to change the shooting condition of the main part 1110, it changes the shooting condition of the main part 1110 and the shooting condition of the darkest part 1109. The control unit 33 changes the shooting condition of the darkest part 1109 by changing the shooting condition of the main part 1110.

[0271] However, the control unit 33 changes the shooting condition of the darkest portion 1109 in such a way that the brightness of the darkest portion 1109 does not fall below the prescribed lower limit 1120. That is, the control unit 33 stops changing the shooting condition of the darkest portion 1109 when the brightness of the darkest portion 1109 reaches the prescribed lower limit 1120. In other words, Fig. 22 In (c), when the user attempts to move the set 1122 of the main part 1110 to the left of the brightness axis, which causes the set 1121 of the darkest part 1109 to exceed the predetermined lower limit 1120, the control unit 33 moves only the set 1122 of the main part 1110 to the left of the brightness axis, and stops the set 1121 of the darkest part 1109 from moving. In this case, the brightness difference D1' between the set 1121 of the darkest part 1109 and the set 1122 of the main part 1110 changes.

[0272] Furthermore, the control unit 33 changes the shooting condition of the main unit 1110 so that the brightness of the main unit 1110 does not fall below the prescribed lower limit 1120. That is, the control unit 33 stops changing the shooting condition of the main unit 1110 when the brightness of the main unit 1110 reaches the prescribed lower limit 1120. In other words, Fig. 22 In (c), when the user attempts to move the set 1122 of the main part 1110 to the left of the brightness axis, which will cause the set 1122 of the main part 1110 to exceed the specified lower limit 1120, the control unit 33 stops the set 1122 of the main part 1110 from moving.

[0273] Furthermore, when the user performs a change operation to change the shooting condition of the main part 1110 in such a way that the main part 1110 becomes brighter (the brightness of the main part 1110 becomes higher), the control unit 33 changes the shooting condition of the darkest part 1109 in such a way that the brightness difference between the set 1121 of the darkest part 1109 and the set 1122 of the main part 1110 is maintained. In other words, the shooting condition of the darkest part 1109 is changed to a shooting condition that makes the darkest part 1109 appear brighter. In other words, Fig. 22 In (c), when the user wants to move the set 1122 of the main part 1110 to the right of the brightness axis, the control unit 33 moves the set 1121 of the darkest part 1109 to the right of the brightness axis in conjunction with the user. That is, when the control unit 33 receives an operation from the user to change the shooting condition of the main part 1110, it changes the shooting condition of the main part 1110 and the shooting condition of the darkest part 1109. The control unit 33 changes the shooting condition of the darkest part 1109 by changing the shooting condition of the main part 1110.

[0274] However, the control unit 33 changes the shooting condition of the main unit 1110 so that the brightness of the main unit 1110 does not exceed the prescribed upper limit 1130. That is, the control unit 33 stops changing the shooting condition of the main unit 1110 when the brightness of the main unit 1110 reaches the prescribed upper limit 1130. In other words, Fig. 22 In (c), when the user attempts to move the set 1122 of the main part 1110 to the right of the brightness axis, which will cause the set 1122 of the main part 1110 to exceed the specified upper limit value 1130, the control unit 33 stops the set 1122 of the main part 1110 from moving.

[0275] Furthermore, in the above changing operation, the shooting condition of the brightest portion 1108 does not change. In other words, Fig. 22 In (c), even if the user attempts to move the set 1122 of the main part 1110 to either the left or right side of the brightness axis, the set 1123 of the brightest part 1108 does not move.

[0276] (D) Both underexposure and overexposure exist

[0277] Fig. 22 (d) is a bar graph showing the brightness of each unit group 32 when both underexposure and overexposure exist. When both underexposure and overexposure exist, that is, the brightness of the main part 1110 is within a specified range (within the range of the lower limit value to the upper limit value), but the brightness of the darkest part 1109 and the brightness of the brightest part 1108 are out of the range, the set 1122a of the main part 1110 converges between the specified lower limit value 1120 and the specified upper limit value 1130, at least a part of the set 1121a of the darkest part 1109 exceeds the specified lower limit value 1120 and is located on the left side, and at least a part of the set 1123a of the brightest part 1108 exceeds the specified upper limit value 1130 and is located on the right side.

[0278] In this case, the control unit 33 sets the initial conditions of the darkest portion 1109, the brightest portion 1108, and the main portion 1110 to shooting conditions in which neither underexposure nor overexposure exists, while maintaining the relative positional relationship (brightness relationship). That is, the control unit 33 sets the initial condition of the darkest portion 1109 to a shooting condition in which the darkest portion 1109 is photographed brightly, compared to the prescribed shooting condition determined by the automatic exposure control, and sets the initial condition of the brightest portion 1108 to a shooting condition in which the brightest portion 1108 is photographed darker, compared to the prescribed shooting condition determined by the automatic exposure control. Thus, Fig. 22 In the initial condition indicated by the dotted lines in (d), the set 1121a of the darkest part 1109 and the set 1123a of the brightest part 1108 are moved to the range of the lower limit 1120 to the upper limit 1130 (respectively). Fig. 22 The initial conditions of the main part 1110 are set to the shooting conditions corresponding to the initial conditions of the darkest part 1109 and the initial conditions of the brightest part 1108.

[0279] Specifically, when the absolute value of the difference between the mode of the brightness of the darkest part 1109 in the specified shooting conditions determined by automatic exposure control and the mode of the brightness of the main part 1110 is set to D1, the absolute value of the difference between the mode of the brightness of the main part 1110 in the shooting conditions and the mode of the brightness of the brightest part 1108 is set to D2, the absolute value of the difference between the mode of the brightness of the darkest part 1109 in the specified shooting conditions determined by automatic exposure control and the mode of the brightness of the darkest part 1109 in the above-mentioned initial conditions is set to da, and the absolute value of the difference between the mode of the brightness of the brightest part 1108 in the specified shooting conditions determined by automatic exposure control and the mode of the brightness of the brightest part 1108 in the above-mentioned initial conditions is set to db, the shooting condition such as D5 defined by the following formula (3) in which the brightness change of the main part 1110 is set as the initial condition of the main part 1110.

[0280] D5=(db×D1-da×D2) / (D1+D2)···(3)

[0281] That is, the initial conditions of the darkest part 1109, the main part 1110, and the brightest part 1108 are set in a manner that eliminates underexposure and overexposure while maintaining the relative relationship between the brightness of the darkest part 1109, the brightness of the main part 1110, and the brightness of the brightest part 1108 in the prescribed shooting conditions determined by automatic exposure control.

[0282] When the user performs a change operation to change the shooting condition of the main portion 1110, the control unit 33 does not change the shooting conditions of the darkest portion 1109 and the brightest portion 1108, but only changes the shooting conditions of the main portion 1110. In other words, Fig. 22 In (d), when the user wants to move the set 1122 of the main part 1110 in the left-right direction of the brightness axis, the control unit 33 moves only the set 1122 of the main part 1110 in the direction of the brightness axis desired by the user while fixing the set 1121 of the darkest part 1109 and the set 1123 of the brightest part 1108.

[0283] However, the control unit 33 changes the shooting condition of the main unit 1110 in such a way that the brightness of the main unit 1110 does not fall below the prescribed lower limit 1120 and does not exceed the prescribed upper limit 1130. In other words, the control unit 33 prohibits changes in the shooting condition of the main unit 1110 such that the brightness of the main unit 1110 falls below the prescribed lower limit 1120, and also prohibits changes in the shooting condition of the main unit 1110 such that the brightness of the main unit 1110 exceeds the prescribed upper limit 1130. In other words, Fig. 22In (d), when the user attempts to move the set 1122 of the main part 1110 to the left of the brightness axis, which causes the set 1122 of the main part 1110 to exceed the specified lower limit 1120, the control unit 33 stops the set 1122 of the main part 1110 from moving. In addition, when the user attempts to move the set 1122 of the main part 1110 to the right of the brightness axis, which causes the set 1122 of the main part 1110 to exceed the specified upper limit 1130, the control unit 33 stops the set 1122 of the main part 1110 from moving.

[0284] In the above description, the initial values ​​of the shooting conditions are set to values ​​automatically determined by the control unit 33 according to known automatic exposure control, but they may be set to manual values ​​designated by the user.

[0285] According to the imaging device of the fourth embodiment described above, the following effects can be obtained.

[0286] (1) The imaging element 22 (imaging unit) is configured to be able to set different imaging conditions for each of the plurality of unit groups 32 (imaging areas). The control unit 33 causes the unit group 32 for which the first imaging condition is set to capture the main portion 1110 of the subject, and causes the second other unit group 32 for which the second imaging condition is set to capture the darkest portion 1109 and the brightest portion 1108 of the subject. The control unit 33 changes the first imaging condition in accordance with the user's operation, and changes the second imaging condition based on the change. Thus, it is possible to provide an imaging device 3010 that is easy to use and can easily set the desired imaging conditions for each imaging area.

[0287] (2) Before the first shooting condition is changed based on the user operation, the control unit 33 sets the initial value of the first shooting condition and the initial value of the second shooting condition so that the shooting signal of the unit group 32 for shooting the main part 1110 and the shooting signal of the unit group 32 for shooting the darkest part 1109 and the brightest part 1108 are within the range of the lower limit 1120 to the upper limit 1130. As a result, the dynamic range of the shot image is improved. In other words, a shot image with a large dynamic range can be obtained.

[0288] (3) The control unit 33 prohibits the change of the second shooting condition in which the shooting signal of the unit group 32 for shooting the darkest part 1109 and the brightest part 1108 deviates from the range of the lower limit value 1120 to the upper limit value 1130. Thus, underexposure and overexposure will not occur due to the change of the shooting condition, so the user can freely set the shooting condition without worrying about underexposure and overexposure.

[0289] (4) The control unit 33 prohibits the second imaging condition from being changed such that the magnitude relationship between the imaging signal of the main portion 1110, the imaging signal of the darkest portion 1109, and the imaging signal of the brightest portion 1108 is reversed. This prevents the generation of an image with unnatural brightness.

[0290] (5) The control unit 33 determines the amount of change in the imaging condition of the darkest portion 1109 and the amount of change in the imaging condition of the brightest portion 1108 based on the amount of change in the imaging condition of the main portion 1110. This makes it possible to easily set desired imaging conditions for each imaging region.

[0291] The following modifications are also within the scope of the present invention, and one or more of the modifications may be combined with the above-described embodiment.

[0292] (Variant 1)

[0293] Regarding the operation of changing the shooting conditions of the main part 1110, it is explained how the shooting conditions of the darkest part 1109 and the brightest part 1108 are changed and what kind of changes are prohibited, but the above description is an example, and the shooting conditions can also be changed in a mode different from these or the change of the shooting conditions can be prohibited.

[0294] For example, when the user performs a change operation to change the shooting condition of the main portion 1110 so that the main portion 1110 is shot brighter, the control unit 33 may change the shooting condition of the darkest portion 1109 and the shooting condition of the brightest portion 1108 so that both the darkest portion 1109 and the brightest portion 1108 are shot brighter (or darker). That is, when the control unit 33 receives the operation to change the shooting condition of the main portion 1110 from the user, it changes the shooting condition of the main portion 1110, the shooting condition of the brightest portion 1108, and the shooting condition of the darkest portion 1109. The control unit 33 changes the shooting condition of the brightest portion 1108 and the shooting condition of the darkest portion 1109 by changing the shooting condition of the main portion 1110. At this time, the brightness level (for example, the number of shutter speed steps) changed by the user may be different from the brightness level (for example, the number of shutter speed steps) changed in the darkest portion 1109 and the brightest portion 1108.

[0295] However, regardless of the mode, the shooting conditions that cause underexposure in the darkest portion 1109 and overexposure in the brightest portion 1108 should be prohibited. In addition, it is desirable to avoid a mode in which the shooting conditions of the main portion 1110 are changed in conjunction with the change in the shooting conditions of the main portion 1110 (for example, the shutter speed of the main portion 1110 is increased by one level) and the shooting conditions of both the darkest portion 1109 and the brightest portion 1108 are changed in the same manner as the change (for example, the shutter speed of both the darkest portion 1109 and the brightest portion 1108 is increased by one level).

[0296] (Variant 2)

[0297] In the above description, when the shooting conditions are changed, the change is reflected in the subject image displayed on the liquid crystal display 24. However, a difference may be set in the timing of reflecting the change between the main part 1110 and the parts other than the main part 1110 (the darkest part 1109 and the brightest part 1108). For example, when the shooting conditions are changed, the change may be reflected only in the main part 1110 at first, and then the change may be reflected in the darkest part 1109 and the brightest part 1108 after a few seconds.

[0298] (Variant 3)

[0299] In the description of the change of the shooting conditions, the mode of the brightness of the darkest part 1109, the mode of the brightness of the brightest part 1108, and the mode of the brightness of the main part 1110 are described, but these values ​​may not be the mode. For example, they may be the median value of the brightness or the average value.

[0300] (Variant 4)

[0301] In the case of (C) above where there is overexposure but no underexposure, the amount of change in the shooting condition may be different depending on the type of the subject located in the brightest portion 1108. That is, when the control unit 33 receives an operation to change the shooting condition of the main portion 1110 from the user, it changes the shooting condition of the main portion 1110 and the shooting condition of the brightest portion 1108. In this case, the shooting condition of the brightest portion 1108 may be changed by an amount different from the amount of change in the shooting condition of the main portion 1110. For example, the control unit 33 identifies the subject located in the brightest portion 1108 based on information such as the shooting distance. Here, when it is determined that the sky exists in the brightest portion 1108, the control unit 33 changes the shooting condition of the sky portion (brightest portion 1108) by the same amount so that the brightness of the sky, that is, the brightness of the brightest portion 1108 is smaller (darker) in accordance with the fact that the operation of changing the shooting condition of the main portion 1110 is performed so that the main portion 1110 is shot brighter.

[0302] Furthermore, in the case where it is determined that there is a building in the brightest part 1108, similarly, in accordance with the case where a change operation is performed to change the shooting condition of the main part 1110 so as to shoot the main part 1110 brighter, the control unit 33 changes the shooting condition of the building part (brightest part 1108) so as to shoot the brightness of the building, that is, the brightness of the brightest part 1108, smaller (darker). At this time, the amount of change is smaller than the amount of change of the main part 1110. For example, in the case where an operation is performed to slow down the shutter speed of the main part 1110 by one level, the shutter speed of the brightest part 1108 is made faster by one-third level. However, it is desirable that the shooting condition of the brightest part 1108 is changed in a way that does not cause image blur. That is, when the possibility of causing image blur is higher than a certain level, the change of the shooting condition of the brightest part 1108 is prohibited.

[0303] By adopting the above method, a clear photographic result can be obtained in which, for example, the blue sky is more emphasized.

[0304] (Variant 5)

[0305] In the case of (B) above where there is underexposure but no overexposure, the amount of change in the shooting condition may be different depending on the type of the subject located in the darkest portion 1109. That is, when the control unit 33 receives an operation to change the shooting condition of the main portion 1110 from the user, it changes the shooting condition of the main portion 1110 and the shooting condition of the darkest portion 1109. In this case, the shooting condition of the darkest portion 1109 may be changed by an amount different from the amount of change in the shooting condition of the main portion 1110. For example, the control unit 33 determines the subject located in the darkest portion 1109 based on information such as the shooting distance. Here, when it is determined that there is a building in the darkest portion 1109, the control unit 33 changes the shooting condition of the building portion (the darkest portion 1109) by the same amount so that the brightness of the building, that is, the brightness of the darkest portion 1109 is larger (brighter) according to the fact that a change operation is performed to change the shooting condition of the main portion 1110 so that the main portion 1110 is shot brighter. However, it is desirable to change the imaging conditions of the darkest portion 1109 so as not to cause image blurring. That is, when the possibility of image blurring being caused is higher than a certain level, changing the imaging conditions of the darkest portion 1109 is prohibited.

[0306] Furthermore, in the case where it is determined that there is sky (night sky) in the darkest part 1109, similarly, in accordance with the case where a change operation is performed to change the shooting condition of the main part 1110 in such a way that the main part 1110 is shot brighter, the control unit 33 changes the shooting condition of the sky part (darkest part 1109) in such a way that the brightness of the sky, that is, the brightness of the darkest part 1109 is shot larger (brighter). At this time, the amount of change is smaller than the amount of change of the main part 1110. For example, in the case where an operation is performed to slow down the shutter speed of the main part 1110 by 1 level, the shutter speed of the darkest part 1109 is slowed down by 1 / 3 level. However, it is desirable that the shooting condition of the darkest part 1109 is changed in such a way that high-sensitivity noise and image blur are not generated. That is, in the case where the possibility of generating high-sensitivity noise and / or image blur is higher than a certain level, the change of the shooting condition of the darkest part 1109 is prohibited.

[0307] By adopting the above aspect, it is possible to prevent a captured image from being unnatural, such as an overly bright night sky.

[0308] (Variant 6)

[0309] In the above-mentioned embodiment, the main part of the subject and the darkest part and the brightest part of the background part of the subject are determined based on the brightness distribution of the subject, and the shooting conditions of the darkest part and the brightest part of the background part are changed according to the change of the shooting conditions of the main part. The main part of the subject and the background part of the subject may also be determined based on information other than the brightness distribution of the subject. For example, the main part and the background part of the subject may also be determined based on the speed distribution (movement distribution) of the subject.

[0310] For example, the fast-moving part of the subject can be made the main part, and the slow-moving part can be made the background part (such as when photographing a vehicle). In this case, when the control unit 33 receives an operation to change the shooting conditions of the main part from the user, it changes the shooting conditions of the main part and the shooting conditions of the background part. In this case, the shooting conditions of the background part can also be changed by an amount different from the amount of change in the shooting conditions of the main part. For example, the control unit 33 changes the shooting conditions in such a way that the shutter speed of the main part becomes faster, so that the shutter speed of the background part is also accelerated accordingly, but the amount of change is smaller than that of the main part. In addition, by changing the shooting conditions in such a way that the shutter speed of the main part becomes slower, the shutter speed of the background part is also slowed down accordingly, but the amount of change is larger than that of the main part.

[0311] As another example, the slow portion of the subject can be made the main portion, and the fast portion can be made the background portion. In this case, when the control unit 33 receives an operation to change the shooting conditions of the main portion from the user, it changes the shooting conditions of the main portion and the shooting conditions of the background portion. In this case, the shooting conditions of the background portion can also be changed by an amount different from the amount of change in the shooting conditions of the main portion. For example, when the control unit 33 increases the shutter speed of the main portion, the shutter speed of the background portion is also increased accordingly, but the amount of change is smaller than that of the main portion. In addition, when the shutter speed of the main portion is slowed down, the shutter speed of the background portion is also slowed down accordingly, but the amount of change is larger than that of the main portion.

[0312] In any case, it is desirable to set the lower limit of the shutter speed of the background portion so as not to cause image blur. In addition, it is desirable to prohibit a change such that the shutter speed of the main portion and the shutter speed of the background portion are reversed in magnitude.

[0313] (Variant 7)

[0314] In the above embodiment, the change of the shooting condition is performed by receiving a predetermined change operation on the operating component, but the change operation of the shooting condition is not limited to this. For example, the operation screen used in changing the shooting condition may be displayed on the liquid crystal display 24 having a touch sensor that receives an input operation from the user. Specifically, Fig.23 As shown, an operation image 1111 for receiving an operation to change the shooting condition is displayed on the liquid crystal display 24. The operation image 1111 includes an adjustment bar 1112 and a slider 1113. The setting of the shooting condition is determined by the position of the slider 1113 relative to the adjustment bar 1112. The operation to change the shooting condition is performed by the slider operation performed by the user. For example, in the case where the brightness of the main part is desired to be changed as the operation to change the shooting condition, the position of the slider 1113 relative to the adjustment bar 1112 is moved. In the case where the user moves the slider 1113 to the left end side of the adjustment bar, the shooting condition of the main part is set so that it becomes darker than the current shooting condition. In this case, by changing the shooting condition of the main part, the shooting condition of the shooting area other than the main part is also changed. In addition, in the case where the user moves the slider 1113 to the right end side of the adjustment bar, the shooting condition of the main part is set so that it becomes brighter than the current shooting condition. In this case, by changing the shooting condition of the main part, the shooting condition of the shooting area other than the main part is also changed.

[0315] (Variant 8)

[0316] In the above-mentioned embodiment, the shooting conditions are changed for the main part of the subject and the darkest part and the brightest part of the background part of the subject, but the objects to be changed in the shooting conditions are not limited to this. For example, as shown in the first embodiment, the second embodiment and the third embodiment, when multiple subjects are extracted by the object recognition unit 23a, the shooting conditions can be changed for the main subject selected from the multiple subjects and the subjects other than the main subject. That is, the shooting conditions of the other extracted subjects can be changed by changing the shooting conditions of one of the multiple subjects extracted by the object recognition unit 23a. In this case, the amount of change in the shooting conditions of the selected subject and the subjects other than the main subject among the subjects extracted by the object recognition unit 23a can be different. In addition, when the user selects one subject, the shooting conditions of the selected subject can be changed to change the shooting conditions of other subjects (such as people, faces, buildings, the sky, etc.) included in the subject. In this way, the fourth embodiment can also be applied to the first embodiment, the second embodiment and the third embodiment. Thus, even when there are multiple subjects, the shooting conditions of other subjects can be changed by changing the shooting conditions of one subject, so the operation of changing the shooting conditions can be shortened. In addition, the operation of selecting the subject for changing the shooting conditions can be shortened.

[0317] (Variant 9)

[0318] In the above-mentioned embodiments, a single electronic device, namely, an imaging device, having an imaging element 22 and a control unit 33 is described, but the present invention is not limited to such an embodiment. For example, the present invention can also be applied to an electronic device that controls an external imaging element 22. The following describes in detail the form of providing an external imaging element 22.

[0319] Fig.24 This is a block diagram schematically showing the configuration of a photographing system according to modification example 9. Fig.24 The imaging system 4000 shown includes an imaging unit 4001 and an electronic device 4002. The imaging unit 4001 includes the imaging optical system 21 and the imaging element 22 described in the fourth embodiment, and also includes a first communication unit 4003. In addition, the electronic device 4002 includes the control unit 33, the liquid crystal display 24, the memory card 25, the operation unit 26, the DRAM 27, the flash memory 28 and the recording unit 34 described in the fourth embodiment, and also includes a second communication unit 4004. The first communication unit 4003 and the second communication unit 4004 can perform bidirectional data communication using, for example, a well-known wireless communication technology or an optical communication technology.

[0320] In the imaging system 4000 of the modification 9, the control unit 33 controls the imaging element 22 through data communication via the second communication unit 4004 and the first communication unit 4003. For example, by sending and receiving predetermined control data with the imaging unit 4001, different imaging conditions are set for each unit group 32, or imaging signals are read from each unit group 32.

[0321] Moreover, the present invention is not limited by any of the above-mentioned embodiments and includes electronic devices of various structures, which include: a display control unit, which displays the subject image captured by the shooting unit on the display unit, and the shooting unit is controlled according to prescribed shooting conditions for each of a plurality of shooting areas; and a dividing unit which divides the subject image into at least one partial area, and the display control unit causes the display unit to display the partial image and the subject image captured by the shooting unit of the shooting area corresponding to the partial area.

[0322] In addition, the present invention is not limited by any of the above-mentioned embodiments, and includes shooting devices of various structures, which include: the above-mentioned electronic device; a shooting unit that is controlled according to prescribed shooting conditions for each of a plurality of shooting areas; a shooting control unit that controls each of a plurality of shooting areas according to prescribed shooting conditions; a recording unit that records the subject image captured by the shooting unit on a recording medium, the shooting control unit controls the corresponding shooting area according to the shooting conditions set by the setting unit to generate a subject image, and the recording unit records the subject image on the recording medium.

[0323] In addition, the present invention is not limited by the above-mentioned embodiments, and includes shooting systems of various structures, which include the above-mentioned electronic device and the shooting device, the electronic device also includes a first communication unit capable of communicating with the shooting device, and the shooting device also includes: a shooting unit controlled by prescribed shooting conditions for each of a plurality of shooting areas; a shooting control unit, which controls each of the plurality of shooting areas under prescribed shooting conditions to generate a subject image; a second communication unit capable of communicating with the electronic device, the second communication unit sends the subject image to the electronic device via the first communication unit, and the first communication unit sends the shooting conditions set by the setting unit for each partial area in the subject image displayed by the display control unit to the shooting device.

[0324] As described above, in the imaging system 4000, the control unit 33 performs control for each unit group 32. The electronic device 4002 does not have the imaging element 22, but performs the same control as the first embodiment by controlling the imaging element 22 (imaging unit 4001) provided outside the electronic device 4002. That is, the present invention can be applied to electronic devices that do not have the imaging element 22.

[0325] The present invention is not limited to the above-described embodiments unless the features of the present invention are impaired, and other embodiments that can be conceived within the scope of the technical concept of the present invention are also included in the scope of the present invention.

[0326] The disclosures of the following priority basic applications are incorporated herein by reference.

[0327] Japanese Patent Application No. 2014-202104 (filed on September 30, 2014)

[0328] Japanese Patent Application No. 2014-202105 (filed on September 30, 2014)

[0329] Description of Reference Numerals

[0330] 10 information terminal device, 21 shooting optical system, 22 shooting element, 23, 33 control unit, 23a object recognition unit, 23b weighting setting unit, 23c area segmentation unit (area division unit), 23d shooting condition setting unit, 23e image processing unit, 23f shooting control unit, 24 liquid crystal display, 25 memory card, 26 operation unit, 27 DRAM, 28 flash memory, 29 posture detection unit, 34 recording unit, 3010 shooting device.

Claims

1. A photographing element, characterized in that: include: a pixel region having a first region and a second region, wherein a first pixel including a first photoelectric conversion unit for converting light into electric charge is arranged in the first region, and a second pixel including a second photoelectric conversion unit for converting light into electric charge is arranged in the second region; and The control unit performs control so that, when the first imaging condition is set for the first pixel based on the user's operation on the first area, the second imaging condition is set for the second pixel even when the user's operation on the second area is not performed.

2. The imaging element according to claim 1, wherein: In the pixel region, the second pixel is arranged on the row direction side with respect to the first pixel.

3. The imaging element according to claim 2, wherein: have: a first control line to which a first control signal for controlling the first pixel is output; and A second control line to which a second control signal for controlling the second pixel is outputted.

4. The imaging element according to claim 3, wherein: outputting the first control signal for controlling the accumulation time of the charge converted by the first photoelectric conversion unit to the first control line, The second control signal for controlling the accumulation time of the charges converted by the second photoelectric conversion unit is output to the second control line.

5. The imaging element according to claim 4, wherein: As the first imaging condition, the control unit controls the accumulation time of the charges converted by the first photoelectric conversion unit so that the accumulation time of the charges converted by the first photoelectric conversion unit becomes a first accumulation period. As the second imaging condition, the control unit controls an accumulation time of the charges converted by the second photoelectric conversion unit so that the accumulation time of the charges converted by the second photoelectric conversion unit becomes a second accumulation period different from the first accumulation period.

6. The imaging element according to claim 3, wherein: outputting the first control signal for controlling the frame rate of a first signal based on the charge converted by the first photoelectric conversion unit from the first pixel to the first control line, The second control signal for controlling a frame rate of a second signal based on the charge converted by the second photoelectric conversion unit for reading out from the second pixel is output to the second control line.

7. The imaging element according to claim 6, wherein: As the first imaging condition, the control unit performs control so that a frame rate for reading the first signal from the first pixel becomes a first frame rate. As the second imaging condition, the control unit performs control such that a frame rate for reading the second signal from the second pixel becomes a second frame rate different from the first frame rate.

8. The imaging element according to claim 3, wherein: The first pixel has a first transfer unit that transfers the charge converted by the first photoelectric conversion unit. The second pixel has a second transfer unit that transfers the charge converted by the second photoelectric conversion unit. The first transmission unit is electrically connected to the first control line. The second transmission unit is electrically connected to the second control line.

9. The imaging element according to claim 8, wherein: outputting the first control signal for controlling the timing of transferring the charge from the first photoelectric conversion unit to the first control line, The second control signal for controlling the timing of transferring the charge from the second photoelectric conversion unit is output to the second control line.

10. The imaging element according to claim 9, wherein: As the first imaging condition, the control unit performs control so that the timing of transferring the charge from the first photoelectric conversion unit becomes a first timing, As the second imaging condition, the control unit performs control so that the timing of transferring the charge from the second photoelectric conversion unit becomes a second timing different from the first timing.

11. The imaging element according to claim 8, wherein: have: a third control line to which a third control signal for controlling the first pixel is output; and a fourth control line to which a fourth control signal for controlling the second pixel is output, The first pixel has a first reset portion that releases the charge converted by the first photoelectric conversion portion. The second pixel has a second reset portion that releases the charge converted by the second photoelectric conversion portion. The first reset portion is electrically connected to the third control line. The second reset portion is electrically connected to the fourth control line.

12. The imaging element according to claim 11, wherein: outputting the first control signal for controlling the timing of transferring the charge from the first photoelectric conversion unit to the first control line, outputting the second control signal for controlling the timing of transferring the charge from the second photoelectric conversion unit to the second control line, outputting the third control signal for controlling the timing of releasing the charge from the first photoelectric conversion unit to the third control line, The fourth control signal for controlling the timing of discharging the charge from the second photoelectric conversion portion is output to the fourth control line.

13. The imaging element according to claim 12, wherein: As the first imaging condition, the control unit performs control so that the timing of transferring the charge from the first photoelectric conversion unit becomes the first timing and the timing of releasing the charge from the first photoelectric conversion unit becomes the second timing. As the second imaging condition, the control unit performs control such that a timing of transferring charges from the second photoelectric conversion unit becomes a third timing different from the first timing and a timing of releasing charges from the second photoelectric conversion unit becomes a fourth timing different from the second timing.

14. The imaging element according to claim 3, wherein: The first pixel has a first reset portion that releases the charge converted by the first photoelectric conversion portion. The second pixel has a second reset portion that releases the charge converted by the second photoelectric conversion portion. The first reset portion is electrically connected to the first control line. The second reset section is electrically connected to the second control line.

15. The imaging element according to claim 14, wherein: outputting the first control signal for controlling the timing of releasing the charge from the first photoelectric conversion unit to the first control line, The second control signal for controlling the timing of discharging the charge from the second photoelectric conversion portion is output to the second control line.

16. The imaging element according to claim 15, wherein: As the first imaging condition, the control unit performs control so that the timing of discharging the charge from the first photoelectric conversion unit becomes a first timing, As the second imaging condition, the control unit performs control so that the timing of discharging the electric charge from the second photoelectric conversion unit becomes a second timing different from the first timing.

17. The imaging element according to any one of claims 1 to 16, characterized in that: The control unit sets the second imaging condition within a range in which a brightness value of a subject imaged in the second area does not exceed a predetermined upper limit value.

18. The imaging element according to any one of claims 1 to 13, characterized in that: The control unit sets the second imaging condition within a range such that a brightness value of a subject imaged in the second area does not exceed a predetermined lower limit value.

19. The imaging element according to any one of claims 1 to 16, characterized in that: The control unit sets the second imaging condition within a range in which a brightness value of the subject imaged in the second area does not exceed a predetermined upper limit value and a brightness value of the subject imaged in the second area does not exceed a predetermined lower limit value.

20. The imaging element according to any one of claims 1 to 16, characterized in that: The control unit sets the second imaging condition so as to maintain a difference between a brightness value of a subject imaged in the first area and a brightness value of a subject imaged in the second area before and after the first imaging condition is set for the first area.

21. The imaging element according to any one of claims 1 to 16, characterized in that: The control unit sets the second imaging condition so that a difference between a brightness value of a subject captured by the first area and a brightness value of a subject captured by the second area becomes larger before and after the first imaging condition is set for the first area.

22. A photographing device, characterized in that: A device comprising the imaging element according to any one of claims 1 to 16.

23. The photographing device according to claim 22, characterized in that: An operation unit is provided for accepting an operation of the first area by a user.

24. The photographing device according to claim 23, characterized in that: The operation unit receives an operation of the first area by a user using a display surface that displays an image.

25. A photographing device, characterized in that: A camera device according to claim 17 is provided.

26. The photographing device according to claim 25, characterized in that: An operation unit is provided for accepting an operation of the first area by a user.

27. The photographing device according to claim 26, wherein: The operation unit receives an operation of the first area by a user using a display surface that displays an image.

28. A photographing device, characterized in that: A camera device according to claim 18 is provided.

29. The photographing device according to claim 28, characterized in that: An operation unit is provided for accepting an operation of the first area by a user.

30. The photographing device according to claim 29, wherein: The operation unit receives an operation of the first area by a user using a display surface that displays an image.

31. A photographing device, characterized in that: A camera device according to claim 19 is provided.

32. The photographing device according to claim 31, characterized in that: An operation unit is provided for accepting an operation of the first area by a user.

33. The photographing device according to claim 32, characterized in that: The operation unit receives an operation of the first area by a user using a display surface that displays an image.

34. A photographing device, characterized in that: A camera device according to claim 20 is provided.

35. The photographing device according to claim 34, characterized in that: An operation unit is provided for accepting an operation of the first area by a user.

36. The photographing device according to claim 35, characterized in that: The operation unit receives an operation of the first area by a user using a display surface that displays an image.

37. A photographing device, characterized in that: A camera device according to claim 21 is provided.

38. The photographing device according to claim 37, characterized in that: An operation unit is provided for accepting an operation of the first area by a user.

39. The photographing device according to claim 38, characterized in that: The operation unit receives an operation of the first area by a user using a display surface that displays an image.

40. A photographing element, characterized in that: have: a pixel region configured with a plurality of pixels; and a control unit configured to set a second shooting condition for shooting the subject in a second area of ​​the pixel area when a first shooting condition for shooting the subject is set in a first area of ​​the pixel area, The control unit performs control so as to automatically set the second imaging condition in conjunction with setting of the first imaging condition.

41. The imaging element according to claim 40, wherein: A first pixel including a first photoelectric conversion unit for converting light into electric charges is arranged in the first region. A second pixel including a second photoelectric conversion unit for converting light into electric charges is arranged in the second region. The second pixel is arranged on the row direction side with respect to the first pixel in the pixel region.

42. The imaging element according to claim 41, wherein: have: a first control line to which a first control signal for controlling the first pixel is output; and A second control line to which a second control signal for controlling the second pixel is outputted.

43. The imaging element according to claim 42, wherein: outputting the first control signal for controlling the accumulation time of the charge converted by the first photoelectric conversion unit to the first control line, The second control signal for controlling the accumulation time of the charges converted by the second photoelectric conversion unit is output to the second control line.

44. The imaging element according to claim 43, wherein: As the first imaging condition, the control unit controls the accumulation time of the charges converted by the first photoelectric conversion unit so that the accumulation time of the charges converted by the first photoelectric conversion unit becomes a first accumulation period. As the second imaging condition, the control unit controls an accumulation time of the charges converted by the second photoelectric conversion unit so that the accumulation time of the charges converted by the second photoelectric conversion unit becomes a second accumulation period different from the first accumulation period.

45. The imaging element according to claim 42, wherein: The first pixel has a first transfer unit that transfers the charge converted by the first photoelectric conversion unit. The second pixel has a second transfer unit that transfers the charge converted by the second photoelectric conversion unit. The first transmission unit is electrically connected to the first control line. The second transmission unit is electrically connected to the second control line.

46. ​​The imaging element according to claim 45, wherein: outputting the first control signal for controlling the timing of transferring the charge from the first photoelectric conversion unit to the first control line, The second control signal for controlling the timing of transferring the charge from the second photoelectric conversion unit is output to the second control line.

47. The imaging element according to claim 46, wherein: As the first imaging condition, the control unit performs control so that the timing of transferring the charge from the first photoelectric conversion unit becomes a first timing, As the second imaging condition, the control unit performs control so that the timing of transferring the charge from the second photoelectric conversion unit becomes a second timing different from the first timing.

48. The imaging element according to claim 42, wherein: The first pixel has a first reset portion that releases the charge converted by the first photoelectric conversion portion. The second pixel has a second reset section that releases the charges converted by the second photoelectric conversion section. The first reset portion is electrically connected to the first control line. The second reset section is electrically connected to the second control line.

49. The imaging element according to claim 48, wherein: outputting the first control signal for controlling the timing of releasing the charge from the first photoelectric conversion unit to the first control line, The second control signal for controlling the timing of discharging the charge from the second photoelectric conversion portion is output to the second control line.

50. The imaging element according to claim 49, wherein: As the first imaging condition, the control unit performs control so that the timing of discharging the charge from the first photoelectric conversion unit becomes a first timing, As the second imaging condition, the control unit performs control so that the timing of discharging the electric charge from the second photoelectric conversion unit becomes a second timing different from the first timing.

51. A photographing device, characterized in that: A method comprising: providing an imaging element according to any one of claims 40 to 50.

52. The photographing device according to claim 51, characterized in that: An operation unit is provided for accepting a user's operation for setting the first imaging condition in the first area.

53. The photographing device according to claim 52, characterized in that: The operation unit receives a user's operation for setting the first imaging condition in the first area using a display surface that displays an image.

Citation Information

Patent Citations

  • Semiconductor module and MOS solid-state imaging device

    JP2006049361A

  • Speedfactory 3D

    JP2015077787A

  • Image pick-up apparatus and image pick-up method

    CN102088558A

  • Imaging device

    JP2013258438A

  • Image capture with adjustment of imaging properties at transitions between regions

    US20120069212A1