Imaging element

Abstract

To solve a problem in which the more image data sent from an image sensor to an external circuit, the longer the processing time for outputting the data from the image sensor becomes.SOLUTION: An imaging element 3 includes a first substrate 111 having a plurality of pixels that outputs signals based on photoelectrically converted charges, a second substrate 112 having an A / D conversion unit that converts into digital signals a first signal output from at least a first pixel among the plurality of pixels and a second signal output from the first pixel after the first signal, and a third substrate 113 having a calculation unit that controls so as to output at least one of the first digital signal and the second digital signal, on the basis of a first evaluation value obtained using the first digital signal obtained by converting the first signal into a digital signal in the A / D conversion unit and a second evaluation value obtained using the second digital signal obtained by converting the second signal into a digital signal in the A / D conversion unit.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an imaging device. This application claims priority based on Japanese Patent Application No. 2021-061148, filed on March 31, 2021, the contents of which are incorporated herein by reference. [Background technology]

[0002] The image data captured by the image sensor is processed by an external circuit called an image processing engine. The more image data sent from the image sensor to the external circuit, the longer the processing time to output the data from the image sensor becomes. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-30097 A Summary of the Invention

[0004] An imaging element according to a first aspect of the present invention comprises a first substrate having a plurality of pixels that output signals based on photoelectrically converted charges; a second substrate having a first conversion unit that converts into digital signals a first signal output from at least a first pixel among the plurality of pixels and a second signal output from the first pixel after the first signal; and a third substrate having a calculation unit that controls to output at least one of the first digital signal and the second digital signal based on a first evaluation value obtained using a first digital signal obtained by converting the first signal into a digital signal in the first conversion unit and a second evaluation value obtained using a second digital signal obtained by converting the second signal into a digital signal in the first conversion unit. [Brief description of the drawings]

[0005] [Figure 1] 1 is a block diagram illustrating a configuration of an imaging device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a cross-sectional structure of an imaging element. [Diagram 3] 2 is a block diagram illustrating an example of the configuration of each layer of a first substrate to a fourth substrate in an imaging element. FIG. [Figure 4] 2 is a schematic diagram illustrating data transfer between an imaging element and an image processing engine according to the first embodiment. FIG. [Diagram 5] FIG. 2 is a diagram illustrating a shooting range captured by an image sensor. [Figure 6] Fig. 6(a) is a diagram illustrating an example of an image acquired during a half-press operation. Figs. 6(b) to 6(e) are diagrams illustrating an example of an image acquired during a full-press operation. Figs. 6(f) to 6(i) are diagrams each showing a schematic diagram of an image shift amount of the pair of images acquired by a pixel having a photoelectric conversion unit for focus detection when capturing the images illustrated in Figs. 6(b) to 6(e), respectively. [Figure 7] 13 is a schematic diagram illustrating data transfer between an imaging element and an image processing engine according to a second embodiment. FIG. [Figure 8] Fig. 8(a) is a diagram illustrating an example of an image acquired during a half-press operation. Figs. 8(b) to 8(e) are diagrams illustrating an example of an image acquired during a full-press operation. Figs. 8(f) to 8(i) are diagrams each showing a schematic diagram of an image shift amount of the pair of images acquired by a pixel having a photoelectric conversion unit for focus detection when capturing the images illustrated in Figs. 8(b) to 8(e), respectively. [Figure 9] FIG. 13 is a schematic diagram illustrating data exchange between an imaging element and an image processing engine according to a third embodiment. [Figure 10] Fig. 10(a) is a diagram illustrating an example of an image acquired during a half-press operation. Fig. 10(b) to Fig. 10(e) are diagrams illustrating an example of an image acquired during a full-press operation. Fig. 10(f) is a diagram illustrating a typical difference in the data of the images illustrated in Fig. 10(b) and Fig. 10(c), Fig. 10(g) is a diagram illustrating a typical difference in the data of the images illustrated in Fig. 10(c) and Fig. 10(d), and Fig. 10(h) is a diagram illustrating a typical difference in the data of the images illustrated in Fig. 10(d) and Fig. 10(e). [Figure 11] FIG. 2 is a diagram illustrating an example of a configuration of a calculation unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] Hereinafter, an embodiment of the invention will be described with reference to the drawings. (First embodiment) <Configuration of imaging device> FIG. 1 is a block diagram illustrating a configuration of an imaging device equipped with an imaging element according to an embodiment. The imaging device 1 includes a photographing optical system 2, an imaging element 3, and a control unit 4, and is configured so that a storage medium 5 such as a memory card can be detachably attached. The imaging device 1 is, for example, a camera. The photographing optical system 2 forms a subject image on the imaging element 3. The imaging element 3 captures the subject image formed by the photographing optical system 2 and generates an image signal. The imaging element 3 is, for example, a CMOS image sensor. The control unit 4 outputs a control signal for controlling the operation of the imaging element 3 to the imaging element 3. The control unit 4 further functions as an image generating unit that performs various image processes on the image signal output from the imaging element 3 and generates image data. The generated image data is recorded in a predetermined file format in the storage medium 5. The photographic optical system 2 may be configured to be detachable from the imaging device 1.

[0007] <Cross-sectional structure of image sensor> FIG. 2 is a diagram illustrating a cross-sectional structure of the imaging element 3 in FIG. 1. The imaging element 3 shown in FIG. 2 is a back-illuminated imaging element. The imaging element 3 includes a first substrate 111, a second substrate 112, a third substrate 113, and a fourth substrate 114. The first substrate 111, the second substrate 112, the third substrate 113, and the fourth substrate 114 are each formed of a semiconductor substrate or the like. The first substrate 111 is laminated on the second substrate 112 via a wiring layer 140 and a wiring layer 141. The second substrate 112 is laminated on the third substrate 113 via a wiring layer 142 and a wiring layer 143. The third substrate 113 is laminated on the fourth substrate 114 via a wiring layer 144 and a wiring layer 145.

[0008] Incident light L, indicated by a white arrow, is incident in the positive direction of the Z axis. As shown in the coordinate system, the right direction on the paper, perpendicular to the Z axis, is the positive direction of the X axis, and the front direction on the paper, perpendicular to the Z axis and the X axis, is the positive direction of the Y axis. The imaging element 3 has a first substrate 111, a second substrate 112, a third substrate 113, and a fourth substrate 114 stacked in the direction in which the incident light L is incident.

[0009] The imaging element 3 further includes a microlens layer 101, a color filter layer 102, and a passivation layer 103. The passivation layer 103, the color filter layer 102, and the microlens layer 101 are laminated on a first substrate 111 in this order. The microlens layer 101 has a plurality of microlenses ML. The microlenses ML focus incident light onto a photoelectric conversion unit, which will be described later. The color filter layer 102 has a plurality of color filters F. The passivation layer 103 is made of a nitride film or an oxide film.

[0010] The first substrate 111, the second substrate 112, the third substrate 113, and the fourth substrate 114 each have a first surface 105a, 106a, 107a, 108a on which a gate electrode and a gate insulating film are provided, and a second surface 105b, 106b, 107b, 108b different from the first surface. In addition, various elements such as transistors are provided on the first surface 105a, 106a, 107a, 108a. Wiring layers 140, 141, 144, 145 are laminated and provided on the first surface 105a of the first substrate 111, the first surface 106a of the second substrate 112, the first surface 107a of the third substrate 113, and the first surface 108a of the fourth substrate 114, respectively. Moreover, wiring layers (inter-substrate connection layers) 142, 143 are respectively laminated on the second surface 106b of the second substrate 112 and the second surface 107b of the third substrate 113. The wiring layers 140 to 145 are layers including a conductor film (metal film) and an insulating film, and each has a plurality of wirings, vias, and the like arranged therein.

[0011] The elements on the first surface 105a of the first substrate 111 and the elements on the first surface 106a of the second substrate 112 are electrically connected by connection parts 109 such as bumps and electrodes via wiring layers 140 and 141. Similarly, the elements on the first surface 107a of the third substrate 113 and the elements on the first surface 108a of the fourth substrate 114 are electrically connected by connection parts 109 such as bumps and electrodes via wiring layers 144 and 145. In addition, the second substrate 112 and the third substrate 113 have a plurality of through electrodes 110. The through electrodes 110 of the second substrate 112 connect the circuits provided on the first surface 106a and the second surface 106b of the second substrate 112 to each other, and the through electrodes 110 of the third substrate 113 connect the circuits provided on the first surface 107a and the second surface 107b of the third substrate 113 to each other. The circuit provided on the second surface 106b of the second substrate 112 and the circuit provided on the second surface 107b of the third substrate 113 are electrically connected by connection parts 109 such as bumps or electrodes via inter-substrate connection layers 142, 143. In the embodiment, a case in which first substrate 111, second substrate 112, third substrate 113, and fourth substrate 114 are stacked is illustrated, but the number of stacked substrates may be more or less than that in the embodiment. Furthermore, first substrate 111, second substrate 112, third substrate 113, and fourth substrate 114 may be referred to as the first layer, the second layer, the third layer, and the fourth layer, respectively.

[0012] <Example of imaging element configuration> FIG. 3 is a block diagram illustrating the configuration of each layer of the first substrate 111 to the fourth substrate 114 in the imaging element 3 according to the embodiment. The first substrate 111 has, for example, a plurality of pixels 10 arranged two-dimensionally and a signal readout section 20 (referred to as a pixel array 210). The pixels 10 are arranged in the X-axis direction (row direction) and the Y-axis direction (column direction) shown in FIG. 2. The pixels 10 have a photoelectric conversion section such as a photodiode (PD) and convert incident light L into an electric charge. The readout section 20 is provided for each pixel 10 and reads out a signal (photoelectric conversion signal) based on the electric charge photoelectrically converted by the corresponding pixel 10. A readout control signal required for the readout section 20 to read out a signal from the pixel 10 is supplied to the readout section 20 from the scanning section 260 of the second substrate 112. The signal read out by the readout section 20 is sent to the second substrate 112.

[0013] The second substrate 112 has, for example, an A / D conversion unit 230 and a scanning unit 260. The A / D conversion unit 230 converts a signal output from a corresponding pixel 10 into a digital signal. The signal converted by the A / D conversion unit 230 is sent to the third substrate 113. The scanning unit 260 generates a read control signal for the reading unit 20 based on an instruction signal input via the input unit 290 of the fourth board 114. The instruction signal is sent from the image processing engine 30, which will be described later with reference to FIG. 4. The read control signal generated by the scanning unit 260 is sent to the first board 111.

[0014] The third substrate 113 has, for example, a memory 250, a calculation unit 240, and an evaluation value storage unit 280. The memory 250 stores the digital signals converted by the A / D conversion unit 230. The memory 250 has, for example, a storage capacity capable of storing at least two frames of images generated based on the digital signals converted by the A / D conversion unit 230. In the embodiment, one image is referred to as one frame of image.

[0015] The calculation unit 240 performs a predetermined calculation using at least one of the digital signals stored in the memory 250 and the digital signal converted by the A / D conversion unit 230. This calculation calculates an evaluation value for the image. The evaluation value indicates, for example, any one of the focus adjustment state of the photographing optical system 2, the brightness of the image generated based on the digital signal, and the state of movement of the main subject. The calculation unit 240 can judge the superiority or inferiority of multiple images based on the calculated evaluation value. The evaluation value storage unit 280 stores a reference value for objectively evaluating the evaluation value calculated by the calculation unit 240. The reference value is recorded in the evaluation value storage unit 280 in advance. Incidentally, since the evaluation value storage unit 280 is used in the second and third embodiments, it may be omitted in the first embodiment.

[0016] The fourth board 114 has, for example, an output unit 270 and an input unit 290. The output unit 270 sends the digital signal stored in the memory 250 or the digital signal converted by the A / D conversion unit 230 to the image processing engine 30 (FIG. 4) described later. For example, when data of an image for recording, which will be described later, is sent to the image processing engine 30, the digital signal stored in the memory 250 is sent as data of the image for recording to the image processing engine 30. When data of an image for monitor display, which will be described later, is sent to the image processing engine 30, the digital signal converted by the A / D conversion unit 230 is sent to the image processing engine 30 as data of the image for monitor display. An instruction signal is input to the input section 290 from the image processing engine 30 via the input section 290 of the fourth board 114. The instruction signal is sent to the second board 112.

[0017] <Image sensor and image processing engine> 4 is a schematic diagram illustrating data transfer between the image sensor 3 according to the first embodiment and the image processing engine 30. For example, when an instruction to shoot for recording is input from the image processing engine 30, the image sensor 3 captures a plurality of images while the shooting instruction is continuing, selects a superior image from the plurality of images based on an evaluation value, and sends data of the selected image to the image processing engine 30 as data of an image for recording. Furthermore, when an instruction to shoot for monitor display is input from the image processing engine 30, the image sensor 3 repeatedly captures an image for monitor display, and sends data of the captured image to the image processing engine 30 as data of an image for monitor display (also called a through image or a live view image). The image processing engine 30 constitutes a part of the control unit 4, and has an image sensor control unit 310, an input unit 320, an image processing unit 330, and a memory 340.

[0018] The operation members 6 including a release button, an operation switch, etc. are provided, for example, on the exterior surface of the imaging device 1. The operation members 6 send operation signals corresponding to user operations to the image sensor control unit 310. The user operates the operation members 6 to give the imaging device 1 instructions for shooting and settings such as shooting conditions.

[0019] When an instruction to set shooting conditions, etc. is given, the image sensor control unit 310 sends information indicating the set shooting conditions to the image sensor 3. Furthermore, when a half-press operation signal indicating that the release button has been half-pressed with a stroke shorter than the stroke when fully pressed is input from the operation member 6, the image sensor control unit 310 sends an instruction signal to the image sensor 3 instructing to start shooting for monitor display in order to continuously display images for monitor display on a display unit or viewfinder (not shown). Furthermore, when a full press operation signal indicating that the release button has been fully pressed with a longer stroke than when it is half-pressed is input from the operating member 6, the image sensor control unit 310 sends an instruction signal to the image sensor 3 to instruct it to start taking a still image for recording.

[0020] The input unit 320 receives the digital signal output from the imaging element 3. The input digital signal is sent to the image processing unit 330. The image processing unit 330 performs predetermined image processing on the digital signal obtained from the imaging element 3 to generate image data. The generated image data for recording is recorded in the memory 340 or used to display a confirmation image after shooting. The generated image data for monitor display is used for display in a viewfinder or the like. The image data recorded in the memory 340 can be recorded in the storage medium 5.

[0021] <Detailed explanation of the calculation unit> In the first embodiment, the calculation section 240 of the image sensor 3 calculates an evaluation value indicating the state of focus adjustment of the photographing optical system 2. The pixels 10 constituting the pixel array 210 of the image sensor 3 have a photoelectric conversion unit for image generation. However, in a part or all of the area corresponding to the focus point, pixels 10 having a photoelectric conversion unit for focus detection instead of a photoelectric conversion unit for image generation are arranged.

[0022] The focus points will now be described. Fig. 5 is a diagram illustrating a shooting range 50 captured by the image sensor 3. In the shooting range 50, for example, a plurality of focus points P are provided in advance. Fig. 5 illustrates the focus points P together with a captured live view image. The focus point P indicates a position in the shooting range 50 where focus adjustment of the shooting optical system 2 is possible, and is also called a focus detection area, focus detection position, or distance measurement point. It should be noted that the number of focus points P and their positions in the shooting range 50 shown in the figure are merely an example, and are not limited to the embodiment shown in FIG.

[0023] The calculation unit 244 of the calculation unit 240 calculates, as an evaluation value, an image shift amount (phase difference) between a pair of images caused by a pair of light beams passing through different regions of the photographing optical system 2, based on, for example, a photoelectric conversion signal from a pixel 10 having a photoelectric conversion unit for focus detection. The image shift amount can be calculated for each focus point P. The image shift amount between the pair of images is a value that is used to calculate the defocus amount, which is the shift amount between the position of the image of the subject formed by the light beam that has passed through the photographing optical system 2 and the position of the imaging surface of the image sensor 3. The defocus amount can be calculated by multiplying the image shift amount between the pair of images by a predetermined conversion coefficient.

[0024] The calculation unit 244 determines, for example, the amount of image shift between the pair of images calculated at a focus point P located approximately in the center of the shooting range 50 among the multiple focus points P, as the evaluation value for that image. In addition, the calculation unit 240 can automatically select a focus point P to be used for calculating the image shift amount between the pair of images (in other words, calculating the evaluation value) from among all focus points P, or can select a focus point P specified by user operation.

[0025] The calculation unit 244 calculates an evaluation value for each of the multiple images captured while the recording shooting instruction is continuing. The selection unit 245 of the calculation unit 240 selects a superior image from the multiple images based on the evaluation value. In the first embodiment, the image with the smallest image shift amount between the pair of images, in other words, the image with the best focus, is selected. Data of the image selected by the selection unit 245 is sent to the image processing engine 30 via the output unit 270.

[0026] The process performed by the calculation unit 240 will be described in detail with reference to Fig. 6. Fig. 6(a) to Fig. 6(e) are diagrams illustrating images captured by pixels 10 having photoelectric conversion units for generating images. Fig. 6(a) is an example of an image captured before an instruction to capture for recording is given, in other words, during a half-press operation. Fig. 6(b) to Fig. 6(e) are examples of images captured while an instruction to capture for recording is continuing, in other words, during a full-press operation.

[0027] Figures 6(f) to 6(i) are schematic diagrams showing the amount of image shift between the pair of images acquired by a pixel 10 having a photoelectric conversion unit for focus detection when capturing the images illustrated in Figures 6(b) to 6(e), respectively. A pixel 10 having a photoelectric conversion unit for focus detection obtains a pair of images by a pair of light beams passing through different regions of the photographing optical system 2. When the photographing optical system 2 forms a sharp image at a planned focal plane, that is, when the photographing optical system 2 is in focus, the pair of images relatively coincide (the amount of deviation is almost zero), as shown in Fig. 6(g) as an example. In contrast, when the photographing optical system 2 forms a sharp image in front of the planned focal plane, that is ... is in a front-focus state, and when the photographing optical system 2 forms a sharp image behind the planned focal plane, that is, when the photographing optical system 2 is in a back-focus state, the pair of images are relatively far apart, and the amount of image deviation increases, as shown in Fig. 6(f) or 6(h) as an example. The calculation unit 240 calculates the amount of image shift by performing a known image shift detection calculation process (correlation calculation process, phase difference detection process) on a signal sequence indicating the pair of images acquired by a plurality of pixels 10 having photoelectric conversion units for focus detection arranged in a line in the X-axis direction, for example.

[0028] In FIG. 6(a), a user half-presses the release button to prepare to capture an image of an object 70 suspended by a string with the imaging device 1. The object 70 is assumed to be swinging left and right and in the depth direction of the figure. The user holds the imaging device 1 so that the swinging object 70 is located approximately in the center of the shooting range. When the user fully presses the button, the imaging device 1 captures multiple images with the imaging element 3 while the button is being fully pressed, as exemplified in FIG. 6(b) to FIG. 6(e). In this embodiment, four images are captured. In the embodiment, it is assumed that an area 60 for calculating an evaluation value by the calculation unit 240 is set in advance in approximately the center of the shooting range. By setting the area 60 in a part of the shooting range, the processing load of the calculation unit 240 for calculating an evaluation value can be reduced compared to when the area 60 is set in the entire shooting range. The calculation unit 240 can set the area 60 automatically, or can set the area 60 to a range specified by a user operation.

[0029] The data of the first image captured by the image sensor 3 during the full press operation is stored in the first area 251 of the memory 250. The data of the second image captured by the image sensor 3 during the full press operation is stored in the second area 252 of the memory 250. The image data stored in the memory 250 includes an image captured by the pixel 10 having a photoelectric conversion unit for image generation and the pair of images acquired by the pixel 10 having a photoelectric conversion unit for focus detection. The selection unit 245 judges the superiority or inferiority between the first image and the second image based on the evaluation value calculated by the calculation unit 244 for the images stored in the first area 251 and the second area 252 of the memory 250. In the first image illustrated in FIG. 6(b), the object 70 is farther away in the depth direction than in the second image illustrated in FIG. 6(c). Therefore, as illustrated in FIG. 6(f), the pair of images corresponding to the first image are relatively farther away than the pair of images (FIG. 6(g)) corresponding to the second image.

[0030] The selection unit 245 selects the second image illustrated in FIG. 6(c) because the image shift amount of the pair of images illustrated in FIG. 6(g) is smaller (in other words, the evaluation value is higher) than the image shift amount of the pair of images illustrated in FIG. 6(f). The calculation unit 240 makes the data of the unselected images erasable. The reason for making the data of the unselected images erasable is to store the data of the next image in the same area of ​​the memory 250. The calculation unit 240 stores the third image captured next in the same area of ​​the memory 250 in which the data of the image made erasable was stored. In the embodiment, the data of the unselected first image is stored in the first area 251 of the memory 250 in which the data of the image made erasable was stored.

[0031] The selection unit 245 judges the relative merits between the third image and the second image based on the evaluation values ​​for the images stored in the first area 251 and the second area 252 of the memory 250. In the third image illustrated in Fig. 6(d), the object 70 is closer to the front than in the second image illustrated in Fig. 6(c). Therefore, as illustrated in Fig. 6(h), the pair of images corresponding to the third image are relatively farther apart than the pair of images (Fig. 6(g)) corresponding to the second image.

[0032] The selection unit 245 selects the second image shown in FIG. 6(c) because the image shift amount of the pair of images shown in FIG. 6(g) is smaller (in other words, the evaluation value is higher) than the image shift amount of the pair of images shown in FIG. 6(h). The calculation unit 240 makes the data of the images not selected erasable. The calculation unit 240 stores the fourth image captured next in the same area of ​​the memory 250 in which the data of the image made erasable was stored. In the embodiment, the calculation unit 240 stores the data of the third image not selected in the first area 251 of the memory 250 in which the data of the image made erasable was stored.

[0033] The selection unit 245 judges the relative merits between the fourth image and the second image based on the evaluation values ​​for the images stored in the first area 251 and the second area 252 of the memory 250. In the fourth image illustrated in FIG. 6(e), the object 70 is slightly farther away in the depth direction than in the second image illustrated in FIG. 6(c). Therefore, as illustrated in FIG. 6(i), the pair of images corresponding to the fourth image are relatively slightly farther apart than the pair of images corresponding to the second image (FIG. 6(g)).

[0034] The selection unit 245 selects the second image shown in FIG. 6(c) because the image shift amount of the pair of images shown in FIG. 6(g) is smaller (in other words, has a higher evaluation value) than the image shift amount of the pair of images shown in FIG. 6(i).

[0035] As described above, when captured image data is stored in the first area 251 or the second area 252 of the memory 250, the calculation unit 240 of the imaging element 3 calculates an evaluation value for that image (the image shift amount between the pair of images) by the calculation unit 244. Then, based on the calculated evaluation value, the selection unit 245 selects one image in which the object 70 is best focused from among the four images captured by the imaging element 3 during the full press operation. Only the data of the image selected by the selection unit 245 is sent to the image processing engine 30 via the output unit 270.

[0036] The image processing engine 30 receives the image data output from the imaging element 3 , and performs a predetermined image processing in the image processing unit 330 to generate image data for recording, which is then recorded in the memory 340 .

[0037] According to the first embodiment described above, the following advantageous effects can be obtained. (1) The image sensor 3 includes a plurality of pixels 10 that output signals based on photoelectrically converted charges, a calculation unit 244 that calculates an evaluation value of an image generated based on the above signal, a selection unit 245 that selects one image from the plurality of images based on the evaluation values ​​of each of the plurality of images calculated by the calculation unit 244, and an output unit 270 that outputs data of the image selected by the selection unit 245. With this configuration, it is possible to reduce the amount of image data output from the imaging element 3 compared to when data for all of the multiple images captured by the imaging element 3 is output from the imaging element 3 to the image processing engine 30. This makes it possible to reduce the processing time required to output data and the power consumption in the imaging element 3.

[0038] (2) The selection unit 245 of the imaging element 3 selects one image with the highest evaluation value from among the multiple images. With this configuration, it is possible to reduce the image data output from the imaging element 3 to the data for one image. Also, it is possible to output data for an image that is considered to be the best of the multiple images based on the evaluation value to the image processing engine 30.

[0039] (4) The multiple pixels 10 of the image sensor 3 photoelectrically convert the light that has passed through the photographing optical system 2 , and the calculation unit 244 calculates an evaluation value that indicates the state of focus adjustment of the photographing optical system 2 . With this configuration, data of the image that is best in focus among the multiple images can be output to the image processing engine 30.

[0040] (5) The calculation unit 244 of the image sensor 3 calculates an evaluation value based on a signal that corresponds to a predetermined region 60 of the image, out of the above signals. With this configuration, the burden of the calculation unit 244 in calculating the evaluation value can be reduced compared to when the evaluation value is calculated based on signals corresponding to the entire shooting range 50.

[0041] (Second embodiment) In the first embodiment, one image in which the object 70 is best in focus is selected from among the multiple images captured by the imaging element 3 during a full press operation, and data of the selected image is sent to the image processing engine 30. Instead, in the second embodiment, data of the image in which the object 70 is in focus is sent to the image processing engine 30 from among the multiple images captured by the imaging element 3 during a full press operation. The number of images sent to the image processing engine 30 is not limited to one. Such a configuration will be described below.

[0042] <Image sensor and image processing engine> 7 is a schematic diagram for explaining data exchange between an image sensor 3A according to the second embodiment and an image processing engine 30. The image sensor 3A is used in place of the image sensor 3 according to the first embodiment. The image sensor 3A differs from the image sensor 3 in that it has a calculation unit 240A and an evaluation value storage unit 280. Therefore, the following explanation will focus on these differences, and the same components as those in the image sensor 3 are denoted by the same reference numerals as in FIG. 4 and will not be explained further.

[0043] For example, when an instruction to capture a photograph for recording is input from the image processing engine 30, the image sensor 3A captures a plurality of images while the capture instruction is continuing, and transmits data of an image among the plurality of images that is determined to be in focus on the object 70 to the image processing engine 30 as data of an image for recording. In addition, when an instruction to capture an image for monitor display is input from the image processing engine 30, the image capture for monitor display is performed repeatedly, and data of the captured image is sent to the image processing engine 30 as data of the image for monitor display, just like in the case of the image sensor 3.

[0044] <Detailed explanation of the calculation unit> In the second embodiment, the calculation unit 240A of the image sensor 3A calculates an evaluation value indicating the state of focus adjustment of the photographing optical system 2. This evaluation value is similar to the evaluation value calculated by the calculation unit 240 of the image sensor 3 according to the first embodiment.

[0045] The calculation unit 244 of the calculation unit 240A calculates an evaluation value for each of the multiple images captured while the instruction to capture for recording is continuing. The selection unit 245 of the calculation unit 240A compares the evaluation value calculated by the calculation unit 244 with a reference value previously recorded in the evaluation value storage unit 280. The reference value is, for example, a tolerable value for the image shift amount of the pair of images. The selection unit 245 selects an image whose calculated evaluation value satisfies the reference value, in other words, whose image shift amount of the pair of images as an evaluation value is smaller than the image shift amount corresponding to the reference value. In the second embodiment, when an image has an evaluation value higher than a reference value, in other words, when the evaluation value satisfies the reference value, the image is selected. Data of the image selected by the selection unit 245 is sent to the image processing engine 30 via the output unit 270.

[0046] The process performed by the calculation unit 240A will be described in detail with reference to Fig. 8. Fig. 8(a) to Fig. 8(e) are diagrams illustrating images captured by pixels 10 having photoelectric conversion units for generating images. Fig. 8(a) is an example of an image captured before an instruction to capture for recording is given, in other words, during a half-press operation. Fig. 8(b) to Fig. 8(e) are examples of images captured while an instruction to capture for recording is continuing, in other words, during a full-press operation.

[0047] 8(f) to 8(i) are diagrams each showing a schematic diagram of the image shift amount of the pair of images acquired by a pixel 10 having a photoelectric conversion unit for focus detection when capturing the images shown in Fig. 8(b) to 8(e), respectively. The calculation unit 240A calculates the image shift amount of the pair of images in the same manner as the calculation unit 240 of the first embodiment.

[0048] In Fig. 8(a), a user presses the release button halfway to prepare to use the imaging device 1 to capture an image of an object 70 suspended by a string. As in the first embodiment, the user holds the imaging device 1 so that the shaking object 70 is located approximately in the center of the shooting range. When the user presses the button all the way down, the imaging device 1 captures a number of images, as exemplified in Figs. 8(b) to 8(e), using the imaging element 3A while the button is being pressed all the way down. In this embodiment, four images are captured.

[0049] Data of the first image captured by the imaging element 3A during the full press operation is stored in a predetermined area of ​​the memory 250. The image data stored in the memory 250 includes an image captured by the pixel 10 having a photoelectric conversion unit for image generation and the pair of images acquired by the pixel 10 having a photoelectric conversion unit for focus detection. The selection unit 245 compares the evaluation value calculated by the calculation unit 244 for the first image with a reference value recorded in the evaluation value storage unit 280.

[0050] In the first image shown in Fig. 8(b), the object 70 is moving away in the depth direction. Therefore, the pair of images corresponding to the first image are relatively far apart, as shown in Fig. 8(f). For example, if the reference value is set to be equal to the image shift amount of the pair of images shown in Fig. 8(g), it can be said that the evaluation value for the first image (Fig. 8(b)) does not satisfy the reference value.

[0051] The selection unit 245 does not select the first image shown in FIG. 8(b) because the image shift amount of the pair of images shown in FIG. 8(f) is greater than the image shift amount corresponding to the reference value (in other words, the evaluation value does not satisfy the reference value). The calculation unit 240A makes the data of the first image erasable. The data of the first image is made erasable in order to overwrite the data of the second image in the same area of ​​the memory 250.

[0052] Data of the second image captured by the imaging element 3A during the full press operation is stored in the same area of ​​the memory 250 in which the data of the erasable image was stored. The selection unit 245 compares the evaluation value for the second image with a reference value recorded in the evaluation value storage unit 280. As shown in FIG. 8(g), the pair of images corresponding to the second image (FIG. 8(c)) are relatively consistent (the amount of deviation is almost zero). When the reference value is set to be equal to the amount of image deviation of the pair of images shown in FIG. 8(g), it can be said that the evaluation value for the second image (FIG. 8(c)) satisfies the reference value.

[0053] If the image shift amount of the pair of images illustrated in FIG. 8(g) is smaller than or equal to the image shift amount corresponding to the reference value (in other words, the evaluation value meets the reference value), the selection unit 245 selects the second image illustrated in FIG. 8(c). The calculation unit 240A sets the data of the second image selected by the selection unit 245 as the output target. After sending the data of the image set as the output target to the image processing engine 30, the calculation unit 240A sets the data of the second image illustrated in FIG. 8(c) as erasable.

[0054] The data of the third image captured by the imaging element 3A during the full press operation is stored in the same area of ​​the memory 250 in which the data of the erasable image was stored. The selection unit 245 compares the evaluation value for the third image with the reference value recorded in the evaluation value storage unit 280.

[0055] In the third image shown in Fig. 8(d), the object 70 is approaching toward the viewer. Therefore, the pair of images corresponding to the third image are relatively far apart, as shown in Fig. 8(h). If the reference value is set to be equal to the image shift amount of the pair of images shown in Fig. 8(g), it can be said that the evaluation value for the third image does not satisfy the reference value.

[0056] The selection unit 245 does not select the third image shown in Fig. 8(d) because the image shift amount of the pair of images shown in Fig. 8(h) is greater than the image shift amount corresponding to the reference value (in other words, the evaluation value does not satisfy the reference value). The calculation unit 240A makes the data of the third image erasable.

[0057] The data of the fourth image captured by the imaging element 3A during the full press operation is stored in the same area of ​​the memory 250 in which the data of the erasable image was stored. The selection unit 245 compares the evaluation value for the fourth image with a reference value recorded in the evaluation value storage unit 280. As shown in FIG. 8(i) as an example, the pair of images corresponding to the fourth image (FIG. 8(d)) are relatively consistent (the amount of deviation is almost zero). If the reference value is set to be equal to the amount of image deviation of the pair of images shown in FIG. 8(g), it can be said that the evaluation value for the fourth image (FIG. 8(d)) satisfies the reference value.

[0058] If the image shift amount of the pair of images illustrated in FIG. 8(i) is smaller than or equal to the image shift amount corresponding to the reference value (in other words, the evaluation value meets the reference value), the selection unit 245 selects the fourth image illustrated in FIG. 8(d) as an example. The calculation unit 240A sets the data of the fourth image selected by the selection unit 245 as an output target. After sending the data of the image set as an output target to the image processing engine 30, the calculation unit 240A sets the data of the fourth image illustrated in FIG. 8(d) as erasable.

[0059] As described above, when data of a captured image is stored in a predetermined area of ​​memory 250, calculation section 240A of image sensor 3A calculates an evaluation value for that image (amount of image shift between the pair of images) using calculation section 244. Then, if the calculated evaluation value meets a reference value recorded in evaluation value storage section 280, selection section 245 selects that image. Only data of the image selected by selection section 245 is sent to image processing engine 30 via output section 270.

[0060] The image processing engine 30 receives the image data output from the imaging element 3 A, and performs a predetermined image processing in the image processing unit 330 to generate image data for recording.

[0061] According to the second embodiment described above, the following advantageous effects can be obtained. (1) The imaging element 3A includes a plurality of pixels 10 that output signals based on photoelectrically converted charges, a calculation unit 244 that calculates an evaluation value of an image generated based on the above signals, a selection unit 245 that selects at least one image from the plurality of images based on the evaluation values ​​of each of the plurality of images calculated by the calculation unit 244, and an output unit 270 that outputs data of the image selected by the selection unit 245. With this configuration, it is possible to reduce the amount of image data output from the imaging element 3A compared to when data for all of the multiple images captured by the imaging element 3A is output from the imaging element 3A to the image processing engine 30. This makes it possible to reduce the processing time required to output data and the power consumption in the imaging element 3A.

[0062] (2) The selection unit 245 of the imaging element 3A selects an image from among the multiple images whose evaluation value exceeds a predetermined value. With this configuration, the amount of image data output from the imaging element 3A can be reduced compared to when the data of all images is output. Also, the data of images from among the multiple images whose evaluation value is higher than a predetermined value (in other words, satisfying a reference value) can be output to the image processing engine 30.

[0063] (3) The multiple pixels 10 of the image sensor 3A photoelectrically convert the light that has passed through the photographing optical system 2, and the calculation unit 244 calculates an evaluation value that indicates the state of focus adjustment of the photographing optical system 2. With this configuration, it is possible to output to the image processing engine 30 data on images that are in focus to a certain extent among a plurality of images.

[0064] (4) The calculation unit 244 of the image sensor 3A calculates an evaluation value based on the signal that corresponds to a predetermined region 60 of the image, out of the above signals. With this configuration, the burden of the calculation unit 244 in calculating the evaluation value can be reduced compared to when the evaluation value is calculated based on signals corresponding to the entire shooting range 50.

[0065] (Third embodiment) In the third embodiment, from among a plurality of images captured by the imaging element 3 during a full press operation, data of an image in which the subject in the image has little movement is sent to the image processing engine 30. The number of images sent to the image processing engine 30 is not limited to one. Such a configuration will be described below.

[0066] <Image sensor and image processing engine> 9 is a schematic diagram for explaining data transfer between an image sensor 3B according to the third embodiment and an image processing engine 30. The image sensor 3B is used in place of the image sensor 3 according to the first embodiment or the image sensor 3A according to the second embodiment. The image sensor 3B differs from the image sensors 3 and 3A in that it has a calculation unit 240B. Therefore, the following explanation will focus on these differences, and the same components as those in the image sensors 3 and 3A are denoted by the same reference numerals as those in FIGS. 4 and 7 and will not be explained further.

[0067] For example, when an instruction to shoot for recording is input from image processing engine 30, image sensor 3B captures a plurality of images while the shooting instruction is continuing, and sends data of an image among the plurality of images in which the main subject has little movement, in other words, in which it is estimated that image blur of the main subject is small, to image processing engine 30 as data of an image for recording. In addition, when an instruction to capture an image for monitor display is input from the image processing engine 30, the image capture device 31 repeatedly captures images for monitor display and sends data of the captured images to the image processing engine 30 as data of images for monitor display, similar to the case of the image sensors 3 and 3A.

[0068] <Detailed explanation of the calculation unit> In the third embodiment, an evaluation value indicating the magnitude of movement of the main subject is calculated by calculation section 240B of image sensor 3B. This evaluation value is based on the difference in data between images captured one after the other among a plurality of images.

[0069] The calculation unit 240B calculates an evaluation value for each of the multiple images captured while the instruction to capture for recording is continuing. The calculation of the evaluation value is performed by the calculation unit 244, which will be described later with reference to FIG. 11. The calculation unit 240B compares the evaluation value calculated by the calculation unit 244 with a reference value previously recorded in the evaluation value storage unit 280. The comparison of the evaluation values ​​is performed by the selection unit 245, which will be described later with reference to FIG. 11. The selection unit 245 selects an image whose calculated evaluation value satisfies the reference value, in other words, whose evaluation value is smaller than the reference value. Data of the image selected by the selection unit 245 is sent to the image processing engine 30 via the output unit 270.

[0070] The process performed by the calculation unit 240B will be described in detail with reference to Fig. 10. Fig. 10(a) to Fig. 8(e) are diagrams illustrating images captured by pixels 10 having photoelectric conversion units for generating images. Fig. 10(a) is an example of an image captured before an instruction to capture for recording is given, in other words, during a half-press operation. Fig. 10(b) to Fig. 10(e) are examples of images captured while an instruction to capture for recording is continuing, in other words, during a full-press operation.

[0071] Fig. 10(f) is a diagram showing a typical difference in data between the images shown in Fig. 10(b) and Fig. 10(c) captured during a full press operation. Fig. 10(g) is a diagram showing a typical difference in data between the images shown in Fig. 10(c) and Fig. 10(d) captured during a full press operation. Fig. 10(h) is a diagram showing a typical difference in data between the images shown in Fig. 10(d) and Fig. 10(e) captured during a full press operation.

[0072] In Fig. 10(a), a user presses the release button halfway to prepare to capture a picture of a person 80 as a main subject with the imaging device 1. The user holds the imaging device 1 so that the person 80 is positioned approximately in the center of the shooting range. When the user presses the button all the way down, the imaging device 1 captures a number of images, as exemplified in Figs. 10(b) to 10(e), with the imaging element 3B while the button is being pressed all the way down. In this embodiment, four images are captured.

[0073] 11 is a diagram illustrating an example of the configuration of the calculation unit 240 B. The calculation unit 240 B has a memory reading unit 241 that reads data from the memory 250, a difference calculation unit 242, a cumulative addition unit 243, a calculation unit 244, and a selection unit 245.

[0074] The calculation unit 240B stores data of a first image (FIG. 10(b)) captured by the imaging element 3B during a full press operation, for example, in the second area 252 of the memory 250. The calculation unit 240B stores data of a second image (FIG. 10(b)) captured by the imaging element 3B during a full press operation, for example, in the first area 251 of the memory 250.

[0075] The calculation unit 240B calculates the difference between the data of the second image stored in the first area 251 of the memory 250 and the data of the first image stored in the second area 252 of the memory 250 for each corresponding pixel by the difference calculation unit 242. When such a calculation is performed, as shown in Fig. 10(f), the difference value calculated for the pixels included in the area where the image of the person 80 (main subject) has moved (in other words, the area where the person 80 has moved between the first image and the second image) is large, and the difference value calculated for the pixels included in the area where the image of the person 80 does not move is small. In the embodiment, a region of the subject image where there is movement is set as region 60 for which calculation unit 240B calculates an evaluation value. By setting region 60 as a part of the shooting range, the processing load of calculation unit 240B for calculating an evaluation value can be reduced compared to when region 60 is set to the entire shooting range. Calculation unit 240B can set region 60 automatically, or can set a range designated by a user's operation as region 60.

[0076] The calculation unit 240B adds up the multiple difference values ​​calculated for each pixel included in the region 60 using the cumulative addition unit 243. The calculation unit 244 sets the cumulative addition value calculated by the cumulative addition unit 243 as the evaluation value for the second image. The selection unit 245 compares the evaluation value for the second image with a reference value recorded in the evaluation value storage unit 280.

[0077] For example, if the reference value is set to be equal to the difference illustrated in FIG. 10(h), the evaluation value for the second image (FIG. 10(c)) does not satisfy the reference value. 10(b) is not selected by the selection unit 245 because the difference illustrated in FIG. 10(f) is greater than the difference corresponding to the reference value (in other words, the evaluation value does not satisfy the reference value). The calculation unit 240B makes the data of the first image erasable. The reason for making the data of the first image erasable is to overwrite the data of the second image in the second area 252 of the memory 250.

[0078] The calculation section 240B stores in the first area 251 of the memory 250 the data of the third image (FIG. 10(d)) captured by the imaging element 3B during the full press operation. The calculation unit 240B calculates the difference between the data of the third image stored in the first area 251 of the memory 250 and the data of the second image stored in the second area 252 of the memory 250 for each corresponding pixel using the difference calculation unit 242.

[0079] The calculation unit 240B adds up the multiple difference values ​​calculated for each pixel included in the region 60 using the cumulative addition unit 243. The calculation unit 244 sets the cumulative addition value calculated by the cumulative addition unit 243 as the evaluation value for the third image. The selection unit 245 compares the evaluation value for the third image with a reference value recorded in the evaluation value storage unit 280.

[0080] As described above, if the reference value is set to be equal to the difference exemplified in FIG. 10(h), it can be said that the evaluation value for the third image (FIG. 10(d)) does not satisfy the reference value. The selection unit 245 does not select the second image illustrated in FIG. 10(c) because the difference illustrated in FIG. 10(g) is greater than the difference corresponding to the reference value (in other words, the evaluation value does not satisfy the reference value). The calculation unit 240B makes the data of the second image erasable. The reason for making the data of the second image erasable is to overwrite the data of the third image in the second area 252 of the memory 250.

[0081] In this way, for the second and subsequent images captured by the imaging element 3B during a full press operation, the data of the newly captured image is stored in the first area 251 of the memory 250, while the difference between the data of the image stored in the second area 252 of the memory 250 is calculated by the difference calculation unit 242 for each corresponding pixel.

[0082] The calculation section 240B stores in the first area 251 of the memory 250 the data of the fourth image (FIG. 10(e)) captured by the imaging element 3B during the full press operation. The calculation unit 240B calculates the difference between the data of the fourth image stored in the first area 251 of the memory 250 and the data of the third image stored in the second area 252 of the memory 250 for each corresponding pixel using the difference calculation unit 242.

[0083] The calculation unit 240B adds up the multiple difference values ​​calculated for each pixel included in the region 60 using the cumulative addition unit 243. The calculation unit 244 sets the cumulative addition value calculated by the cumulative addition unit 243 as the evaluation value for the fourth image. The selection unit 245 compares the evaluation value for the fourth image with a reference value recorded in the evaluation value storage unit 280.

[0084] As shown in Fig. 10(h), the data of the third image and the data of the fourth image are relatively consistent (the difference value is approximately 0). If the reference value is set to be equal to the difference value shown in Fig. 10(h), the evaluation value for the fourth image (10(e)) can be said to satisfy the reference value.

[0085] If the difference illustrated in Fig. 10(h) is smaller than or equal to the difference corresponding to the reference value (in other words, the evaluation value meets the reference value), the selection unit 245 selects the fourth image illustrated in Fig. 10(e). The calculation unit 240B sets the data of the fourth image selected by the selection unit 245 as the output target. After sending the data of the image set as the output target to the image processing engine 30, the calculation unit 240B makes the sent data of the fourth image erasable.

[0086] As described above, when a newly captured image is stored in first area 251 of memory 250, calculation section 240B of image sensor 3B calculates an evaluation value for that image (a cumulative sum of differences from a previously captured image) using difference calculation section 242, cumulative addition section 243 and calculation section 244. Then, if the calculated evaluation value meets the reference value recorded in evaluation value storage section 280, selection section 245 selects that image. Only data of the image selected by selection section 245 is sent to image processing engine 30 via output section 270.

[0087] The image processing engine 30 receives the image data output from the imaging element 3 B, and performs a predetermined image processing in the image processing unit 330 to generate image data for recording.

[0088] According to the third embodiment described above, the following advantageous effects can be obtained. (1) The imaging element 3B includes a plurality of pixels 10 that output signals based on photoelectrically converted charges, a calculation unit 244 that calculates an evaluation value of an image generated based on the above signal, a selection unit 245 that selects at least one image from the plurality of images based on the evaluation values ​​of each of the plurality of images calculated by the calculation unit 244, and an output unit 270 that outputs an image signal of the image selected by the selection unit 245. With this configuration, it is possible to reduce the amount of image data output from the imaging element 3B compared to a case where data for all of the multiple images captured by the imaging element 3B is output from the imaging element 3B to the image processing engine 30. This makes it possible to reduce the processing time required to output data and the power consumption in the imaging element 3B.

[0089] (2) The calculation unit 244 of the imaging element 3B calculates an evaluation value based on the difference between the images of the current frame and the previous frame, and the selection unit 245 selects from among the multiple images an image whose evaluation value is less than a predetermined value. With this configuration, it is possible to reduce the amount of image data output from the imaging element 3B compared to a case where data for all of the multiple images captured by the imaging element 3B is output from the imaging element 3B to the image processing engine 30. This makes it possible to reduce the processing time required to output data and the power consumption in the imaging element 3B.

[0090] (3) The calculation unit 244 of the image sensor 3B calculates an evaluation value indicating the movement of a subject (e.g., person 80) appearing in an image. With this configuration, for example, data of an image in which image blur of person 80 is minimal among a plurality of images can be output to the image processing engine 30.

[0091] (4) The calculation unit 244 of the image sensor 3B calculates an evaluation value based on the signal that corresponds to a predetermined region 60 of the image, out of the above signals. With this configuration, the burden of the calculation unit 244 in calculating the evaluation value can be reduced compared to when the evaluation value is calculated based on signals corresponding to the entire shooting range 50.

[0092] (Variation 1) In the above-described first to third embodiments, the imaging elements 3, 3A, and 3B are configured as a back-illuminated type. Alternatively, the imaging elements 3, 3A, and 3B may be configured as a front-illuminated type in which the wiring layer 140 is provided on the light-incident surface side.

[0093] (Variation 2) In the above-described first to third embodiments, the photodiode is used as the photoelectric conversion section, but a photoelectric conversion film may be used as the photoelectric conversion section.

[0094] (Variation 3) The imaging elements 3, 3A, and 3B may be applied to cameras, smartphones, tablets, cameras built into PCs, car-mounted cameras, and the like.

[0095] (Variation 4) In the above-mentioned first to third embodiments, an example has been described in which an evaluation value indicating the state of focus adjustment of the photographing optical system 2 or an evaluation value indicating the magnitude of movement of the main subject is calculated. As an evaluation value indicating information other than the state of focus adjustment and the magnitude of movement of the main subject, for example, an evaluation value indicating the brightness of the photographed image, an evaluation value indicating the color temperature (which may be called white balance) of the photographed image, or an evaluation value indicating the contrast of the photographed image may be calculated. The selection unit selects an image to be sent to the image processing engine 30 based on the calculated evaluation value.

[0096] (Variation 5) In the above-described first to third embodiments, for example, a case has been described in which a plurality of images are captured by the imaging element 3, 3A or 3B while the release button is continuously pressed all the way down, and an image to be sent from the imaging element 3, 3A or 3B to the image processing engine 30 is selected from the plurality of images based on an evaluation value calculated for each image. The selection of an image to be sent from the imaging element 3, 3A or 3B to the image processing engine 30 from the plurality of images is not limited to a case in which the release button is continuously pressed all the way down, but may also be applied to a case in which the imaging device 1 automatically starts shooting, such as when a shooting setting is set in the imaging device 1 to start shooting a plurality of images at a predetermined time, when a shooting setting is set in the imaging device 1 to start shooting a plurality of images when a subject moves into a predetermined area in the shooting range 50, or when an instruction to start shooting is sent from an external device to the imaging device 1.

[0097] (Variation 6) In the above-mentioned first to third embodiments, it has been described that the same shooting conditions are applied to the multiple images captured by the imaging element 3, 3A or 3B without changing the shooting conditions. The shooting conditions are, for example, shutter speed, aperture value, ISO sensitivity, etc. Instead, when capturing multiple images by the imaging element 3, 3A or 3B, bracket shooting may be performed in which the images are captured while at least a part of the shooting conditions is changed in stages. In the sixth modification, the image to be sent from the imaging element 3, 3A or 3B to the image processing engine 30 is selected from the multiple images captured by bracket shooting.

[0098] (Variation 7) As a modification of the third embodiment described above, a third area may be reserved in memory 250 described in the third embodiment, and data of an image newly captured by image sensor 3B (referred to as data A) may be stored in the third area. In modification 7, first area 251 and second area 252 of memory 250 store data of a thinned image (referred to as data A-) in which the number of data items has been reduced by thinning out data A at a predetermined thinning rate, and data of a thinned image (referred to as data B-) in which the number of data items has been reduced by thinning out data of an image captured by image sensor 3B before data A (referred to as data B) at the same thinning rate as data A, respectively.

[0099] The calculation unit 240B calculates an evaluation value (a value obtained by accumulating the differences in data) for an image newly captured by the imaging element 3B (the image data of which is stored in the third area of ​​the memory 250) based on the data A- and data B- stored in the first area 251 and second area of ​​the memory 250, and if the calculated evaluation value meets the reference value recorded in the evaluation value memory unit 280, it sends the data of the image stored in the third area of ​​the memory 250 to the image processing engine 30.

[0100] In the seventh modification described above, the thinning rate of image data stored in the first region 251 and the second region 252 is set so that the sum of the storage capacities of the first region 251 and the second region 252 of the memory 250 is smaller than the storage capacity of the third region of the memory 250. By configuring in this way, it is possible to keep the storage capacity of the memory 250 (the sum of the first region 251, the second region 252, and the third region) required in the seventh modification smaller than the storage capacity of the memory 250 (the sum of the first region 251 and the second region 252) required in the third embodiment.

[0101] Although various embodiments and modifications have been described above, the present invention is not limited to these. Aspects in which the configurations shown in the embodiments and modifications are used in combination are also included within the scope of the present invention. Other aspects that are conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0102] 1...imaging device, 3, 3A, 3B...imaging element, 4...control unit, 10...pixel, 20...reading unit, 30...image processing engine, 60...area, 210...pixel array, 240, 240A, 240B...arithmetic unit, 244...calculation unit, 245...selection unit, 250...memory, 251...first area, 252...second area, 260...scanning unit, 270...output unit, 280...evaluation value storage unit

Claims

[Claim 1] a first substrate having a plurality of pixels that output signals based on photoelectrically converted charges; a second substrate having a first conversion unit that converts, into digital signals, a first signal output from at least a first pixel among the plurality of pixels and a second signal output from the first pixel after the first signal; a third substrate having a calculation unit configured to control to output at least one of the first digital signal and the second digital signal based on a first evaluation value obtained by using a first digital signal converted from the first signal by the first conversion unit and a second evaluation value obtained by using a second digital signal converted from the second signal by the first conversion unit; An imaging element comprising:

Images