Image processing apparatus and imaging apparatus

Abstract

To facilitate the use of an imaging element having a high reading speed.SOLUTION: An image signal processing device includes an original image signal reception unit that receives an original image signal including a plurality of frames from an imaging unit, a transmission image signal generation unit that generates a transmission image signal including a smaller number of frames than a number of the plurality of frames included in the original image signal received by the original image signal reception unit with reference to the original image signal, and a transmission image signal transmission unit that transmits the transmission image signal generated by the transmission image signal generation unit to an external image processing circuit via a signal transmission path. In the image signal processing device, the transmission image signal generation unit may generate the transmission image signal by executing noise suppression processing on the original image signal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device and an imaging device. [Background technology]

[0002] As the resolution of an imaging unit that generates an image signal improves, the amount of image signals transmitted from the imaging unit to an image processing circuit also increases (see, for example, Patent Document 1). Patent Document 1: Japanese Patent Application Laid-Open No. 2008-219319 Summary of the Invention [Problem to be solved by the invention]

[0003] To increase the capacity of the signal transmission path by parallelization, both the imaging unit and the image processing circuit must be made compatible with parallel signal transmission, which requires significant changes to the components of the imaging device. [Means for solving the problem]

[0004] In a first aspect of the present invention, there is provided an image signal processing device comprising: an original image signal receiving unit that receives an original image signal including a plurality of frames from an imaging unit; a transmission image signal generating unit that references the original image signal and generates a transmission image signal including a number of frames that is fewer than the number of frames included in the original image signal received by the original image signal receiving unit; and a transmission image signal transmitting unit that transmits the transmission image signal generated by the transmission image signal generating unit to an external image processing circuit via a signal transmission path.

[0005] In a second aspect of the present invention, there is provided an image signal processing unit comprising an imaging section that generates an original image signal for each exposure, an original image signal receiving section that receives an original image signal including a plurality of frames from the imaging section, a transmission image signal generating section that references the original image signal and generates a transmission image signal including a number of frames that is fewer than the number of frames included in the original image signal received by the original image signal receiving section, and a transmission image signal transmitting section that transmits the transmission image signal generated by the transmission image signal generating section to an external image processing circuit via a signal transmission path.

[0006] In a third aspect of the present invention, there is provided an imaging device including the image signal processing unit described above.

[0007] The above summary of the invention does not list all of the necessary features of the present invention. Subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of an imaging device 100. [Figure 2] FIG. 2 is a block diagram of a transmission image signal generation unit 201. [Figure 3] 10 is a flowchart showing a control procedure of a switching circuit 221. [Figure 4] 1 is a schematic diagram illustrating an example of an implementation of an image signal processing device 200. FIG. [Figure 5] 1 is a schematic diagram illustrating an example of an implementation of an image signal processing device 200. FIG. [Figure 6] 1 is a schematic cross-sectional view illustrating an example of an implementation of an image signal processing device 200. FIG. [Figure 7] 1 is a schematic cross-sectional view illustrating an example of an implementation of an image signal processing device 200. FIG. [Figure 8] FIG. 2 is a block diagram of a transmission image signal generation unit 202. [Figure 9] FIG. 2 is a block diagram of a transmission image signal generation unit 203. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will be described through the embodiments of the invention. The following embodiments do not limit the scope of the invention. Not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] 1 is a block diagram of an imaging device 100. The imaging device 100 includes an imaging unit 120, an image signal processing device 200, a system control unit 140, and an image processing unit 150.

[0011] The imaging unit 120 includes an imaging element that receives incident field light through the optical system 110 and converts it into an electrical signal, and generates an original image signal corresponding to a frame generated each time the imaging element is exposed. The optical system 110 includes optical components such as a diaphragm, a shutter, an optical filter, etc. in addition to multiple lenses.

[0012] The imaging element of the imaging unit 120 can be a CMOS sensor, a CCD sensor, or the like. The imaging element can have multiple A / D converters or the like inside the device to perform parallel readout and output image signals at high resolution and a high frame rate. If the imaging element of the imaging unit 120 has a color filter such as a Bayer array, the original image signal output by the imaging unit 120 may be an image signal before pixels are interpolated for each color.

[0013] The image signal processing device 200 has an original image signal receiving unit 210, a transmission image signal generating unit 201, a frame memory 230, and a transmission image signal transmitting unit 240. The original image signal receiving unit 210 acquires an original image signal generated by the imaging unit 120 and including a plurality of frames.

[0014] If the imaging element has, for example, a 20 megapixel resolution and a readout speed of 240 FPS, the output bit rate of the original image signal output from the imaging unit 120 to the image signal processing device 200 will be as high as 57.6 Gbps or more. In contrast, the original image signal receiving unit 210 in the image signal processing device 200 is connected to the imaging unit 120 via, for example, a 48-channel multi-parallel signal line. This reduces the signal speed per channel in the signal transmission line, and the imaging unit 120 and the original image signal receiving unit 210 can be connected by a flexible cable.

[0015] The transmission image signal generation unit 201 processes the acquired original image signal to generate a transmission image signal. The transmission image signal includes an image signal in units of frames. The processing in the transmission image signal generation unit 201 will be described later with reference to other figures.

[0016] The frame memory 230 includes a DRAM that can be read and written at any time, and serves as a work area for the transmission image signal generation unit 201. Therefore, the frame memory 230 has a capacity that can store at least a plurality of frames of image signals.

[0017] The transmission image signal transmitting unit 240 transmits the transmission image signal generated by the transmission image signal generating unit 201 to the image processing unit 150 disposed outside the image signal processing device 200. Note that the original image signal receiving unit 210, the transmission image signal generating unit 201, and the transmission image signal transmitting unit 240 may be implemented as programs executed in a common general-purpose processor.

[0018] The transmission image signal output by the transmission image signal transmitting unit 240 is transmitted in a signal format that can be received by the image processing unit 150 through a signal transmission path 130 arranged externally to the image signal processing device 200. The signal transmission path 130 is formed by, for example, a flexible substrate that connects the image signal processing device 200 and the image processing unit 150, a printed wiring on a printed circuit board, a wire, or the like.

[0019] The image processing unit 150 in the imaging device 100 performs processes such as interpolation, compression, and reduction of the image signal. The image processing unit 150 also converts the processed image signal into an image file in accordance with a predetermined file format such as JPEG. As a result, the processed transmitted image signal is displayed, for example, as a through image on the display unit 180. The transmitted image signal processed by the image processing unit 150 is also stored as an image file on the secondary recording medium 190.

[0020] The work memory 160 includes a DRAM that can be read and written at any time, and serves as a work area for the image processing unit 150. Therefore, the work memory 160 has a capacity that can store at least a plurality of frames of image signals.

[0021] The system control unit 140 comprehensively controls the operations of the imaging unit 120, the image signal processing device 200, and the image processing unit 150. That is, when an operating member 170 such as a shutter button or a command dial is operated from the outside, the system control unit 140 causes the imaging unit 120 to perform an operation corresponding to the operation, for example, by changing the frame rate of the original image signal in the imaging unit 120.

[0022] The system control unit 140 also controls communication such as the transfer of original image signals from the imaging unit 120 to the image signal processing device 200 and the transfer of transmission image signals from the image signal processing device 200 to the image processing unit 150. Furthermore, internal processing in each of the image signal processing device 200 and the image processing unit 150 is also executed under the control of the system control unit 140. Note that the system control unit 140 and the image processing unit 150 may be implemented as a program executed in a common ASIC.

[0023] 2 is a block diagram showing an example of the transmission image signal generation unit 201. The transmission image signal generation unit 201 has a switching circuit 221, a bypass circuit 222, and a main circuit 223. Furthermore, the main circuit 223 includes a resizing circuit 224, an alignment detection circuit 225, an alignment adjustment circuit 226, and a frame synthesis circuit 227.

[0024] In response to an instruction from the system control unit 140, the switching circuit 221 selectively couples an image signal including multiple frames, for example, two frames 301 and 302, received by the original image signal receiving unit 210 to either the bypass circuit 222 or the main circuit 223. When the input is coupled to the bypass circuit 222, the original image signal is directly connected to the transmission image signal transmitting unit 240 via the bypass circuit 222. Therefore, in this case, the original image signal received by the original image signal receiving unit 210 is transmitted to the transmission image signal transmitting unit 240 as is.

[0025] When the switching circuit 221 couples the input to this circuit 223, the original image signal is transmitted to both the resizing circuit 224 and the alignment adjustment circuit 226. The resizing circuit 224 reduces the resolution of the original image signal to the extent that the alignment detection circuit 225 can detect misalignment of images between the frames 301 and 302. This allows the alignment detection circuit 225 to quickly detect misalignment of images between frames in the original image signal.

[0026] The alignment detection circuit 225 detects misalignment of images between frames and notifies the alignment adjustment circuit 226 of the detection result. The alignment adjustment circuit 226 adjusts the position of the image for each frame based on the detection result obtained from the alignment detection circuit 225, thereby eliminating the misalignment between frames. The frame synthesis circuit 227 synthesizes multiple frames in which the misalignment has been eliminated, thereby generating an image signal of a single image with noise suppressed. Here, an arithmetic averaging method can be given as an example of a method for synthesizing image signals, but is not limited to this method.

[0027] The single-frame image signal thus generated is transmitted as a transmission image signal to the transmission image signal transmitting unit 240. The transmission image signal transmitting unit 240 transmits the transmission image signal for one frame generated from the original image signal for two frames to the image processing unit 150 via the signal transmission path 130.

[0028] In this way, the image signal processing device 200 references a plurality of frames of original image signals, generates a transmission image signal with a smaller number of frames, and transmits it to the image processing unit 150. This makes it possible to reduce the signal speed in the signal transmission path 130, and even when an imaging element having a readout speed higher than the transmission speed of the signal transmission path 130 and the processing speed of the image processing unit 150 is introduced, the existing signal transmission path 130 and image processing unit 150 can be used.

[0029] Furthermore, since noise suppression processing has already been performed in the transmission image signal generation unit 201, the image processing load in the image processing unit 150 can be reduced. Note that the transmission image signal generated from multiple frames of original image signals does not have the number of pixels, bit depth, etc. reduced or suppressed. Therefore, the transmission image signal maintains the quality of the original image signal captured by the imaging unit 120.

[0030] In the above example, it is described that the transmission image signal generation unit 201 generates one frame 303 from the image signals of two frames 301 and 302. However, the transmission image signal generation unit 201 only needs to receive an original image signal including N (N is an integer of 2 or more) frames that are consecutive in time from the imaging element and generate a transmission image signal including M (M is an integer of 1 or more) frames, which is less than N, and is not limited to the above number of frames.

[0031] 3 is a flowchart showing the control procedure of the switching circuit 221 by the system control unit 140. The system control unit 140 executes control to change the readout speed of the image sensor in the imaging unit 120 according to the operation mode of the imaging device 100. The operation mode of the imaging device 100 includes an operation mode selected in advance by the user, such as a live view mode, a still image shooting mode, or a video shooting mode. The system control unit 140 instructs the imaging unit 120 on the frame rate of the original image signal to be output according to the set operation mode.

[0032] If the frame rate specified by the system control unit 140 is equal to or lower than a preset threshold (step S101: NO), the system control unit 140 switches the switching circuit 221 to the bypass circuit 222 side (step S102). As a result, the original image signal received by the original image signal receiving unit 210 is transmitted as is to the transmitted image signal transmitting unit 240 and transmitted to the image processing unit 150.

[0033] Furthermore, if the readout speed set by the system control unit 140 exceeds a preset threshold (step S101: YES), the system control unit 140 switches the switching circuit 221 to the main circuit 223 side (step S103). As a result, the transmission image signal generation unit 201 generates a transmission image signal with a smaller number of frames from the multiple frames of original image signals received by the original image signal reception unit 210, and transmits the transmission image signal from the transmission image signal transmission unit 240 to the image processing unit 150.

[0034] Here, the threshold value at which the system control unit 140 switches the switching circuit 221 is determined, for example, by the limit of the frame rate of the transmission image signal transmitted through the signal transmission path 130. In other words, the frame rate of the original image signal output by the imaging unit 120 changes under the control of the system control unit 140.

[0035] When the frame rate of the original image signal output by the imaging unit 120 is faster than the speed that can be transmitted through the signal transmission path 130 formed using a flexible substrate or the like, the system control unit 140 switches the switching circuit 221 to the side of this circuit 223. As a result, the frame rate of the transmission image signal transmitted through the signal transmission path 130 is reduced to the frame rate that can be transmitted through the signal transmission path 130.

[0036] If the frame rate of the original image signal output by the imaging unit 120 is slower than the speed that can be transmitted through the signal transmission path 130 formed using a flexible substrate or the like, the system control unit 140 switches the switching circuit 221 to the bypass circuit 222. This causes the transmission image signal generation unit 201 to send the original image signal as is to the signal transmission path 130 as a transmission image signal without processing it.

[0037] Furthermore, the threshold value at which the system control unit 140 switches the switching circuit 221 may be determined according to the upper limit of the frame rate that can be processed by the image processing unit 150. As a result, regardless of the frame rate of the original image signal output by the imaging unit 120, a transmission image signal with a frame rate within the processable range is transmitted to the image processing unit 150.

[0038] When the transmission image signal generation unit 201 does not process the original image signal, the frame memory 230 is not used. Therefore, the frame memory 230 may be used as a buffer memory for storing still images captured during video capture.

[0039] 4 is a schematic diagram illustrating an example of the implementation of the imaging unit 120 and the image signal processing device 200. In the illustrated example, the board 250 on which the image signal processing device 200 including the transmission image signal generation unit 201 is mounted is disposed immediately adjacent to the board 121 on which the imaging unit 120 including the image sensor 122 is mounted. This allows the dimensions of the flexible cable 124 that forms multiple parallel signal lines and connects the pair of boards 121 and 250 to be shortened, thereby suppressing attenuation of the transmitted original image signal.

[0040] The transmission image signal output from the image signal processing device 200 is also transmitted via the flexible cable 131. However, the location where the imaging unit 120 including the image sensor 122 can be arranged is limited by optical conditions for the optical system 110. On the other hand, the image processing unit 150 has a large circuit scale, and its arrangement in the imaging device 100 is limited. For this reason, the length of the flexible cable 131 forming the signal transmission path 130 coupled to the image processing unit 150 is often longer than the flexible cable 124 coupled to the imaging unit 120.

[0041] However, by using the transmission image signal generation unit 201, for example, it is possible to process a noise-suppressed 60 FPS signal using an imaging element having a readout speed of 240 FPS, using a signal transmission path 130 that is not multi-parallelized, and an image processing circuit that does not have a multi-parallel I / O circuit. Also, since the transmission image signal received by the image processing unit 150 has already undergone some image processing in the image signal processing device 200, the load on the image processing unit 150 is reduced.

[0042] In the above example, the substrates 121 and 250 are connected by a flexible cable, but it goes without saying that they may be connected by using other members such as a printed circuit board or wire.

[0043] 5 is a schematic diagram illustrating another implementation form of the imaging unit 120 and the image signal processing device 200. In the illustrated example, the image signal processing device 200 is mounted on a substrate 121 on which the imaging unit 120 including the image sensor 122 is mounted. This allows the imaging unit 120 and the original image signal receiving unit 210 to be connected without using a flexible cable, thereby suppressing attenuation of the original image signal.

[0044] Furthermore, by handling the imaging unit 120 and the image signal processing device 200 as an integrated unit, the imaging unit 120 having a high readout speed can be handled in the same way as a conventional imaging unit 120 having a low readout speed. This allows the advantages of the imaging unit 120 having a high readout speed to be utilized at low development costs.

[0045] 6 is a schematic cross-sectional view illustrating another implementation form of the imaging unit 120 and the image signal processing device 200. In the illustrated example, a substrate 250 on which the image signal processing device 200 is mounted is stacked on a substrate 121 on which the imaging unit 120 including the image sensor 122 is mounted. In addition, the image signal processing device 200 on the substrate 250 is electrically coupled to the imaging unit 120 through a plurality of vias 126 formed through the substrate 121. This minimizes the signal transmission path between the imaging unit 120 and the original image signal receiving unit 210, further suppressing attenuation of the original image signal.

[0046] Furthermore, by stacking the substrate 250 on the substrate 121, an imaging unit including the imaging section 120 and the image signal processing device 200 can be mounted in a space that is almost the same as that of a single imaging section 120. Therefore, the imaging section 120 having a high readout speed can be handled in the same way as a conventional imaging section 120, without increasing the scale of the imaging device 100.

[0047] In the above example, the substrates 121 and 250 are connected by the via 126. However, it goes without saying that other members such as bonding wires and bumps may also be used for connection.

[0048] 7 is a schematic cross-sectional view illustrating yet another implementation form of the imaging unit 120 and the image signal processing device 200. In the illustrated example, a substrate 250 on which the image signal processing device 200 is mounted is stacked on a back-illuminated imaging unit 120.

[0049] That is, the imaging unit 120 has a light receiving substrate 123 and a logic substrate 125 stacked on top of each other. The light receiving substrate 123 receives field light from the rear surface of the substrate 121, which has been thinned by polishing, and converts the light into an electrical signal. The logic substrate 125 is equipped with a logic circuit 128, such as an analog / digital converter, that digitizes the electrical signal generated on the light receiving substrate 123, and generates an original image signal.

[0050] The light-receiving substrate 123 and the logic substrate 125 are fabricated separately and then laminated and bonded together. The image sensor 122 of the light-receiving substrate 123 and the logic circuit 128 of the logic substrate 125 are coupled through vias 126 or the like formed to penetrate the light-receiving substrate 123 in the thickness direction.

[0051] Substrate 250 on which image signal processing device 200 is formed is fabricated separately from light receiving substrate 123 and logic board 125 of imaging unit 120 described above, and is further laminated and attached to logic board 125 of imaging unit 120. Image signal processing device 200 on substrate 250 is electrically coupled to logic circuit 128 mounted on logic board 125 through a plurality of vias 127 and the like formed to penetrate logic board 125 in the thickness direction. With this structure, the signal transmission path from image sensor 122 through logic circuit 128 to original image signal receiving unit 210 is shortened to the utmost, suppressing attenuation of the original image signal.

[0052] In the above example, the light receiving substrate 123, the logic substrate 125, and the substrate 250 are connected by the vias 126. However, it goes without saying that other members such as bonding wires and bumps may also be used for connection.

[0053] 8 is a block diagram of another transmission image signal generation unit 202 that can be used in place of the transmission image signal generation unit 201. The transmission image signal generation unit 202 has a switching circuit 221, a bypass circuit 222, and a main circuit 223. Furthermore, the main circuit 223 includes a synthesis map generation circuit 261, a frame synthesis circuit 262, a moving object region extraction circuit 263, and a moving object region replacement circuit 264, and executes so-called HDR synthesis (High Dynamic Range Imaging) processing.

[0054] In response to an instruction from the system control unit 140, the switching circuit 221 selectively couples the original image signal including a plurality of frames 311, 312, and 313 received by the original image signal receiving unit 210 to either the bypass circuit 222 or the main circuit 223. The selection of the coupling destination by the switching circuit 221 is determined by the frame rate of the original image signal received by the original image signal receiving unit 210, as in the case of the transmission image signal generating unit 201 described above.

[0055] That is, when the frame rate of the original image signal is within the range that can be transmitted in the band of the signal transmission path 130 and is also within the range that can be processed by the image processing unit 150, the switching circuit 221 connects the original image signal receiving unit 210 to the bypass circuit 222. As a result, the original image signal is transmitted as is from the transmission image signal transmitting unit 240 as a transmission image signal.

[0056] Furthermore, if the frame rate of the original image signal exceeds the range that can be transmitted through the bandwidth of the signal transmission path 130, or exceeds the range that can be processed by the image processing unit 150, the switching circuit 221 connects the original image signal receiving unit 210 to this circuit 223. As a result, the original image signal is transmitted to both the composite map generation circuit 261 and the moving object region extraction circuit 263.

[0057] The composite map generation circuit 261 calculates the proportions at which to combine multiple frames 311, 312, and 313 in order to make the most of the dynamic range of the display device. The frame composition circuit 262 combines the multiple frames 311, 312, and 313 in accordance with the proportions calculated by the composite map generation circuit 261 to generate a single frame 314. In this way, a single frame 314 is generated that is expressed by effectively utilizing the dynamic range.

[0058] However, when the brightness of images of a subject that has moved between multiple frames 311, 312, and 313 is combined, the contours become blurred. Therefore, the moving object region extraction circuit 263 extracts a region including the subject that has moved between the multiple frames 311, 312, and 313, and the moving object region replacement circuit 264 replaces the moving object region with a clear image in the generated frame 314. As a result, the frame 314 effectively utilizes the dynamic range and becomes an image with a clear contour of the subject.

[0059] In this way, the image signal processing device 200 equipped with the transmission image signal generation unit 202 synthesizes multiple frames, each captured under different exposure conditions, to synthesize a single image that makes the most of the dynamic range of the display device. Therefore, the transmission image signal generation unit 202 that performs HDR synthesis processing can generate a transmission image signal with a smaller number of frames from multiple frames of original image signals and send it to the image processing unit 150.

[0060] This allows the use of the existing signal transmission path 130 and image processing unit 150 even when an imaging element is introduced that has a readout speed faster than the transmission speed of the signal transmission path 130 and the processing speed of the image processing unit 150. Furthermore, because the HDR synthesis process has already been performed in the transmission image signal generation unit 201, the image processing load on the image processing unit 150 is further reduced.

[0061] Note that when the switching circuit 221 connects the original image signal receiving unit 210 to this circuit 223, HDR synthesis processing is always performed on the original image signal. However, if the user does not want HDR synthesis processing, the transmission image signal generation unit 202 may reduce the frame rate of the original image signal by other processing.

[0062] 9 is a block diagram of another transmission image signal generation unit 203 that can be used in place of the transmission image signal generation units 201 and 202. The transmission image signal generation unit 203 has a switching circuit 221, a bypass circuit 222, and a main circuit 223. Furthermore, the main circuit 223 includes a face detection circuit 271, a face image generation circuit 272, a facial expression evaluation circuit 273, and a frame selection circuit 274.

[0063] Similar to the transmission image signal generation units 201 and 202, the switching circuit 221 selectively couples the image signal including multiple frames 321, 322, 323, 324, and 325 received by the original image signal receiving unit 210 to either the bypass circuit 222 or the main circuit 223 in response to an instruction from the system control unit 140. When the input is coupled to the bypass circuit 222, the original image signal is directly connected to the transmission image signal transmitting unit 240 via the bypass circuit 222.

[0064] When the switching circuit 221 connects the input to the main circuit 223, the original image signal including multiple frames 321 to 325 is first subjected to face detection by the face detection circuit 271, which detects faces as subjects included in each image. Next, the face image generation circuit 272 creates an image of each detected face. As a result, the facial expression of each detected face appears in the image.

[0065] Next, the facial expression evaluation circuit 273 calculates an evaluation value of the facial expression for each of the generated face images. An example of an evaluation value is detection of a smile based on the shape of the corners of the mouth. Other evaluation values ​​may also be used, such as whether the detected face is facing the image capture device 100, whether the eyes are open in the detected image, etc. Furthermore, a comprehensive evaluation may be calculated by referring to multiple evaluation values.

[0066] Once the evaluation value is calculated for each of the multiple frames 321 to 325, the frame selection circuit 274 selects only the frame 322 with the high evaluation value to generate a transmission image signal. Therefore, the frame 322 with the high evaluation value is transmitted to the image processing unit 150, and the image that is most likely to reflect the user's intention is processed by the image processing unit 150 and ultimately recorded.

[0067] Note that the frame selection circuit 274 does not necessarily select a single frame 322; multiple frames 322 exceeding a preset threshold may be selected. However, in either case, the number of frames 322 ultimately included in the transmitted image signal is fewer than the number of frames included in the original image signal. Therefore, even if an imaging element with a readout speed faster than the transmission speed of the signal transmission path 130 and the processing speed of the image processing unit 150 is introduced, the existing signal transmission path 130 and image processing unit 150 can be used.

[0068] Furthermore, although evaluation based on face detection and smiles has been described as an example, it goes without saying that frames can be selected based on other selection criteria. The image signal processing device 200 may identify a main subject by moving object tracking or the like, and select frames to be sent to the image processing unit 150 based on or in combination with other evaluation values, such as whether the identified main subject occupies a large proportion of the area of ​​the entire image, whether the main subject is located at the center of the image, or whether the exposure balance of the main subject in the image is good.

[0069] The frame selection process described above is always performed when the switching circuit 221 connects the original image signal receiving unit 210 to this circuit 223. However, if the user does not want automatic frame selection, the transmission image signal generating unit 203 may reduce the frame rate of the original image signal by other processing.

[0070] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0071] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using words such as "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0072] 100 imaging device, 110 optical system, 120 imaging unit, 121, 250 substrate, 122 imaging element, 123 light receiving substrate, 124, 131 flexible cable, 125 logic board, 126, 127 via, 128 logic circuit, 130 signal transmission path, 140 system control unit, 150 image processing unit, 160 work memory, 170 operation member, 180 display unit, 190 secondary recording medium, 200 image signal processing device, 210 original image signal receiving unit, 201, 202, 203 transmission image signal generating unit, 221 switching circuit, 222 bypass circuit, 223 main circuit, 224 resizing circuit, 225 alignment detection circuit, 226 alignment adjustment circuit, 227, 262 frame synthesis circuit, 230 frame memory, 240 transmission image signal transmitting unit, 261 Synthesis map generation circuit, 263 moving object region extraction circuit, 264 moving object region replacement circuit, 271 face detection circuit, 272 face image generation circuit, 273 facial expression evaluation circuit, 274 frame selection circuit, 301, 302, 303, 311, 312, 313, 314, 321, 322, 323, 324, 325 frames

Claims

1. An imaging device including a plurality of stacked substrates, The plurality of substrates include: a first substrate that converts light into an electrical signal and outputs the electrical signal at a first frame rate; a second substrate that converts the electrical signal output from the first substrate into a digital signal and outputs the digital signal; a third substrate that performs signal processing on the signal output from the second substrate and outputs the processed signal at a second frame rate that is lower than the first frame rate; and the third substrate outputs either a first signal among the signals output from the second substrate or a second signal among the signals output from the second substrate at the second frame rate; Imaging device.

2. 2. The imaging device according to claim 1, the second substrate is disposed between the first substrate and the third substrate in a stacking direction in which the plurality of substrates are stacked; Imaging device.

3. 3. The imaging device according to claim 2, The second substrate and the third substrate are electrically connected by a through electrode that penetrates the second substrate. Imaging device.

4. 4. The imaging device according to claim 3, the third substrate performs the signal processing on the signal output to the through electrode. Imaging device.

5. 5. The imaging device according to claim 3, wherein: the first substrate and the second substrate are electrically connected by a member different from the through electrode; Imaging device.

6. 6. The imaging device according to claim 5, The first substrate and the second substrate are electrically connected by bumps. Imaging device.

7. 7. The imaging device according to claim 1, the third substrate controls to output one of the first signal and the second signal at the second frame rate and not to output the other of the first signal and the second signal; Imaging device.

8. 8. The imaging device according to claim 1, the third substrate calculates an evaluation value using the signals output from the second substrate, and selects a signal to be output from among the signals output from the second substrate based on the evaluation value. Imaging device.

9. 9. The imaging device according to claim 1, an image processing unit that performs image processing on the signal output at the second frame rate;

10. 10. The imaging device according to claim 9, the image processing unit receives the signal via a flexible substrate; Imaging device.

11. 11. The imaging device according to claim 1, An imaging device comprising an optical system that emits light to the first substrate.

Images