Imaging element, imaging device, method of operating an imaging element, and computer-readable storage medium

Abstract

An imaging element, an image pickup device, a method of operating an imaging element, and a computer-readable storage medium include a reception interface that receives, from outside the imaging element, an image pickup synchronization signal related to a time of image pickup and at least one output synchronization signal related to a time of outputting image data obtained through the image pickup; a memory that stores the image data obtained through the image pickup at a first frame rate in accordance with the image pickup synchronization signal received by the reception interface and is built into the imaging element; and an output circuit that outputs the image data stored in the memory at a second frame rate in accordance with the output synchronization signal received by the reception interface and is built into the imaging element, the first frame rate being higher than or equal to the second frame rate.

Description

[0001] This invention is a divisional application of the following application, the original application information of which is as follows:

[0002] Application date: January 28, 2020

[0003] Application Number: 202080015089.2

[0004] Invention Title: Imaging Element, Camera Device, Method for Operating the Imaging Element, and Computer-Readable Storage Medium Technical Field

[0005] The present invention relates to an imaging element, a camera device, a method of operating the imaging element, and a computer-readable storage medium. Background Technology

[0006] Re-publication No. 2014-7004 discloses an imaging element comprising a chip having a signal processing unit, a memory unit, a data processing unit, and a control unit, and a chip having a pixel array unit, stacked together. In the imaging element described in Re-publication No. 2014-7004, pixel data read from and digitized from each pixel of the pixel array unit is transmitted to the memory unit at a first speed, and pixel data is read from the memory unit at a second speed, slower than the first speed. The imaging element described in Re-publication No. 2014-7004 operates based on reference signals such as a horizontal synchronization signal, a vertical synchronization signal, and a master clock supplied from outside the chip.

[0007] Japanese Patent Application Publication No. 2007-295096 discloses a digital camera in which a synchronization signal generated outside the imaging element is input to the imaging element, and the imaging element operates according to the synchronization signal input from the outside. Summary of the Invention

[0008] One embodiment of the present invention provides an imaging element, an imaging device, an imaging element operation method, and a computer-readable storage medium capable of capturing and outputting images in accordance with external conditions of the imaging element.

[0009] means for solving technical problems

[0010] The first aspect of the present invention relates to an imaging element comprising: a receiving unit that receives from outside the imaging element a camera synchronization signal related to the time of shooting and at least one output synchronization signal related to the time of outputting image data obtained by shooting; a storage unit that stores image data obtained by shooting at a first frame rate according to the camera synchronization signal received by the receiving unit and is built into the imaging element; and an output unit that outputs the image data stored in the storage unit at a second frame rate according to the output synchronization signal received by the receiving unit and is built into the imaging element, wherein the first frame rate is at least the second frame rate. Thus, it is possible to achieve shooting and output corresponding to the external conditions of the imaging element.

[0011] The second aspect of the technology of the present invention is the imaging element described in the first aspect, wherein the imaging synchronization signal includes at least one of a vertical synchronization signal and a horizontal synchronization signal for imaging. Therefore, the shooting time can be adjusted externally to the imaging element.

[0012] The third aspect of the technology of the present invention is the imaging element described in the first or second aspect, wherein the output synchronization signal is a signal that includes at least one of an output vertical synchronization signal and an output horizontal synchronization signal. Therefore, the output timing can be adjusted externally to the imaging element.

[0013] The fourth aspect of the technology of the present invention is an imaging element according to any one of the first to third aspects, wherein the output unit has a plurality of output interfaces, and the receiving unit receives a plurality of output synchronization signals corresponding to each of the plurality of output interfaces. Therefore, even if there are multiple output destinations for image data, the image data can be processed according to the conditions of each output destination.

[0014] The fifth aspect of the technology of the present invention is an imaging element according to any one of the first to fourth aspects, wherein the output unit outputs the latest image data stored in the storage unit at the time point at which the receiving unit receives the output synchronization signal. Therefore, compared to the case where image data stored before the time point of receiving the output synchronization signal is output, the real-time performance between shooting and output can be improved.

[0015] The sixth aspect of the technology of the present invention is an imaging element according to any one of the first to fourth aspects, wherein the output unit outputs the latest image data as the latest image data among a plurality of image data. If the storage of one frame of image data related to the subject captured at the current time in the storage unit is completed within a predicted output completion time (the time when the output of one frame of image data is completed), the latest image data is one frame of image data related to the subject captured at the current time. If the storage of one frame of image data related to the subject captured at the current time in the storage unit is not completed within the output completion time, the latest image data is one frame of image data already stored in the storage unit. Thus, both real-time performance between capture and output and the avoidance of image data output stagnation can be achieved.

[0016] The seventh aspect of the present invention relates to the imaging element described in the sixth aspect, wherein, if the storage of the latest image data to the storage unit is not completed within the output completion time, the latest image data is the latest image data stored in the storage unit. Therefore, compared to the case where the output is image data that has been stored in the storage unit for several frames earlier than the image data currently being stored in the storage unit, real-time performance between shooting and output can be ensured.

[0017] The eighth aspect of the technology of the present invention is the imaging element described in the sixth or seventh aspect, wherein the output synchronization signal is a signal that includes an output horizontal synchronization signal, and the output completion time is a time predicted based on the period of the output horizontal synchronization signal received by the receiving unit. Therefore, compared with the case where the output completion time is determined using a signal other than the output horizontal synchronization signal, the output completion time can be determined more accurately.

[0018] The ninth aspect of the present invention relates to an imaging element as described in any one of the first to eighth aspects, wherein the imaging synchronization signal is a signal including a vertical synchronization signal for imaging, and the imaging element further includes: an imaging system holding unit that holds imaging system driving mode indication information indicating the driving mode of the imaging system; and an imaging system control unit that, upon receiving the vertical synchronization signal for imaging from the receiving unit, controls the imaging system to drive in the driving mode indicated by the imaging system driving mode indication information held by the imaging system holding unit. Thus, the imaging system can be driven in the driving mode indicated by the imaging system driving mode indication information for each frame.

[0019] The tenth aspect of the present invention relates to an imaging element as described in the ninth aspect, wherein a receiving unit receives camera system rewrite content information representing rewrite content of camera system drive mode indication information, and upon receiving the camera system rewrite content information, the camera system drive mode indication information held by the camera system holding unit is rewritten to the content represented by the camera system rewrite content information. Therefore, it is possible to rewrite the content of the camera system drive mode indication information held within the imaging element from outside the imaging element.

[0020] The eleventh aspect of the present invention relates to the imaging element described in the tenth aspect, wherein the camera system driving mode indication information includes at least one of the following: information related to the imaging area, information related to pixel interval rejection, information related to the pixel addition operation method, information related to the exposure time, information related to conversion gain switching, information related to analog gain, and information related to A / D conversion accuracy. Therefore, it is possible to rewrite from outside the imaging element the content of information containing at least one of the following: information related to the imaging area, information related to pixel interval rejection, information related to the pixel addition operation method, information related to the exposure time, information related to conversion gain switching, information related to analog gain, and information related to A / D conversion accuracy, all stored within the imaging element.

[0021] The 12th aspect of the present invention relates to an imaging element according to any one of the 1st to 11th aspects, wherein the output synchronization signal is a signal including an output vertical synchronization signal, and the imaging element further includes: an output system holding unit that holds output system driving mode indication information indicating the driving mode of the output system of the imaging element; and an output system control unit that, upon receiving the output vertical synchronization signal from the receiving unit, controls the output system to drive in the driving mode indicated by the output system driving mode indication information held by the output system holding unit. Thus, the output system can be driven in the driving mode indicated by the output system driving mode indication information for each frame.

[0022] The 13th aspect of the present invention relates to the imaging element described in the 12th aspect, wherein the receiving unit receives output system rewrite content information representing rewrite content of output system drive mode indication information, and when the receiving unit receives the output system rewrite content information, the output system drive mode indication information held by the output system holding unit is rewritten to the content represented by the output system rewrite content information. Therefore, it is possible to rewrite the content of the output system drive mode indication information held within the imaging element from outside the imaging element.

[0023] The 14th aspect of the technology of this invention is the imaging element described in the 12th or 13th aspect, wherein the output system drive mode indication information includes at least one of the following: information related to the output destination, information related to digital interval rejection, information related to the digital addition operation method, information related to the average number of output frames, information related to digital gain, information related to the A / D output bit depth, and information related to the low-order idle bit filling method. Therefore, it is possible to rewrite from outside the imaging element the content of information containing at least one of the following: information related to the output destination, information related to digital interval rejection, information related to the digital addition operation method, information related to the average number of output frames, information related to digital gain, information related to the A / D output bit depth, and information related to the low-order idle bit filling method.

[0024] The 15th aspect of the technology of the present invention is an imaging element according to any one of the 1st to 14th aspects, which is formed by at least integrating the photoelectric conversion element and the storage unit into a single chip. Therefore, compared with an imaging element that does not integrate the photoelectric conversion element and the storage unit into a single chip, the portability of the imaging element is improved.

[0025] The 16th aspect of the present invention relates to the imaging element described in the 15th aspect, wherein the imaging element is a stacked imaging element on which a storage unit is stacked on a photoelectric conversion element. Therefore, compared to the case where the photoelectric conversion element and the storage unit are not stacked, the transmission speed of image data from the photoelectric conversion element to the storage unit can be improved.

[0026] The 17th aspect of the present invention relates to a camera device, comprising: an imaging element as described in any one of the first to 16 aspects; and a display control unit that controls the display unit to display an image based on image data output by the output unit. Thus, it is possible to achieve imaging and output corresponding to the external conditions of the imaging element.

[0027] The 18th aspect of the present invention relates to a method of operating an imaging element, the imaging element including a receiving unit, a storage unit, and an output unit, and having the storage unit and output unit built-in. The method of operating the imaging element includes the following steps: the receiving unit receives from outside the imaging element a camera synchronization signal related to the time of shooting and at least one output synchronization signal related to the time of outputting image data obtained by shooting; the storage unit stores image data obtained by shooting at a first frame rate according to the camera synchronization signal received by the receiving unit; and the output unit outputs the image data stored in the storage unit at a second frame rate according to the output synchronization signal received by the receiving unit, setting the first frame rate to or higher than the second frame rate. Thus, it is possible to achieve shooting and output corresponding to the external conditions of the imaging element.

[0028] The 19th aspect of the present invention is a program for enabling a computer to function as a receiver and output unit included in an imaging element. The imaging element includes a receiver, a storage unit, and an output unit, with the storage unit and output unit built-in. The receiver receives from outside the imaging element a camera synchronization signal related to the time of shooting and at least one output synchronization signal related to the time of outputting image data obtained through shooting. The storage unit stores image data obtained by shooting at a first frame rate according to the camera synchronization signal received by the receiver. The output unit outputs the image data stored in the storage unit at a second frame rate according to the output synchronization signal received by the receiver, where the first frame rate is greater than or equal to the second frame rate. Thus, shooting and output corresponding to the external conditions of the imaging element can be achieved.

[0029] The 20th aspect of the present invention is an imaging element comprising a processor and a memory respectively built into the imaging element. The processor receives from outside the imaging element a camera synchronization signal related to the time of shooting and at least one output synchronization signal related to the time of outputting image data obtained by shooting. The memory stores image data obtained by shooting at a first frame rate according to the camera synchronization signal received by the processor. The processor outputs the image data stored in the memory at a second frame rate according to the received output synchronization signal. The first frame rate is greater than or equal to the second frame rate. Attached Figure Description

[0030] Figure 1 This is a perspective view showing an example of the appearance of the camera device according to embodiments 1 to 5.

[0031] Figure 2 It means Figure 1 A rear view of an example of the appearance of the rear side of the camera device shown.

[0032] Figure 3 This is a block diagram illustrating an example of the structure of the camera device according to embodiments 1 to 4.

[0033] Figure 4A This is a conceptual diagram used to illustrate the imaging frame rate of the imaging element included in the imaging device according to embodiments 1 to 5.

[0034] Figure 4B This is a conceptual diagram used to illustrate the output frame rate of the imaging element included in the camera device according to embodiments 1 to 5.

[0035] Figure 5 This is a block diagram illustrating an example of the structure of the electrical system of the main body of the camera device according to embodiments 1 to 4.

[0036] Figure 6 This is a schematic structural diagram showing the structure of the hybrid viewfinder included in the camera device according to embodiments 1 to 5.

[0037] Figure 7 This is a block diagram illustrating an example of the stacked structure of the imaging elements included in the camera device according to embodiments 1 to 4, and an example of the connection relationship between the imaging elements, the signal processing unit, and the controller.

[0038] Figure 8 This is a block diagram illustrating an example of the structure of the electrical system of the imaging element included in the camera device according to the first embodiment.

[0039] Figure 9 This is a state transition diagram illustrating an example of the processing content in the time sequence of imaging processing and output processing performed by the imaging element included in the imaging device according to the first embodiment.

[0040] Figure 10 This is a timing diagram illustrating an example of the operation flow of the imaging element included in the camera device according to the first embodiment.

[0041] Figure 11 This is a conceptual diagram illustrating an example of the stored content of the storage circuit within the imaging element included in the camera device according to embodiments 1 to 5.

[0042] Figure 12 This is a flowchart illustrating an example of the read-and-store processing flow according to the first embodiment.

[0043] Figure 13 This is a flowchart illustrating an example of the output processing flow involved in the first embodiment.

[0044] Figure 14 This is a flowchart illustrating an example of the camera system drive processing flow according to the first embodiment.

[0045] Figure 15 This is a flowchart illustrating an example of the output system drive processing flow according to the first embodiment.

[0046] Figure 16 This is a flowchart illustrating an example of the rewrite process involved in the first embodiment.

[0047] Figure 17 This is a block diagram illustrating an example of the structure of the electrical system of the imaging element included in the camera device according to the second embodiment.

[0048] Figure 18 This is a flowchart illustrating an example of the read-and-store processing flow involved in the second embodiment.

[0049] Figure 19 This is a flowchart illustrating an example of the output processing flow involved in the second embodiment.

[0050] Figure 20 This is a timing diagram illustrating an example of the operation flow of the imaging element included in the camera device according to the third embodiment.

[0051] Figure 21 This is a flowchart illustrating an example of the output processing flow involved in the third embodiment.

[0052] Figure 22 This is a block diagram illustrating the storage contents of the storage circuit within the imaging element included in the camera device according to the fourth embodiment.

[0053] Figure 23 This is a block diagram illustrating an example of a characteristic portion of the structure of the electrical system of the imaging element included in the camera device according to the fourth embodiment.

[0054] Figure 24 This is a conceptual diagram illustrating the processing content of the determination section within the control circuit of the imaging element included in the camera device according to the fourth embodiment.

[0055] Figure 25A This is a flowchart illustrating an example of the output processing flow involved in the fourth embodiment.

[0056] Figure 25B yes Figure 25A Continuing from the flowchart shown.

[0057] Figure 26 This is a block diagram illustrating an example of the stacked structure of the imaging element included in the camera device according to the fifth embodiment, and an example of the connection relationship between the imaging element, the first signal processing unit, the second signal processing unit, and the controller.

[0058] Figure 27 This is a block diagram illustrating a characteristic portion of the structure of the electrical system within the processing circuit of the imaging element included in the camera device according to the fifth embodiment, and an example of the connection relationship between the processing circuit, the first signal processing unit, the second signal processing unit, and the controller.

[0059] Figure 28 This is a conceptual diagram illustrating an example of how various programs are installed into a computer from a storage medium containing various programs.

[0060] Figure 29 This is a block diagram illustrating an example of the schematic structure of a smart device assembled with the imaging elements according to embodiments 1 to 5. Detailed Implementation

[0061] Hereinafter, an example of an embodiment of the camera device according to the present invention will be described with reference to the accompanying drawings.

[0062] First, the meanings of the terms used in the following description will be explained.

[0063] In the following descriptions, CPU stands for "Central Processing Unit". RAM stands for "Random Access Memory". ROM stands for "Read Only Memory". DRAM stands for "Dynamic Random Access Memory". SRAM stands for "Static Random Access Memory".

[0064] Furthermore, in the following descriptions, LSI stands for "Large-Scale Integration." Also, in the following descriptions, ASIC stands for "Application Specific Integrated Circuit." Also, in the following descriptions, PLD stands for "Programmable Logic Device." And, in the following descriptions, FPG stands for "Field-Programmable Gate Array."

[0065] Furthermore, in the following descriptions, SSD stands for "Solid State Drive". DVD-ROM stands for "Digital Versatile Disc Read-Only Memory". USB stands for "Universal Serial Bus". HDD stands for "Hard Disk Drive". EE PROM stands for "Electrically Erasable and Programmable Read Only Memory".

[0066] Short for "Electrically Erasable Programmable Read-Only Memory".

[0067] Furthermore, in the following descriptions, CCD stands for "Charge Coupled Device." Also, in the following descriptions, CMOS stands for "Complementary Metal Oxide Semiconductor." Also, in the following descriptions, EL stands for "Electro-Luminescence." Also, in the following descriptions, A / D stands for "Analog / Digital." Also, in the following descriptions, I / F stands for "Interface." And in the following descriptions, UI stands for "User Interface."

[0068] [First Implementation]

[0069] As an example, such as Figure 1 As shown, the camera device 10 is an interchangeable lens camera. The camera device 10 is a digital camera, which includes a camera device body 12, an interchangeable lens 14 that is interchangeably mounted on the camera device body 12, and omits a reflex mirror.

[0070] An imaging element 44 is provided on the camera device body 12. When the interchangeable lens 14 is mounted on the camera device body 12, the light representing the subject passes through the interchangeable lens 14 and forms an image on the imaging element 44, and the imaging element 44 generates image data 69 representing the image of the subject (for example, see reference). Figure 4A and Figure 4B ).

[0071] A hybrid viewfinder (registered trademark) 16 is provided on the main body 12 of the camera device. The hybrid viewfinder 16 referred to here means a viewfinder that selectively uses, for example, an optical viewfinder (hereinafter referred to as "OVF") and an electronic viewfinder (hereinafter referred to as "EVF"). Furthermore, OVF is an abbreviation for "optical viewfinder," and EVF is an abbreviation for "electronic viewfinder."

[0072] A viewfinder switching lever 18 is provided on the front surface of the camera unit body 12. By rotating the viewfinder switching lever 18 in the direction of arrow SW, the optical image that can be recognized by the OVF (Optical Viewfinder) and the electronic image that can be recognized by the EVF (Electronic Viewfinder) are switched, i.e., the live preview image. The "live preview image" referred to here is a dynamic image for display based on the image data 69 obtained by the imaging element 44. The live preview image is also commonly referred to as a live view image. A release button 20 and a dial 22 are provided on the upper surface of the camera unit body 12. The dial 22 is operated when setting the operating mode of the camera system and the operating mode of the playback system, etc.

[0073] The release button 20 functions as both a camera preparation indicator and a camera indication indicator, and can detect press operations in both the camera preparation indicator state and the camera indication state. The camera preparation indicator state refers, for example, to the state where the button is pressed from the standby position to the middle position (half-press position), and the camera indication state refers to the state where the button is pressed to the final pressed position (fully pressed position) beyond the middle position. Furthermore, hereinafter, the state of "pressing from the standby position to the half-press position" will be referred to as the "half-press state," and the state of "pressing from the standby position to the fully pressed position" will be referred to as the "fully pressed state."

[0074] In the camera device 10, shooting mode and playback mode are selectively set as action modes according to the user's instructions. Shooting modes are broadly divided into shooting mode for displaying animation and shooting mode for recording.

[0075] As an example, such as Figure 2 As shown, a touch panel display 26, indicator keys 28, and a viewfinder eyepiece 30 are provided on the back of the camera device body 12.

[0076] Touch panel display 26 includes a first display 32 and a touch panel 34 (see also) Figure 5 As an example of the first display 32, a liquid crystal display (LCD) can be cited. Alternatively, the first display 32 may not be an LCD, but rather an organic EL display or other types of display.

[0077] The first display 32 displays images and character information, etc. The first display 32 is used to display real-time preview images obtained through continuous shooting when the camera device 10 is in shooting mode. Furthermore, the first display 32 is also used to display still images obtained through shooting when an instruction to shoot a still image is given. In addition, the first display 32 is also used to display playback images and menu screens when the camera device 10 is in playback mode.

[0078] Touch panel 34 is a transmissive touch panel and overlaps with the surface of the display area of ​​the first display 32. Touch panel 34 detects contact based on a pointer such as a finger or stylus.

[0079] Indicator key 28 receives various instructions such as selecting one or more menus, confirming the selection, deleting the selection, zooming, and frame transmission.

[0080] As an example, such as Figure 3 As shown, the interchangeable lens 14 includes an imaging lens 40. The imaging lens 40 includes an objective lens 40A, a focusing lens 40B, and an aperture 40C. The objective lens 40A, focusing lens 40B, and aperture 40C are arranged along the optical axis L1 from the subject side to the imaging device body 12 side in the order of objective lens 40A, focusing lens 40B, and aperture 40C. The focusing lens 40B and aperture 40C operate by being powered by a drive source such as a motor (not shown). That is, the focusing lens 40B and aperture 40C move along the optical axis L1 according to the applied power. Furthermore, the aperture 40C adjusts the exposure by operating according to the applied power.

[0081] The main body 12 of the camera device includes a mechanical shutter 42 and an imaging element 44. The mechanical shutter 42 operates by being powered by a drive source such as a motor (not shown). When the interchangeable lens 14 is mounted on the main body 12 of the camera device, the light of the subject passes through the imaging lens 40 and is imaged on the light-receiving surface 44A of the imaging element 44 via the mechanical shutter 42.

[0082] The main body 12 of the camera device includes a controller 46, a UI system device 48, and a signal processing unit 50. The controller 46 and the signal processing unit 50, which respectively correspond to the downstream circuit of the imaging element 44, are examples of the "external part of the imaging element" involved in the technology of the present invention.

[0083] The controller 46 controls the entire camera device 10. The UI system device 48 is a device that provides information to the user or receives instructions from the user. The UI system device 48 is connected to the controller 46, and the controller 46 acquires various information from the UI system device 48 and controls the UI system device 48.

[0084] Imaging element 44 is connected to controller 46 via communication line 57, and under the control of controller 46, it captures the subject, thereby generating image data 69 representing the image of the subject.

[0085] Imaging element 44 is connected to signal processing unit 50 via communication lines 53 and 55. Signal processing unit 50 is an LSI, specifically, a device including ASIC. Controller 46 is connected to signal processing unit 50 via communication line 60. Controller 46 acquires various information from signal processing unit 50 and controls signal processing unit 50.

[0086] The signal processing unit 50 outputs a signal to the imaging element 44 via the communication line 55. Details will be described later. The imaging element 44 performs actions corresponding to the signals input from the signal processing unit 50 via the communication line 55. Image data 69 is input from the imaging element 44 to the signal processing unit 50 via the communication line 53. The signal processing unit 50 performs various signal processing operations on the image data 69 input from the imaging element 44 via the communication line 53. These various signal processing operations include, for example, known signal processing operations such as white balance adjustment, sharpness adjustment, gamma correction, color space conversion processing, and chromatic aberration correction.

[0087] Furthermore, in this embodiment, an ASIC-based device is illustrated as the signal processing unit 50, but the technology of the present invention is not limited thereto. The signal processing unit 50 can be a device including an ASIC, an FPGA, and / or a PLD. Moreover, the signal processing unit 50 can also be a computer including a CPU, ROM, and RAM. There can be one or more CPUs. Furthermore, the signal processing unit 50 can be implemented through a combination of hardware and software structures.

[0088] Imaging element 44 is an example of a "stacked imaging element" according to the technology of the present invention. In this embodiment, imaging element 44 is a CMOS image sensor. Furthermore, while a CMOS image sensor is illustrated here as imaging element 44, the technology of the present invention is not limited thereto; for example, the technology of the present invention also applies even if imaging element 44 is a CCD image sensor.

[0089] In the imaging element 44, by capturing the subject at a camera frame rate, as shown in FIG4, multiple image data 69 representing the subject are generated. Furthermore, the generated multiple image data 69 are output at an output frame rate in the imaging element 44. Both the camera frame rate and the output frame rate are variable frame rates. Additionally, the camera frame rate is an example of the "first frame rate" according to the technology of this invention, and the output frame rate is an example of the "second frame rate" according to the technology of this invention.

[0090] There is a relationship between camera frame rate and output frame rate: "camera frame rate ≥ output frame rate". For example, ... Figure 4A As shown, the camera frame rate is the frame rate at which 8 frames are captured within a period T, such as... Figure 4B As shown, the output frame rate is the frame rate at which two frames are output within a period T. Specifically, as an example of camera frame rate, 240 fps (frames per second) can be cited, and as an example of output frame rate, 60 fps can be cited.

[0091] As an example, such as Figure 5As shown, the controller 46 includes a CPU 46A, a ROM 46B, a RAM 46C, a connection I / F 46D, and an input I / F 46E. The CPU 46A, ROM 46B, RAM 46C, connection I / F 46D, and input I / F 46E are interconnected via a bus 88.

[0092] Various programs are stored in ROM 46B. CPU 46A reads various programs from ROM 46B and expands the read programs into RAM 46C. CPU 46A controls the entire camera device 10 according to the various programs expanded into RAM 46C.

[0093] The I / F46D is a communication device with an FPGA, connected to the imaging element 44 via communication line 57. The CPU 46A controls the imaging element 44 via the I / F46D.

[0094] The input I / F46E is a communication device with an FPGA and is connected to the signal processing unit 50 via communication line 60. Image data 69 (reference) undergoes various signal processing by the signal processing unit 50. Figure 4A and Figure 4B The image data 69 input from the signal processing unit 50 is transmitted to the input I / F46E via the communication line 60. The input I / F46E transmits the image data 69 input from the signal processing unit 50 to the CPU 46A.

[0095] Auxiliary storage device 80 and external I / F 82 are connected to bus 88. Auxiliary storage device 80 is a non-volatile memory such as SSD, HDD, or EEPROM. CPU 46A reads and writes various information to auxiliary storage device 80.

[0096] The external I / F82 is a communication device with an FPGA. External devices such as USB memory and memory cards (not shown) are connected to the external I / F82. The external I / F82 controls the exchange of various information between the CPU46A and the external devices.

[0097] UI system device 48 includes a hybrid viewfinder 16, a touch panel display 26, and a receiver 84. The first display 32 and the touch panel 34 are connected to the bus 88. Therefore, the CPU 46A causes the first display 32 to display various information and performs actions according to various instructions received from the touch panel 34.

[0098] The receiving device 84 includes a touch panel 34 and a hard key unit 25. The hard key unit 25 has multiple hard keys and includes a release button 20, a rotary dial 22, and an indicator key 28. The hard key unit 25 is connected to a bus 88, and the CPU 46A operates according to various instructions received from the hard key unit 25.

[0099] The hybrid viewfinder 16 includes a second display 86, which is used by the CPU 46A to display various information. An example of the second display 86 is a liquid crystal display (LCD). However, the second display 86 may not be an LCD, but rather another type of display such as an organic EL display.

[0100] As an example, such as Figure 6 As shown, the hybrid viewfinder 16 includes an OVF90 and an EVF92. The OVF90 is a reverse Galilean viewfinder and has an eyepiece lens 94, a prism 96, and an objective lens 98. The EVF92 has a second display 86, a prism 96, and an eyepiece lens 94.

[0101] A liquid crystal shutter 100 is positioned along the optical axis L2 of the objective lens 98, closer to the subject than the objective lens 98. When using the EVF 92, the liquid crystal shutter 100 blocks light to prevent the optical image from entering the objective lens 98.

[0102] Prism 96 reflects the electronic image or various information displayed on the second display 86 and guides it to eyepiece lens 94, and combines the optical image with the electronic image and / or various information displayed on the second display 86. As an example of the electronic image displayed on the second display 86, a real-time preview image 102 based on image data 69 can be shown.

[0103] In OVF mode, CPU46A controls the liquid crystal shutter 100 to be in an unshaded state so that the optical image can be visually recognized through the eyepiece lens 94. Furthermore, in EVF mode, CPU46A controls the liquid crystal shutter 100 to be in a shaded state so that the electronic image displayed on the second display 86 can be visually recognized solely through the eyepiece lens 94.

[0104] Additionally, for ease of explanation, the first display 32 (reference) will not be discussed further below unless otherwise specified. Figure 2 and Figure 5 In the case of the second display 86 and the third display 86, the term "display" is used without the designation. The display is an example of a "display unit (shower)" according to the technology of this invention. Furthermore, the CPU 46A is an example of a "display control unit (display processor)" according to the technology of this invention.

[0105] As an example, such as Figure 7As shown, the imaging element 44 incorporates a photoelectric conversion element 61, a processing circuit 62, and a memory 64. The imaging element 44 is an imaging element formed by integrating the photoelectric conversion element 61, the processing circuit 62, and the memory 64 onto a single chip. That is, the photoelectric conversion element 61, the processing circuit 62, and the memory 64 are packaged. In the imaging element 44, the processing circuit 62 and the memory 64 are stacked on top of the photoelectric conversion element 61. Specifically, the photoelectric conversion element 61 and the processing circuit 62 are electrically connected to each other via conductive bumps (not shown) such as copper, and the processing circuit 62 and the memory 64 are also electrically connected to each other via conductive bumps (not shown) such as copper. Here, a three-layer structure of the photoelectric conversion element 61, the processing circuit 62, and the memory 64 is illustrated, but the technology of the present invention is not limited to this; it can also be a two-layer structure with the processing circuit 62 and the memory 64 as one layer (memory layer) and the photoelectric conversion element 61. Furthermore, the memory 64 is an example of a "storage unit" involved in the technology of the present invention.

[0106] The processing circuit 62 is, for example, an LSI, and the memory 64 is, for example, DRAM. However, the technology of the present invention is not limited thereto, and SRAM can be used instead of DRAM as the memory 64.

[0107] The processing circuit 62 is a device including an ASIC and an FPGA, which controls the entire imaging element 44 according to the instructions of the controller 46. Furthermore, while this example illustrates the processing circuit 62 implemented using a device including an ASIC and an FPGA, the technology of the present invention is not limited to this; for example, it could also be a device including an ASIC, an FPGA, and / or a PLD. Moreover, the processing circuit 62 can be a computer including a CPU, ROM, and RAM. There can be one or more CPUs. Furthermore, the processing circuit 62 can be implemented through a combination of hardware and software structures.

[0108] The photoelectric conversion element 61 has a plurality of photodiodes arranged in a matrix. As an example of a plurality of photodiodes, a photodiode with a pixel count of "4896×3265" can be given.

[0109] Color filters are disposed in each photodiode included in the photoelectric conversion element 61. The color filters include a G color filter corresponding to G (green), an R color filter corresponding to R (red), and a B color filter corresponding to B (blue), which are most helpful for obtaining the brightness signal. The photoelectric conversion element 61 has R pixels, G pixels, and B pixels. An R pixel is a pixel corresponding to a photodiode with an R color filter, a G pixel is a pixel corresponding to a photodiode with a G color filter, and a B pixel is a pixel corresponding to a photodiode with a B color filter. The R pixels, G pixels, and B pixels are arranged in a predetermined periodicity in both the row direction (horizontal direction) and the column direction (vertical direction). In this embodiment, the R pixels, G pixels, and B pixels are arranged in a periodicity corresponding to the X-Trans (registered trademark) arrangement. Furthermore, while the X-Trans arrangement is illustrated here, the technology of the present invention is not limited to this; the arrangement of the R pixels, G pixels, and B pixels can also be a Bayer arrangement or a honeycomb arrangement, etc.

[0110] Imaging element 44 has a so-called electronic shutter function, which, under the control of controller 46, controls the charge accumulation time of each photodiode in photoelectric conversion element 61 by activating the electronic shutter function. Charge accumulation time refers to the so-called shutter speed.

[0111] In the camera device 10, still image capture and moving image capture are performed using a rolling shutter mode. Still image capture is performed by activating the electronic shutter function and setting the mechanical shutter 42 (reference) to... Figure 3 The dynamic image is captured by activating the electronic shutter function without engaging the mechanical shutter 42. Furthermore, while a rolling shutter mode is illustrated here, the invention is not limited to this and a global shutter mode can be used instead.

[0112] Processing circuit 62 reads out image data 69 obtained by photographing the subject using photoelectric conversion element 61. Image data 69 is signal charge stored in photoelectric conversion element 61. Processing circuit 62 performs A / D conversion on the analog image data 69 read from photoelectric conversion element 61. Processing circuit 62 stores the digital image data 69 obtained by A / D conversion of analog image data 69 in memory 64.

[0113] The processing circuit 62 is connected to the signal processing unit 50 via communication lines 53 and 55. Furthermore, the processing circuit 62 is connected to the controller 46 via communication line 57.

[0114] As an example, such as Figure 8As shown, the processing circuit 62 includes a readout circuit 62A, a digital processing circuit 62B, an image processing circuit 62C, an output circuit 62D, a control circuit 62E, and a storage circuit 62F. The output circuit 62D is an example of an "output unit (output circuit)" according to the technology of this invention. The control circuit 62E is an example of a "camera system control unit (camera system control circuit)" and an "output system control unit (output system control circuit)" according to the technology of this invention.

[0115] The readout circuit 62A is connected to the photoelectric conversion element 61, the digital processing circuit 62B, and the control circuit 62E. The memory 64 is connected to the control circuit 62E. The image processing circuit 62C is also connected to the control circuit 62E. The output circuit 62D is also connected to the control circuit 62E. The storage circuit 62F is also connected to the control circuit 62E.

[0116] As an example, such as Figure 8 As shown, the image data 69 is roughly divided into analog image data 69A and digital image data 69B. Furthermore, for ease of explanation, it will be referred to as "image data 69" hereinafter, unless it is necessary to distinguish between analog image data 69A and digital image data 69B.

[0117] The processing circuit 62 includes a first receiving I / F 63A and a second receiving I / F 63B. Both the first receiving I / F 63A and the second receiving I / F 63B are communication devices with FPGAs. The first receiving I / F 63A and the second receiving I / F 63B are examples of the "receiving unit" involved in the technology of this invention.

[0118] The first receiver I / F 63A is connected to the control circuit 62E. Furthermore, the first receiver I / F 63A is connected to the controller 46 via communication line 57.

[0119] The second receiver I / F 63B is connected to the output circuit 62D. Furthermore, the second receiver I / F 63B is connected to the signal processing unit 50 via the communication line 55.

[0120] Under the control of the control circuit 62E, the readout circuit 62A controls the photoelectric conversion element 61 to read analog image data 69A from the photoelectric conversion element 61. The readout of one frame of analog image data 69A from the photoelectric conversion element 61 is performed according to the camera synchronization signal related to the shooting time.

[0121] The first receiver I / F63A receives the camera synchronization signal. Specifically, the controller 46 outputs the camera synchronization signal to the first receiver I / F63A via communication line 57, and the first receiver I / F63A receives the camera synchronization signal. The first receiver I / F63A outputs the received camera synchronization signal to the control circuit 62E. The camera synchronization signal includes a vertical synchronization signal for imaging. The vertical synchronization signal for imaging is a signal that specifies the start time of readout of one frame of analog image data 69A. In the imaging element 44, images are captured at the camera frame rate according to the vertical synchronization signal for imaging received by the first receiver I / F63A.

[0122] The control circuit 62E transmits the camera vertical synchronization signal input from the controller 46 via the first receiver I / F 63A to the readout circuit 62A. When the camera vertical synchronization signal is input from the control circuit 62E, the readout circuit 62A begins reading one frame of analog image data 69A from the photoelectric conversion element 61. The readout circuit 62A performs analog signal processing on the analog image data 69A read from the photoelectric conversion element 61. The analog signal processing includes known processing such as noise cancellation processing and analog gain processing. Noise cancellation processing is the process of eliminating noise caused by the deviation of characteristics between pixels included in the photoelectric conversion element 61. Analog gain processing is the process of applying gain to the analog image data 69A. The analog image data 69A, thus subjected to analog signal processing, is output to the digital processing circuit 62B through the readout circuit 62A.

[0123] The digital processing circuit 62B includes an A / D converter 62B1. The digital processing circuit 62B performs digital signal processing on the analog image data 69A input from the readout circuit 62A. The digital signal processing includes, for example, correlated double sampling, A / D conversion based on the A / D converter 62B1, and digital gain processing.

[0124] The analog image data 69A is subjected to correlation double sampling by the digital processing circuit 62B. The analog image data 69A, after correlation double sampling, is then converted to digital data by the A / D converter 62B1, thus digitizing the analog image data 69A to obtain digital image data 69B. Furthermore, the digital processing circuit 62B performs digital gain processing on the digital image data 69B. Digital gain processing refers to the process of applying gain to the digital image data 69B.

[0125] The control circuit 62E acquires the digital image data 69B obtained by digital signal processing from the digital processing circuit 62B, and stores the acquired digital image data 69B in the memory 64.

[0126] The memory 64 is a memory capable of storing multiple frames of digital image data 69B. The memory 64 has storage areas in pixel units (not shown), and the digital image data 69B is stored in the corresponding storage areas of the memory 64 in pixel units via the control circuit 62E.

[0127] The control circuit 62E can randomly access the memory 64 to retrieve digital image data 69B. The control circuit 62E outputs the digital image data 69B retrieved from the memory 64 to the image processing circuit 62C. The image processing circuit 62C performs image processing on the digital image data 69B input from the control circuit 62E. Examples of "image processing" include de-mosaic processing, digital interval removal processing, digital addition operations, and data embedding processing.

[0128] Demosaicing is the process of calculating all color information for each pixel based on a mosaic image corresponding to the arrangement of color filters. For example, in the case of an imaging element composed of RGB color filters, all RGB color information is calculated for each pixel based on a mosaic image composed of RGB.

[0129] The digital interval removal process is a process of removing pixels included in the digital image data 69B at row-unit intervals. A row unit refers to, for example, a horizontal row unit and / or a vertical row unit. The digital addition process is, for example, a process of averaging the pixel values ​​of multiple pixels included in the digital image data 69B. The data embedding process can include, for example, a process of embedding specific data into the low-order empty bits of the digital image data 69B. "Specific data" as referred to here can include, for example, information that determines the method of performing the digital interval removal process on the digital image data 69B, or information that determines the frame number, etc.

[0130] The image processing circuit 62C outputs the processed digital image data 69B to the control circuit 62E. The control circuit 62E stores the digital image data 69B input from the image processing circuit 62C in the memory 64.

[0131] Control circuit 62E retrieves digital image data 69B from memory 64 in units of one frame. Control circuit 62E outputs the digital image data 69B retrieved from memory 64 to output circuit 62D.

[0132] The second receiver I / F 63B receives an output synchronization signal related to the timing of outputting the digital image data 69B obtained through imaging. Specifically, the signal processing unit 50 outputs the output synchronization signal to the second receiver I / F 63B via the communication line 55, and the second receiver I / F 63B receives the output synchronization signal. The second receiver I / F 63B outputs the received output synchronization signal to the output circuit 62D. The output synchronization signal includes an output vertical synchronization signal. The output vertical synchronization signal is a signal that defines the start time of outputting one frame of digital image data 69B. In the imaging element 44, the digital image data 69B stored in the memory 64 is output to the signal processing unit 50 at an output frame rate according to the output vertical synchronization signal received by the second receiver I / F 63B.

[0133] The output circuit 62D includes an output I / F 62D1. The output I / F 62D1 is a communication device with an FPGA. The output I / F 62D1 is connected to the signal processing unit 50 via communication line 53. When a vertical synchronization signal for output is input from the signal processing unit 50 via the second receive I / F 63B, the output circuit 62D begins to output one frame of digital image data 69B from the output I / F 62D1 to the signal processing unit 50.

[0134] As an example, such as Figure 9 As shown, imaging element 44 performs processing including image processing and output processing. Image processing is performed according to the image frame rate, and output processing is performed according to the output frame rate.

[0135] In the image processing, the following steps are performed in sequence: exposure, reading out analog image data 69A, resetting photoelectric conversion element 61, analog signal processing, digital signal processing, first storage, first acquisition, image processing, and second storage.

[0136] Exposure is performed by photoelectric conversion element 61. Readout circuit 62A reads out analog image data 69A, resets photoelectric conversion element 61, and processes analog signals. Furthermore, the period during which exposure is performed by photoelectric conversion element 61 is a period during which analog image data 69A is not read out and photoelectric conversion element 61 is not reset.

[0137] Digital signal processing is performed by digital processing circuit 62B. The first storage refers to storing the digital image data 69B obtained through digital signal processing into memory 64. The first retrieval refers to retrieving the digital image data 69B from memory 64 for the first time. The first storage and first retrieval are performed by control circuit 62E. Image processing circuit 62C performs image processing on the digital image data 69B retrieved by control circuit 62E. The second storage refers to storing the image-processed digital image data 69B into memory 64. The second storage is performed by control circuit 62E.

[0138] The output processing involves a second acquisition and the output of digital image data 69B. The second acquisition refers to retrieving the processed digital image data 69B from memory 64. This second acquisition is performed by control circuit 62E. The output of digital image data 69B refers to the output of the processed digital image data 69B retrieved from memory 64 by control circuit 62E via output circuit 62D.

[0139] In the camera processing, the readout of analog image data 69A begins when the camera vertical sync signal is received by the first receiver I / F 63A. Furthermore, the output processing begins when the output vertical sync signal is received by the second receiver I / F 63B.

[0140] exist Figure 10 The diagram illustrates an example of the process of reading image data 69 from frame 1 (1F) to frame 12 (12F) and storing it in memory 64, as well as an example of the process of outputting image data 69 in the form of 8 frames. Additionally, Figure 10 The "F" in 1F to 12F refers to the abbreviation for frame. Furthermore, in Figure 10 In the example shown, for ease of illustration, a method is illustrated for storing a maximum of 4 frames of digital image data 69B in memory 64.

[0141] As an example, such as Figure 10 As shown, whenever the vertical synchronization signal for imaging is received by the first receiver I / F63A, one frame of analog image data 69A is read from the photoelectric conversion element 61. Analog image data 69A is read from each horizontal row from the first row to the last row of the photoelectric conversion element 61. When the reading of each row is completed, the pixels of the horizontal row where reading ended are reset. The analog image data 69A is converted into digital image data 69B, and the digital image data 69B is stored distinguishably in the memory 64 in units of one frame, according to the order of the read analog image data 69A frames.

[0142] When the output vertical synchronization signal is received by the second receiver I / F 63B, the digital image data 69B stored in the memory 64 at the time the output vertical synchronization signal is received by the second receiver I / F 63B becomes the output object based on the output circuit 62D and begins to be output. That is, the latest digital image data 69B stored in the memory 64 becomes the output object based on the output circuit 62D and is output to the signal processing unit 50.

[0143] exist Figure 10In this process, the first frame of digital image data 69B output by the output circuit 62D according to the first output vertical synchronization signal is the first frame of digital image data 69B stored in the memory 64 at the time of receiving the first output vertical synchronization signal. Furthermore, the second frame of digital image data 69B output by the output circuit 62D according to the second output vertical synchronization signal is the third frame of digital image data 69B stored in the memory 64 at the time of receiving the first output vertical synchronization signal. Furthermore, the third frame of digital image data 69B output by the output circuit 62D according to the third output vertical synchronization signal is the fifth frame of digital image data 69B stored in the memory 64 at the time of receiving the third output vertical synchronization signal. Furthermore, the fourth frame of digital image data 69B output by the output circuit 62D according to the fourth output vertical synchronization signal is the seventh frame of digital image data 69B stored in the memory 64 at the time of receiving the fourth output vertical synchronization signal.

[0144] Furthermore, the digital image data 69B of the 5th frame output by the output circuit 62D according to the 5th output vertical synchronization signal is the digital image data 69B of the 8th frame stored in the memory 64 at the time of receiving the 5th output vertical synchronization signal. The digital image data 69B of the 6th frame output by the output circuit 62D according to the 6th output vertical synchronization signal is the digital image data 69B of the 9th frame stored in the memory 64 at the time of receiving the 6th output vertical synchronization signal. The digital image data 69B of the 7th frame output by the output circuit 62D according to the 7th output vertical synchronization signal is the digital image data 69B of the 10th frame stored in the memory 64 at the time of receiving the 7th output vertical synchronization signal. Moreover, the digital image data 69B of the 8th frame output by the output circuit 62D according to the 8th output vertical synchronization signal is the digital image data 69B of the 11th frame stored in the memory 64 at the time of receiving the 8th output vertical synchronization signal.

[0145] As an example, such as Figure 11 As shown, the storage circuit 62F has a camera system storage area 62F1 and an output system storage area 62F2. The camera system storage area 62F1 is an example of the "camera system holding section (camera system holding circuit)" according to the technology of the present invention, which holds camera system driving mode indication information indicating the driving mode of the camera system of the imaging element 44. The "camera system" referred to here refers to the readout circuit 62A and the digital processing circuit 62B.

[0146] The output system storage area 62F2 is an example of an "output system holding unit" according to the technology of this invention, which holds output system drive mode indication information indicating the drive mode of the output system of the imaging element 44. Here, "output system" refers to the digital processing circuit 62B, the image processing circuit 62C, the output circuit 62D, and the control circuit 62E. The control circuit 62E controls the drive of the imaging system of the imaging element 44 according to the imaging system drive mode indication information, and controls the drive of the output system of the imaging element 44 according to the output system drive mode indication information.

[0147] The camera system storage area 62F1 is a non-volatile memory capable of having its contents rewritten. Camera system driving mode indication information is stored in the camera system storage area 62F1, and this information can be rewritten by the control circuit 62E. The camera system driving mode indication information includes camera area information, pixel interval rejection information, pixel addition operation method information, exposure time information, conversion gain switching information, analog gain information, and A / D conversion accuracy information. Furthermore, in this embodiment, information including camera area information, pixel interval rejection information, pixel addition operation method information, exposure time information, conversion gain switching information, analog gain information, and A / D conversion accuracy information is exemplified as camera system driving mode indication information; however, the technology of the present invention is not limited to this. That is, the camera system driving mode indication information can be any information that includes at least one of the following: camera area information, pixel interval rejection information, pixel addition operation method information, exposure time information, conversion gain switching information, analog gain information, and A / D conversion accuracy information.

[0148] The imaging area information is information related to the imaging area. As an example of information related to the imaging area, one could cite the light-receiving surface 44A of the imaging element 44 (see reference). Figure 5 Information used in the effective area of ​​the photograph.

[0149] Pixel interval culling information is information related to pixel interval culling. As an example of information related to pixel interval culling, one could provide information indicating that analog image data 69A is read as image data representing a vertically 1 / Y interval-culling image. Here, "Y" refers to a natural number greater than 2. A vertically 1 / Y interval-culling image refers to an image in which horizontal rows of the image are culled at intervals, skipping Y-1 rows in the vertical direction.

[0150] Furthermore, while an example of removing horizontal row spacing in the vertical direction has been given, the technology of the present invention is not limited to this. Vertical row spacing in the horizontal direction can be removed, or both horizontal and vertical row spacing in the vertical direction can be removed. Thus, various spacing removal methods can be considered.

[0151] Pixel addition method information is information related to the pixel addition method. As an example of information related to the pixel addition method, one could include information indicating that the analog image data 69A is read as image data obtained by averaging the pixel values ​​of multiple adjacent pixels. Exposure time information is information related to the exposure time. As an example of information related to the exposure time, one could include information indicating the exposure time performed by the photoelectric conversion element 61.

[0152] Conversion gain switching information relates to conversion gain switching. One example of information related to conversion gain switching is information indicating the switching of the floating diffusion gain of the photoelectric conversion element 61. Analog gain information relates to analog gain. One example of information related to analog gain is information indicating the gain applicable to analog image data 69A. A / D conversion accuracy information relates to A / D conversion accuracy. One example of information related to A / D conversion accuracy is information indicating the accuracy of the A / D converter 62B1 (reference...). Figure 8 The bit precision information of the A / D conversion performed.

[0153] The output system drive mode indication information includes output destination information, digital interval rejection information, digital addition operation method information, frame count information, digital gain information, A / D output bit depth information, and idle bit filling method information. Furthermore, in this embodiment, information including output destination information, digital interval rejection information, digital addition operation method information, frame count information, digital gain information, A / D output bit depth information, and idle bit filling method information is exemplified as output system drive mode indication information, but the technology of the present invention is not limited to this. The output system drive mode indication information can be any information that includes at least one of the following: output destination information, digital interval rejection information, digital addition operation method information, frame count information, digital gain information, A / D output bit depth information, and idle bit filling method information.

[0154] Output destination information is information related to the output destination. As an example of information related to the output destination, an indication of the output destination from output circuit 62D ( Figure 8 The output destination information of the digital image data 69B is displayed. In this embodiment, the output destination of the digital image data 69B is the signal processing unit 50.

[0155] Number interval rejection information is information related to number interval rejection. As an example of information related to number interval rejection, information indicating the processing method for the aforementioned number interval rejection process can be given.

[0156] Information regarding the method of adding numbers refers to information related to the method of adding numbers. As an example of information related to the method of adding numbers, information indicating the processing method for the aforementioned number interval elimination process can be given.

[0157] Frame count information is information related to the average number of output frames. As an example of information related to the average number of output frames, one could cite information indicating the average number of frames per unit time of the digital image data 69B output by the output circuit 62D. In this case, the control circuit 62E controls the output circuit 62D in such a way as the average number of frames per unit time indicated by the frame count information. For example, in such a way as the average number of frames indicated by the frame count information, the control circuit 62E controls the output circuit 62D as follows: limiting the output of the digital image data 69B, or performing the output of the digital image data 69B, regardless of the vertical synchronization signal used for output.

[0158] Digital gain information is information related to digital gain. As an example of information related to digital gain, one could cite information indicating the gain used in the aforementioned digital gain processing.

[0159] A / D output bit information is information related to the A / D output bits. As an example of information related to A / D output bits, information indicating the number of bits of digital image data 69B output by A / D converter 62B1 can be given.

[0160] The information regarding the filling method for low-order free bits is information related to the filling method for low-order free bits. As an example of information related to the filling method for low-order free bits, information indicating the processing method for the aforementioned data embedding process can be given.

[0161] The controller 46 outputs camera system rewrite information, which indicates the camera system drive mode instruction information, to the first receiver I / F63A via communication line 57. This is in accordance with the receiver 84 (reference). Figure 5 Upon receiving the instruction, the controller 46 outputs the camera system rewrite content information to the first receiver I / F 63A. The first receiver I / F 63A receives the camera system rewrite content information output by the controller 46. The control circuit 62E rewrites the content of the camera system drive mode indication information in the camera system storage area 62F1 according to the camera system rewrite content information received by the first receiver I / F 63A.

[0162] Controller 46 outputs the rewritten output system rewrite information, representing the output system drive mode indication information, to the first receiver I / F63A via communication line 57. This is in accordance with the receiver 84 (reference). Figure 5Upon receiving the instruction, the controller 46 outputs the output system rewrite content information to the first receiver I / F 63A. The first receiver I / F 63A receives the output system rewrite content information output by the controller 46. The control circuit 62E rewrites the content of the output system drive mode indication information in the output system storage area 62F2 according to the output system rewrite content information received by the first receiver I / F 63A.

[0163] In addition, for ease of explanation, the term "rewritten content information" will be used below unless it is necessary to distinguish between the rewritten content information of the camera system and the rewritten content information of the output system.

[0164] Next, the operation of the camera device 10 according to this first embodiment will be explained.

[0165] First, refer to Figure 12 The readout and storage process executed by the processing circuit 62 of the imaging element 44 will be described.

[0166] exist Figure 12 In the read-out storage process shown, firstly, in step ST10, the control circuit 62E determines whether a camera vertical synchronization signal has been received by the first receiver I / F 63A. If, in step ST10, the first receiver I / F 63A does not receive a camera vertical synchronization signal, the process is determined to be "no," and the read-out storage process proceeds to step ST16. If, in step ST10, the first receiver I / F 63A receives a camera vertical synchronization signal, the process is determined to be "yes," and the read-out storage process proceeds to step ST12.

[0167] In step ST12, the readout circuit 62A reads one frame of analog image data 69A from the photoelectric conversion element 61. The analog image data 69A read from the photoelectric conversion element 61 by the readout circuit 62A is converted into digital image data 69B by the digital processing circuit 62B and output to the control circuit 62E.

[0168] In the next step ST14, the control circuit 62E stores the digital image data 69B input from the digital processing circuit 62B into the memory 64, and then reads it out and transfers the storage processing to step ST16.

[0169] In step ST16, the control circuit 62E determines whether the condition for ending the read-out storage process (hereinafter referred to as the "read-out storage process end condition") is met. As an example of a read-out storage process end condition, it can be exemplified by the receiving device 84 (see reference 84). Figure 5The condition for receiving an instruction to end the read storage process is as follows: In step ST16, if the read storage process end condition is not met, the condition is determined as "No," and the read storage process proceeds to step ST10. In step ST16, if the read storage process end condition is met, the condition is determined as "Yes," and the read storage process ends.

[0170] Next, refer to Figure 13 The output processing flow executed by the processing circuit 62 of the imaging element 44 will be described.

[0171] exist Figure 13 In the output processing shown, in step ST20, the control circuit 62E determines whether the output vertical synchronization signal has been received by the second receiver I / F 63B. If the second receiver I / F 63B does not receive the output vertical synchronization signal in step ST20, the determination is "No," and the output processing proceeds to step ST26. If the second receiver I / F 63B receives the output vertical synchronization signal in step ST20, the determination is "Yes," and the output processing proceeds to step ST22.

[0172] In step ST22, the control circuit 62E retrieves one frame of the latest digital image data 69B from the memory 64. Here, the latest digital image data 69B refers to the digital image data 69B currently stored in the memory 64. Through the processing of step ST22, the digital image data 69B retrieved by the control circuit 62E is transmitted to the output circuit 62D.

[0173] In the next step ST24, the output circuit 62D outputs the digital image data 69B input from the control circuit 62E from the output I / F 62D to the signal processing unit 50 via the communication line 53, and then the output processing is transferred to step ST26.

[0174] In step ST26, the control circuit 62E determines whether the conditions for ending the output processing are met (hereinafter referred to as the "output processing end condition"). As an example of an output processing end condition, it can be exemplified by the receiving device 84 (see reference 84). Figure 5 The condition for receiving an instruction to end output processing is as follows: In step ST26, if the condition for ending output processing is not met, the result is determined to be "No," and output processing proceeds to step ST20. In step ST26, if the condition for ending output processing is met, the result is determined to be "Yes," and output processing ends.

[0175] Next, refer to Figure 14 The camera system drive processing flow executed by the processing circuit 62 of the imaging element 44 will be described.

[0176] exist Figure 14In the camera system drive process shown, firstly, in step ST30, the control circuit 62E determines whether a vertical synchronization signal for the camera has been received by the first receiving I / F 63A. If, in step ST30, the first receiving I / F 63A does not receive a vertical synchronization signal for the camera, the process is determined to be "no," and the camera system drive process proceeds to step ST36. If, in step ST30, the first receiving I / F 63A receives a vertical synchronization signal for the camera, the process is determined to be "yes," and the camera system drive process proceeds to step ST32.

[0177] In step ST32, the control circuit 62E retrieves data from the camera system storage area 62F1 (reference). Figure 11 The camera system drive mode indication information is obtained, and then the camera system drive processing is transferred to step ST34.

[0178] In step ST34, control circuit 62E drives the camera system according to the driving mode indicated by the camera system driving mode indication information obtained in step ST32, and then the camera system driving process is transferred to step ST36. Readout circuit 62A is driven according to imaging area information, pixel interval culling information, pixel addition operation method information, exposure time information, and analog gain information. Digital processing circuit 62B is driven according to conversion gain switching information and A / D conversion accuracy information.

[0179] In step ST36, it is determined whether the condition for ending the camera system drive processing (hereinafter referred to as the "camera system drive processing end condition") is met. One example of the camera system drive processing end condition is that it is determined by the receiving device 84 (see reference 84). Figure 5 The condition for receiving an instruction to end the camera system drive processing is as follows: In step ST36, if the condition for ending the camera system drive processing is not met, the result is determined to be "No," and the camera system drive processing proceeds to step ST30. In step ST36, if the condition for ending the camera system drive processing is met, the result is determined to be "Yes," and the camera system drive processing ends.

[0180] Next, refer to Figure 15 The output system drive processing flow executed by the processing circuit 62 of the imaging element 44 will be described.

[0181] exist Figure 15In the output system drive process shown, firstly, in step ST40, the control circuit 62E determines whether the output vertical synchronization signal has been received by the second receiver I / F 63B. If the second receiver I / F 63B does not receive the output vertical synchronization signal in step ST40, the process is determined to be "no," and the output system drive process proceeds to step ST46. If the second receiver I / F 63B receives the output vertical synchronization signal in step ST40, the process is determined to be "yes," and the output system drive process proceeds to step ST42.

[0182] In step ST42, the control circuit 62E outputs data from the system storage area 62F2 (reference). Figure 11 Obtain the output system drive mode indication information, and then transfer the output system drive processing to step ST44.

[0183] In step ST44, control circuit 62E drives the output system according to the driving mode indicated by the output system driving mode indication information obtained in step ST42, and then the output system driving process is transferred to step ST46. Digital processing circuit 62B is driven according to digital gain information and A / D output bit information. Image processing circuit 62C is driven according to digital interval elimination information, digital addition operation method information, and idle bit filling method information. Output circuit 62D is driven according to output destination information. Control circuit 62E is driven according to frame count information.

[0184] In step ST46, it is determined whether the condition for ending the output system drive processing (hereinafter referred to as the "output system drive processing end condition") is met. As an example of the output system drive processing end condition, it can be cited that the condition is met by the receiving device 84 (see reference 84). Figure 5 The condition for receiving an instruction to end the output system driver processing is as follows: In step ST46, if the condition for ending the output system driver processing is not met, the result is determined to be "No," and the output system driver processing proceeds to step ST40. In step ST46, if the condition for ending the output system driver processing is met, the result is determined to be "Yes," and the output system driver processing ends.

[0185] Next, refer to Figure 16 The rewrite process executed by the processing circuit 62 of the imaging element 44 will be described.

[0186] exist Figure 16In the rewrite process shown, firstly, in step ST50, the control circuit 62E determines whether the first receiving I / F 63A has received the rewrite content information. If the first receiving I / F 63A has not received the rewrite content information in step ST50, the determination is "no," and the rewrite process proceeds to step ST58. If the first receiving I / F 63A has received the rewrite content information in step ST50, the determination is "yes," and the rewrite process proceeds to step ST52.

[0187] In step ST52, the control circuit 62E determines whether the rewrite content information received by the first receiving I / F 63A in step ST50 is camera system rewrite content information. In step ST52, if the rewrite content information received by the first receiving I / F 63A in step ST50 is camera system rewrite content information, the determination is "yes," and the rewrite process proceeds to step ST54. In step ST52, if the rewrite content information received by the first receiving I / F 63A in step ST50 is not camera system rewrite content information, the determination is "no," and the rewrite process proceeds to step ST56. Furthermore, the case where the rewrite content information received by the first receiving I / F 63A is not camera system rewrite content information refers to the case where the rewrite content information received by the first receiving I / F 63A is output system rewrite content information.

[0188] In step ST54, the control circuit 62E rewrites the content of the camera system drive mode indication information in the camera system storage area 62F1 according to the camera system rewrite content information received by the first receiving I / F63A in step ST50, and then the rewrite process is transferred to step ST58.

[0189] In step ST56, the control circuit 62E rewrites the output system drive mode indication information in the output system storage area 62F2 according to the output system rewrite content information received by the first receiving I / F63A in step ST50, and then the rewrite process is transferred to step ST58.

[0190] In step ST58, the control circuit 62E determines whether the conditions for ending the rewrite process (hereinafter referred to as the "rewrite process end condition") are met. One example of a rewrite process end condition is determined by the receiving device 84 (see reference 84). Figure 5 The condition for receiving an instruction to end the rewrite process is as follows: In step ST58, if the rewrite process end condition is not met, the result is determined to be "No", and the rewrite process proceeds to step ST50. In step ST58, if the rewrite process end condition is met, the result is determined to be "Yes", and the rewrite process ends.

[0191] As explained above, in the camera device 10, the camera synchronization signal is received by the first receiver I / F 63A, and the output synchronization signal is received by the second receiver I / F 63B. Furthermore, digital image data 69B obtained by capturing images at the camera frame rate according to the camera synchronization signal received by the first receiver I / F 63A is stored in the memory 64. And, digital image data 69B is output at the output frame rate according to the output synchronization signal received by the second receiver I / F 63B.

[0192] Therefore, assuming that the processing speed of the signal processing unit 50, which is the output destination of the digital image data 69B, slows down for some reason, the processing load on the signal processing unit 50 can be reduced compared to the current time point by making the output interval of the output synchronization signal longer than the current time point. Conversely, if the processing capacity of the signal processing unit 50 is sufficient, the processing volume in the signal processing unit 50 can be increased compared to the current time point by making the output interval of the output synchronization signal shorter than the current time point.

[0193] Furthermore, if the output interval of the output synchronization signal is extended while the output interval of the video synchronization signal is also extended, unnecessary shooting can be avoided, thus reducing the power consumption required for shooting compared to the current time point. It also prevents exceeding the capacity of the memory 64 within the imaging element 44. Conversely, if the output interval of the output synchronization signal is shortened while the output interval of the video synchronization signal is also shortened, the number of shots can be increased, allowing more digital image data 69B to be processed. Moreover, regardless of the length of the output interval of the output synchronization signal, the output interval of the video synchronization signal can be extended or shortened, thereby allowing the number of shots to be adjusted externally to the imaging element 44. Thus, according to the imaging device 10, shooting and output corresponding to the external conditions of the imaging element 44 can be achieved.

[0194] Furthermore, the imaging device 10 includes a vertical synchronization signal for imaging in the imaging synchronization signal. Therefore, according to the imaging device 10, the readout time of each frame can be adjusted externally to the imaging element 44. Also, if processing in the subsequent circuitry (controller 46 and / or signal processing unit 50, etc.) of the imaging element 44 is delayed for some reason at the current time point, by making the output interval of the vertical synchronization signal for imaging longer than at the current time point, it is possible to prevent the increase of digital image data 69B that cannot be processed in the subsequent circuitry. Furthermore, by making the output interval of the vertical synchronization signal for imaging longer than at the current time point, the number of shots can be reduced compared to the current time point. This reduces the power consumption required for shooting compared to the current time point. It also prevents exceeding the capacity of the memory 64 within the imaging element 44.

[0195] Furthermore, in the imaging device 10, the output synchronization signal includes an output vertical synchronization signal. Therefore, according to the imaging device 10, the output timing of each frame can be adjusted externally to the imaging element 44. Moreover, if processing in the subsequent circuitry of the imaging element 44 is delayed for some reason at the current time, the number of output frames can be reduced by extending the output interval of the output vertical synchronization signal. This reduces the load on the subsequent circuitry of the imaging element 44 compared to the current time. Furthermore, corresponding to the reduction in the number of output frames, the power consumption required for output can be reduced.

[0196] Furthermore, in the imaging device 10, the digital image data 69B stored in the memory 64 at the time the output vertical synchronization signal is received by the second receiver I / F 63B becomes the output target based on the output circuit 62D and output begins. That is, the latest digital image data 69B stored in the memory 64 becomes the output target based on the output circuit 62D and is output to the signal processing unit 50. Therefore, according to the imaging device 10, compared to the case where the digital image data 69B of a frame that is earlier than the time point of receiving the output vertical synchronization signal, the real-time performance between shooting and output can be improved.

[0197] Furthermore, in the camera device 10, when a vertical synchronization signal for imaging is received by the first receiver I / F63A, control is performed to drive the camera system in the driving mode indicated by the camera system driving mode indication information in the camera system storage area 62F1 (see reference). Figure 14 Therefore, according to the camera device 10, the camera system can be driven in a driving mode indicated by the camera system driving mode indication information in the camera system storage area 62F1 for each frame.

[0198] Furthermore, in the camera device 10, when an output vertical synchronization signal is received by the second receiver I / F63B, control is performed to drive the output system in the driving mode indicated by the output system driving mode indication information in the output system storage area 62F2 (see reference). Figure 15 Therefore, according to the camera device 10, the output system can be driven in a driving mode indicated by the output system driving mode indication information in the output system storage area 62F2 for each frame.

[0199] Furthermore, in the camera device 10, when camera system rewrite content information is received by the first receiver I / F63A, the camera system drive mode indication information in the camera system storage area 62F1 is rewritten to the content represented by the camera system rewrite content information (see reference). Figure 11 and Figure 16 Therefore, according to the imaging device 10, the content of the camera system drive mode indication information held in the imaging element 44 can be rewritten from outside the imaging element 44.

[0200] Furthermore, in the camera device 10, the camera system drive mode indication information includes camera area information, pixel interval culling information, pixel addition operation method information, exposure time information, conversion gain switching information, analog gain information, and A / D conversion accuracy information (see reference). Figure 11 Therefore, according to the imaging device 10, it is possible to rewrite from outside the imaging element 44 the contents of the imaging area information, pixel interval elimination information, pixel addition operation method information, exposure time information, conversion gain switching information, analog gain information and A / D conversion accuracy information stored in the imaging element 44.

[0201] Furthermore, in the camera device 10, when the output system rewrite content information is received by the first receiver I / F63A, the output system drive mode indication information in the output system storage area 62F2 is rewritten to the content represented by the output system rewrite content information (see reference). Figure 11 and Figure 16 Therefore, according to the imaging device 10, the content of the output system drive mode indication information held in the imaging element 44 can be rewritten from outside the imaging element 44.

[0202] Furthermore, in the camera device 10, the output system drive mode indication information includes output destination information, digital interval rejection information, digital addition operation method information, frame count information, digital gain information, A / D output bit depth information, and idle bit filling method information (see reference). Figure 11 Therefore, according to the imaging device 10, the contents of the output destination information, digital interval elimination information, digital addition operation method information, frame number information, digital gain information, A / D output bit information and idle bit filling method information stored in the imaging element 44 can be rewritten from outside the imaging element 44.

[0203] Furthermore, the imaging element 44 is an imaging element that integrates the photoelectric conversion element 61, the processing circuit 62, and the memory 64 onto a single chip. Therefore, compared to imaging elements that do not integrate the photoelectric conversion element 61, the processing circuit 62, and the memory 64 onto a single chip, the portability of the imaging element 44 is improved. Moreover, compared to imaging elements that do not integrate the photoelectric conversion element 61, the processing circuit 62, and the memory 64 onto a single chip, design flexibility is increased. In addition, compared to imaging elements that do not integrate the photoelectric conversion element 61, the processing circuit 62, and the memory 64 onto a single chip, it also contributes to the miniaturization of the camera device body 12.

[0204] And, as Figure 7As shown, the imaging element 44 is a stacked imaging element in which a memory 64 is stacked on top of a photoelectric conversion element 61. This shortens the wiring connecting the photoelectric conversion element 61 and the memory 64, reducing wiring delay. Consequently, compared to the case where the photoelectric conversion element 61 and memory 64 are not stacked, the transmission speed of image data 69 from the photoelectric conversion element 61 to the memory 64 is increased. This increased transmission speed also contributes to higher processing speeds in the overall processing circuit 62. Furthermore, compared to the case where the photoelectric conversion element 61 and memory 64 are not stacked, design flexibility is increased. In addition, compared to the case where the photoelectric conversion element 61 and memory 64 are not stacked, miniaturization of the camera device body 12 is also possible.

[0205] Furthermore, in the camera device 10, a real-time preview image based on digital image data 69B is displayed on the second display 86. This allows the user to visually recognize the image represented by the digital image data 69B.

[0206] Furthermore, in the first embodiment described above, the imaging element 44 is exemplified as an imaging element formed by integrating the photoelectric conversion element 61, the processing circuit 62, and the memory 64 onto a single chip; however, the technology of the present invention is not limited to this. For example, it is sufficient to integrate at least the photoelectric conversion element 61 and the memory 64 into a single chip.

[0207] Furthermore, in the first embodiment described above, the image processing circuit 63C performs image processing on the digital image data 69B obtained through A / D conversion and outputs the processed digital image data 69B. However, the technology of the present invention is not limited to this. Alternatively, the digital image data 69B may not be processed, and the output circuit 62D outputs the digital image data 69B itself. In this case, instead of the image processing circuit 63C, the signal processing unit 50 and / or the controller 46, which are subsequent circuits of the imaging element 44, can perform image processing on the digital image data 69B.

[0208] Furthermore, in the first embodiment described above, a wired communication method was used between the imaging element 44 and the signal processing unit 50 via communication lines 53 and 55. However, the technology of the present invention is not limited to this. For example, communication between the imaging element 44 and the signal processing unit 50 may also be wireless. Similarly, communication between the imaging element 44 and the controller 46 may also be wireless, as may be between the signal processing unit 50 and the controller 46.

[0209] [Second Implementation]

[0210] In the first embodiment described above, an example was given in which the camera synchronization signal includes a camera vertical synchronization signal, and the output synchronization signal includes an output vertical synchronization signal. In this second embodiment, a case is described in which the camera synchronization signal also includes a camera horizontal synchronization signal, and the output synchronization signal also includes an output horizontal synchronization signal. Furthermore, in this second embodiment, the same reference numerals are used for components identical to those in the first embodiment, and their descriptions are omitted. Hereinafter, the parts that differ from those in the first embodiment will be described.

[0211] As an example, such as Figure 17 As shown, the difference between the camera device 10 according to this second embodiment and the first embodiment described above is that the camera synchronization signal further includes a horizontal synchronization signal for camera recording. Furthermore, as an example, such as... Figure 17 As shown, the difference between the camera device 10 according to this second embodiment and the first embodiment described above is that the output synchronization signal also includes an output horizontal synchronization signal. Furthermore, in the description of this second embodiment, for ease of explanation, the camera device 10 according to this second embodiment will be simply referred to as "camera device 10".

[0212] The horizontal synchronization signal for video recording specifies the start time for reading out the analog image data 69A of 1 horizontal line. The horizontal synchronization signal for output specifies the start time for outputting the analog image data 69A of 1 horizontal line.

[0213] The controller 46 receives the I / F63A via communication line 57 and outputs a horizontal synchronization signal for camera use. The signal processing unit 50 receives the I / F63B via communication line 55 and outputs a horizontal synchronization signal for output.

[0214] The first receiver I / F63A receives vertical synchronization signals for video recording on a per-frame basis, and receives multiple horizontal synchronization signals for video recording output from the controller 46 via the communication line 57 between adjacent frames.

[0215] The first receiver I / F 63A outputs the received horizontal sync signal for video recording to the control circuit 62E. The control circuit 62E transmits the horizontal sync signal for video recording input from the controller 46 via the first receiver I / F 63A to the readout circuit 62A. When the horizontal sync signal for video recording is input from the control circuit 62E, the readout circuit 62A begins to read out 1 horizontal line of analog image data 69A from the photoelectric conversion element 61.

[0216] The second receiver I / F63B receives vertical synchronization signals for output in each frame and receives multiple horizontal synchronization signals for output from the signal processing unit 50 via the communication line 55 between adjacent frames.

[0217] The second receiver I / F 63B outputs the received output horizontal synchronization signal to the output circuit 62D. When the output horizontal synchronization signal is input from the signal processing unit 50 via the second receiver I / F 63B, the output circuit 62D starts outputting 1 horizontal line of digital image data 69B from the output I / F 62D1 to the signal processing unit 50.

[0218] Next, the function of the camera device 10 will be explained.

[0219] First, refer to Figure 18 The readout and storage process executed by the processing circuit 62 of the imaging element 44 will be described.

[0220] exist Figure 18 In the read-out storage process shown, firstly, in step ST100, the control circuit 62E determines whether a camera vertical synchronization signal has been received by the first receiving I / F 63A. If, in step ST100, the first receiving I / F 63A does not receive the camera vertical synchronization signal, the process is determined to be "no," and the read-out storage process proceeds to step ST114. If, in step ST100, the first receiving I / F 63A receives the camera vertical synchronization signal, the process is determined to be "yes," and the read-out storage process proceeds to step ST102. When the process is determined to be "yes" in step ST100, the camera vertical synchronization signal is transmitted to the read-out circuit 62A.

[0221] In step ST102, the readout circuit 62A, according to the instruction of the control circuit 62E, resets the frame of the analog image data 69A that is the readout target from the photoelectric conversion element 61. That is, the readout circuit 62A sets the first horizontal row of all horizontal rows included in the photoelectric conversion element 61 as the readout target row of the analog image data 69A.

[0222] In the next step ST104, the control circuit 62E determines whether a camera horizontal synchronization signal has been received by the first receiver I / F 63A. If the first receiver I / F 63A does not receive the camera horizontal synchronization signal in step ST104, the determination is "No," and step ST104 is repeated. If the first receiver I / F 63A receives the camera horizontal synchronization signal in step ST104, the determination is "Yes," and the read-out storage process proceeds to step ST106. When the determination is "Yes" in step ST104, the camera horizontal synchronization signal is transmitted to the read-out circuit 62A.

[0223] In step ST106, the readout circuit 62A reads the analog image data 69A of the Nth row from the photoelectric conversion element 61 as analog image data 69A of a horizontal row quantity. Here, "N" in "Nth row" refers to the value of the horizontal row, that is, the address of the horizontal row. The initial value of "N" is "1".

[0224] The analog image data 69A of one horizontal line read from the photoelectric conversion element 61 by the readout circuit 62A is converted into digital image data 69B by the digital processing circuit 62B and output to the control circuit 62E.

[0225] In the next step ST108, the control circuit 62E stores the digital image data 69B input from the digital processing circuit 62B into the memory 64, and then reads it out and transfers the storage processing to step ST110.

[0226] In step ST110, the control circuit 62E determines whether the processing of steps ST104 to ST108 has been completed up to the horizontal line of the final row. If, in step ST110, the processing of steps ST104 to ST108 has been completed up to the horizontal line of the final row, the determination is "yes," and the read-out storage process proceeds to step ST114. If, in step ST110, the processing of steps ST104 to ST108 has not been completed up to the horizontal line of the final row, the determination is "no," and the read-out storage process proceeds to step ST112.

[0227] In step ST112, the control circuit 62E increments N by 1, and then reads the storage process and transfers it to step ST104.

[0228] In step ST114, similarly to the first embodiment described above, the control circuit 62E determines whether the read-to-store processing end condition is met. In step ST114, if the read-to-store processing end condition is not met, the determination is "No," and the read-to-store processing proceeds to step ST100. In step ST114, if the read-to-store processing end condition is met, the determination is "Yes," and the read-to-store processing ends.

[0229] Next, refer to Figure 19 The output processing flow executed by the processing circuit 62 of the imaging element 44 will be described.

[0230] exist Figure 19In the output processing shown, in step ST120, the control circuit 62E determines whether the output vertical synchronization signal has been received by the second receiver I / F 63B. If the second receiver I / F 63B does not receive the output vertical synchronization signal in step ST120, the determination is "No," and the output processing proceeds to step ST134. If the second receiver I / F 63B receives the output vertical synchronization signal in step ST120, the determination is "Yes," and the output processing proceeds to step ST122.

[0231] In step ST122, the control circuit 62E resets the frame of the digital image data 69B that is to be retrieved from the memory 64. That is, the control circuit 62E sets the first horizontal row of all horizontal rows included in the latest digital image data 69B in the memory 64 as the read-out row of the digital image data 69B. Here, similar to the first embodiment described above, the latest digital image data 69B refers to the digital image data 69B that is currently stored in the memory 64.

[0232] In the next step ST124, the control circuit 62E determines whether the output horizontal synchronization signal has been received by the second receiver I / F 63B. If the second receiver I / F 63B does not receive the output horizontal synchronization signal in step ST124, the determination is "No," and step ST124 is repeated. If the second receiver I / F 63B receives the output horizontal synchronization signal in step ST124, the determination is "Yes," and the output processing proceeds to step ST126. When the determination is "Yes" in step ST124, the output horizontal synchronization signal is transmitted from the second receiver I / F 63B to the control circuit 62E via the output circuit 62D.

[0233] In the next step ST126, the control circuit 62E retrieves the digital image data 69B of the horizontal row of the Nth row included in the latest digital image data 69B from the memory 64, and transmits the retrieved digital image data 69B of the horizontal row of the Nth row to the output circuit 62D.

[0234] In the next step ST128, the output circuit 62D outputs the digital image data 69B of the horizontal row of the Nth row, which was input from the control circuit 62E, from the output I / F 62D to the signal processing unit 50 via the communication line 53.

[0235] In the next step ST130, the control circuit 62E determines whether the processing of steps ST124 to ST128 has been completed up to the horizontal line of the final row. If in step ST130 the processing of steps ST124 to ST128 has been completed up to the horizontal line of the final row, the determination is "yes," and the output processing proceeds to step ST134. If in step ST130 the processing of steps ST124 to ST128 has not been completed up to the horizontal line of the final row, the determination is "no," and the output processing proceeds to step ST132.

[0236] In step ST132, the control circuit 62E increments N by 1, and then the output processing is transferred to step ST124.

[0237] In step ST134, similarly to the first embodiment described above, the control circuit 62E determines whether the output processing end condition is met. In step ST134, if the output processing end condition is not met, the determination is "No," and the output processing proceeds to step ST120. In step ST134, if the output processing end condition is met, the determination is "Yes," and the output processing ends.

[0238] As explained above, in the imaging device 10, the imaging synchronization signal includes a horizontal synchronization signal for imaging. Therefore, according to the imaging device 10, the readout time of the horizontal line can be adjusted from outside the imaging element 44. Furthermore, in this second embodiment, an example has been described where the imaging synchronization signal includes both a vertical synchronization signal and a horizontal synchronization signal for imaging. However, it is also possible for the imaging synchronization signal to include only the horizontal synchronization signal from the vertical and horizontal synchronization signals. In this case, the vertical synchronization signal for imaging is received from the controller 46 via another communication line by the first receiving I / F 63A or other receiving I / F, and the received vertical synchronization signal is transmitted to the control circuit 62E.

[0239] Furthermore, in the imaging device 10, the output synchronization signal includes an output horizontal synchronization signal. Therefore, according to the imaging device 10, the output timing of each horizontal line can be adjusted from outside the imaging element 44. In this second embodiment, an example has been described where the output synchronization signal includes both an output vertical synchronization signal and an output horizontal synchronization signal; however, it is also possible for the output synchronization signal to include only the output horizontal synchronization signal. In this case, the output vertical synchronization signal is received from the signal processing unit 50 via another communication line by the second receiving I / F 63B or other receiving I / F, and the received output vertical synchronization signal is transmitted to the control circuit 62E via the output circuit 62D.

[0240] [Third Implementation]

[0241] In the first and second embodiments described above, an example was given of using digital image data 69B stored in memory 64 at the time the output vertical synchronization signal is received by the second receiving I / F 63B as the output object based on output circuit 62D. In this third embodiment, the case of using one frame of digital image data 69B that has already been stored in memory 64 as the output object based on output circuit 62D will be described. In addition, in this third embodiment, the same reference numerals are used for the same components as in the first and second embodiments described above, and their descriptions are omitted. Hereinafter, the parts that are different from those in the first and second embodiments described above will be described. Furthermore, in the description of this third embodiment, for ease of explanation, the camera device 10 involved in this third embodiment will be simply referred to as "camera device 10".

[0242] As an example, such as Figure 20 As shown, in the camera device 10, when the output vertical synchronization signal is received by the second receiver I / F 63B, the output circuit 62D outputs the latest frame of digital image data 69B that has been stored in the memory 64.

[0243] exist Figure 20 In the example shown, at the moment the first output vertical synchronization signal is received by the second receiving I / F 63B, the digital image data 69B of the second frame is being stored in memory 64. At this time, the digital image data 69B of the first frame, which is one frame earlier than the digital image data 69B of the second frame currently being stored in memory 64, has already been stored in memory 64. In this case, the digital image data 69B of the first frame output by the output circuit 62D is the digital image data 69B of the first frame already stored in memory 64 at the moment the first output vertical synchronization signal is received by the second receiving I / F 63B.

[0244] That is, when the first output vertical synchronization signal is received by the second receiving I / F 63B, the output circuit 62D outputs the digital image data 69B of the first frame that has been stored in the memory 64.

[0245] At the moment when the second vertical synchronization signal for output is received by the second receiving I / F 63B, the digital image data 69B of the fourth frame is being stored in memory 64. At this time, the digital image data 69B of the third frame, which is one frame earlier than the digital image data 69B currently being stored in memory 64, has already been stored in memory 64. In this case, the digital image data 69B of the second frame output by the output circuit 62D is the digital image data 69B of the third frame that has already been stored in memory 64 at the moment when the second receiving I / F 63B receives the second vertical synchronization signal for output.

[0246] That is, when the second vertical synchronization signal for output is received by the second receiver I / F63B, the output circuit 62D outputs the digital image data 69B of the third frame that has been stored in the memory 64.

[0247] At the moment when the third vertical synchronization signal for output is received by the second receiving I / F 63B, the digital image data 69B of the sixth frame is being stored in memory 64. At this time, the digital image data 69B of the fifth frame, which is one frame earlier than the digital image data 69B of the sixth frame currently being stored in memory 64, has already been stored in memory 64. In this case, the digital image data 69B of the third frame output by the output circuit 62D is the digital image data 69B of the fifth frame that has already been stored in memory 64 at the moment when the third vertical synchronization signal for output is received by the second receiving I / F 63B.

[0248] That is, when the third vertical synchronization signal is received by the second receiver I / F 63B, the output circuit 62D outputs the digital image data 69B of the fifth frame that has been stored in the memory 64.

[0249] At the moment when the fourth vertical synchronization signal for output is received by the second receiving I / F 63B, the digital image data 69B of the 8th frame is being stored in memory 64. At this time, the digital image data 69B of the 7th frame, which is one frame earlier than the digital image data 69B currently being stored in memory 64, has already been stored in memory 64. In this case, the digital image data 69B of the 4th frame output by the output circuit 62D is the digital image data 69B of the 7th frame that was already stored in memory 64 at the moment when the fourth vertical synchronization signal for output is received by the second receiving I / F 63B.

[0250] That is, when the fourth vertical synchronization signal for output is received by the second receiving I / F 63B, the output circuit 62D outputs the digital image data 69B of the seventh frame that has been stored in the memory 64.

[0251] Next, refer to Figure 21 The output processing flow executed by the processing circuit 62 of the imaging element 44 will be described.

[0252] exist Figure 21 In the output processing shown, in step ST200, the control circuit 62E determines whether the output vertical synchronization signal has been received by the second receiver I / F 63B. If the second receiver I / F 63B does not receive the output vertical synchronization signal in step ST200, the determination is "No," and the output processing proceeds to step ST214. If the second receiver I / F 63B receives the output vertical synchronization signal in step ST200, the determination is "Yes," and the output processing proceeds to step ST202.

[0253] In step ST202, the control circuit 62E sets the latest frame of digital image data 69B already stored in the memory 64 as the frame of digital image data 69B to be acquired. That is, the control circuit 62E sets the first horizontal row of all horizontal rows included in the latest digital image data 69B in the memory 64 as the row to be read from. Furthermore, here, the latest frame of digital image data 69B refers to digital image data 69B that was stored in the memory 64 one frame earlier than the digital image data 69B currently being stored in the memory 64.

[0254] In the next step ST204, the control circuit 62E determines whether the output horizontal synchronization signal has been received by the second receiver I / F 63B. If the second receiver I / F 63B does not receive the output horizontal synchronization signal in step ST204, the determination is "No," and step ST204 is repeated. If the second receiver I / F 63B receives the output horizontal synchronization signal in step ST204, the determination is "Yes," and the output processing proceeds to step ST206. When the determination is "Yes" in step ST204, the output horizontal synchronization signal is transmitted from the second receiver I / F 63B to the control circuit 62E via the output circuit 62D.

[0255] In the next step ST206, the control circuit 62E acquires the digital image data 69B of the horizontal row of the Nth row included in the latest frame of digital image data 69B that has been stored in the memory 64, and transmits the acquired digital image data 69B of the horizontal row of the Nth row to the output circuit 62D.

[0256] In the next step ST208, the output circuit 62D outputs the digital image data 69B of the horizontal row of the Nth row input from the control circuit 62E from the output I / F 62D to the signal processing unit 50 via the communication line 53.

[0257] In the next step ST210, the control circuit 62E determines whether the processing of steps ST204 to ST208 has been completed up to the horizontal line of the final row. If in step ST210 the processing of steps ST204 to ST208 has been completed up to the horizontal line of the final row, the determination is "yes," and the output processing proceeds to step ST214. If in step ST210 the processing of steps ST204 to ST208 has not been completed up to the horizontal line of the final row, the determination is "no," and the output processing proceeds to step ST212.

[0258] In step ST212, the control circuit 62E increments N by 1, and then the output processing is transferred to step ST204.

[0259] In step ST214, similarly to the first embodiment described above, the control circuit 62E determines whether the output processing end condition is met. If the output processing end condition is not met in step ST214, the determination is "No," and the output processing proceeds to step ST200. If the output processing end condition is met in step ST214, the determination is "Yes," and the output processing ends.

[0260] As explained above, in the camera device 10, one frame of digital image data 69B that is stored in the memory 64 one frame earlier than the time when the vertical synchronization signal for output is received by the second receiver I / F 63B is used as the output target based on the output circuit 62D. When the digital image data 69B that is being stored in the memory 64 is set as the output target, output begins even when one frame of digital image data 69B is not stored in the memory 64, thus requiring a wait until all one frame of digital image data 69B is stored in the memory 64. In contrast, as long as one frame of digital image data is already stored in the memory 64, all one frame of digital image data can be output without waiting. Therefore, according to the camera device 10, output stagnation of digital image data 69B can be avoided.

[0261] [Fourth Implementation]

[0262] In the third embodiment described above, an example was given of outputting only one frame of digital image data 69B already stored in memory 64. In this fourth embodiment, a case is described where both one frame of digital image data 69B already stored in memory 64 and digital image data 69B currently being stored in memory 64 are selectively output. Furthermore, in this fourth embodiment, components identical to those in the first to third embodiments are labeled with the same symbols, and their descriptions are omitted. Hereinafter, the parts that differ from the first to third embodiments will be described. In the description of this fourth embodiment, for ease of explanation, the imaging device 10 involved in this fourth embodiment will be simply referred to as "imaging device 10".

[0263] As an example, such as Figure 22 As shown, the difference between the storage circuit 62F used in the first to third embodiments described above and the storage circuit 62F of the imaging element 44 included in the camera device 10 is that it has an image data related information storage area 62F3.

[0264] The image data-related information storage area 62F3 stores information related to the digital image data 69B. Specifically, the image data-related information storage area 62F3 stores the frame read time and the total number of lines.

[0265] One frame readout time refers to the time it takes for one frame of image data 69 to be read into the imaging element 44. As an example of one frame readout time, it can be given as the time from the start to the end of the imaging process described in the first embodiment above for one frame. The time from the start to the end of the imaging process, for example, refers to the time from the start of the first receiving I / F63A (reference...) Figure 17 The time required from receiving the vertical synchronization signal for the camera to storing one frame of digital image data 69B obtained through shooting in memory 64.

[0266] The total number of rows refers to the total number of horizontal rows included in the photoelectric conversion element 61 that are the readout objects of the analog image data 69A. In this fourth embodiment, for ease of explanation, the total number of horizontal rows that are the readout objects of the analog image data 69A is the number of all horizontal rows included in the photoelectric conversion element 61. Each horizontal row is assigned a row address that can determine the horizontal row. Here, the row address that can determine the horizontal row is the row number that indicates which row each horizontal row from the first row to the last row is.

[0267] As an example, such as Figure 23As shown, the control circuit 62E includes an acquisition unit 62E1, a calculation unit 62E2, and a determination unit 62E3. The storage circuit 62F and the output circuit 62D are connected to the acquisition unit 62E1.

[0268] The vertical synchronization signal output from the signal processing unit 50 via the communication line 55 is received by the second receiver I / F 63B and transmitted to the acquisition unit 62E1 via the output circuit 62D. When the vertical synchronization signal is input from the output circuit 62D, the acquisition unit 62E1 acquires the frame read-in time and total number of lines from the image data related information storage area 62F3. Furthermore, when the vertical synchronization signal is input from the output circuit 62D, the acquisition unit 62E1 acquires the row address (hereinafter referred to as the "read-out row address") of the horizontal line read from the photoelectric conversion element 61 at the current time point from the read-out circuit 62A. The acquisition unit 62E1 outputs the acquired frame read-in time, total number of lines, and read-out row address to the calculation unit 62E2.

[0269] The calculation unit 62E2 performs a first calculation process, a second calculation process, and a third calculation process based on the frame read-in time, total number of lines, and read-out line address input from the acquisition unit 62E1.

[0270] In the first calculation process, the horizontal line read-in time is calculated. The horizontal line read-in time refers to the time it takes for one horizontal line of image data 69 to be read into the imaging element 44. As an example of the horizontal line read-in time, the time from the start to the end of the imaging process described in the first embodiment above can be given. The calculation unit 62E2 calculates the horizontal line read-in time according to the following formula (1).

[0271] (Horizontal line read time) = (1 frame read time) / (Total number of lines) ... (1)

[0272] In the second calculation process, the image data reading completion time is calculated. The image data reading completion time refers to the remaining time required until the storage of one frame of digital image data 69B currently stored in memory 64 is completed. The image data reading completion time is calculated by the calculation unit 62E2 according to the following formula (2). In addition, "current time point" is added as an independent variable in formula (2), but "current time point" can also be "0".

[0273] (Image data reading completion time) = (current time) + (horizontal row reading time) × {(total number of rows) - (reading middle row address)}……(2)

[0274] In the third calculation process, the image data output completion time is calculated. The image data output completion time refers to the time required from the start of outputting one frame of digital image data 69B, which is the output target at the current time point, to its completion. The image data output completion time is an example of "the predicted output completion time as the time for the output of one frame of image data" involved in the technology of the present invention. The image data output completion time is calculated by the calculation unit 62E2 according to the following formula (3). In the following formula (3), "the period of the output horizontal synchronization signal" is the period of the output horizontal synchronization signal received by the second receiving I / F 63B.

[0275] (Image data output completion time) = (current time) + (period of horizontal synchronization signal used for output) × (total number of lines) ... (3)

[0276] The output horizontal synchronization signal received by the second receiver I / F63B is transmitted to the calculation unit 62E2. Therefore, the "period of the output horizontal synchronization signal" is determined by inputting the output horizontal synchronization signal twice into the calculation unit 62E2. That is, the time from the input of the first output horizontal synchronization signal to the input of the second output horizontal synchronization signal is the "period of the output horizontal synchronization signal". In addition, in equation (3), the "current time point" is added as an independent variable, but the "current time point" can also be "0".

[0277] As an example, such as Figure 24 As shown, the determination unit 62E3 obtains the image data read-in completion time and the image data output completion time from the calculation unit 62E2. Furthermore, based on the image data read-in completion time and image data output completion time obtained from the calculation unit 62E2, the determination unit 62E3 determines either the first image data or the second image data as the output target for digital image data 69B. Here, the first image data refers to the digital image data 69B currently stored in the memory 64. The second image data refers to the digital image data 69B that was stored in the memory 64 one frame earlier than the digital image data 69B currently stored in the memory 64. Additionally, the first image data is an example of "latest image data" and "one frame of image data related to the subject captured at the current time" according to the technology of this invention, and the second image data is an example of "latest image data" and "one frame of image data already stored in the memory" according to the technology of this invention. "Digital image data 69B that was stored in memory 64 one frame earlier than the digital image data 69B that is currently stored in memory 64" is an example of "the latest image data stored in the memory unit" that is involved in the technology of the present invention.

[0278] If the image data reading completion time is shorter than the image data output completion time, the determination unit 62E3 determines the first image data as the digital image data 69B to be output. If the image data reading completion time is longer than the image data output completion time, the determination unit 62E3 determines the second image data as the digital image data 69B to be output.

[0279] Next, refer to Figure 25A and Figure 25B The process of output processing performed by the processing circuit 62 of the imaging element 44 will be described.

[0280] exist Figure 25A In the output processing shown, in step ST300, the acquisition unit 62E1 determines whether an output vertical synchronization signal has been input. If no output vertical synchronization signal has been input in step ST300, the determination is "no," and the output processing proceeds to step ST338 (see reference). Figure 25B In step ST300, if an output vertical synchronization signal is input, the condition is determined to be "yes", and the output processing proceeds to step ST302.

[0281] In step ST302, the acquisition unit 62E1 retrieves image data related information from the image data related information storage area 62F3 (reference). Figure 22 and Figure 23 ) Obtain the read time of 1 frame, and then transfer the output processing to step ST304.

[0282] In step ST304, the acquisition unit 62E1 retrieves image data related information from the image data related information storage area 62F3 (reference). Figure 22 and Figure 23 Get the total number of rows, and then transfer the output processing to step ST306.

[0283] In step ST306, the acquisition unit 62E1 acquires the read row address from the read circuit 62A, and then the output processing is transferred to step ST308.

[0284] In step ST308, the calculation unit 62E2 determines whether an output horizontal synchronization signal has been input. If no output horizontal synchronization signal has been input in step ST308, the determination is "No," and step ST308 is repeated. If an output horizontal synchronization signal has been input in step ST308, the determination is "Yes," and the process proceeds to step ST310.

[0285] In step ST310, the calculation unit 62E2 begins to measure the period of the horizontal synchronization signal, and then the output processing is transferred to step ST312. The start of measuring the period of the horizontal synchronization signal refers to the start of timing based on the timer (not shown).

[0286] In the next step ST312, the calculation unit 62E2 determines whether an output horizontal synchronization signal has been input. If no output horizontal synchronization signal has been input in step ST312, the determination is "No," and step ST312 is performed again. If an output horizontal synchronization signal has been input in step ST312, the determination is "Yes," and the process proceeds to step ST314.

[0287] In step ST314, the calculation unit 62E2 finishes measuring the period of the output horizontal synchronization signal, and then the output processing proceeds to step ST316. The end of the measurement of the period of the output horizontal synchronization signal refers to the end of the timing based on the timer mentioned above. The period of the output horizontal synchronization signal is the time from the execution of the processing in step ST310 to the execution of the processing in step ST314, that is, the time measured by the timer mentioned above.

[0288] In step ST316, the calculation unit 62E2 calculates the horizontal line read-in time according to the above formula (1), and then the output processing is transferred to step ST318. In this step ST316, the frame read-in time obtained in step ST302 and the total number of lines obtained in step ST304 are used as the independent variables of the above formula (1).

[0289] In step ST318, the calculation unit 62E2 calculates the image data read-in completion time according to the above formula (2), and then the output processing is transferred to step ST320. In this step 318, the independent variables of the above formula (2) are the current time point, the horizontal row read-in time calculated in step ST316, the total number of rows obtained in step ST304, and the row address obtained in step ST306.

[0290] In step ST320, the calculation unit 62E2 calculates the image data output completion time according to the above formula (3), and then the output processing is transferred to step ST322. In this step ST320, the independent variables of the above formula (3) are the current time point, the period measured by the processing in steps ST310 and ST314, and the total number of rows obtained in step ST304.

[0291] exist Figure 25BIn step ST322, the determining unit 62E3 determines whether the image data read-in completion time calculated in step ST318 is shorter than the image data output completion time calculated in step ST320. In step ST322, if the image data read-in completion time calculated in step ST318 is shorter than the image data output completion time calculated in step ST320, the determination is "yes," and the output processing proceeds to step ST324. In step ST322, if the image data read-in completion time calculated in step ST318 is not shorter than the image data output completion time calculated in step ST320, the determination is "yes," and the output processing proceeds to step ST326.

[0292] Furthermore, the case where "yes" is determined in step ST322 is an example of "the case where the storage of one frame of image data related to the subject captured at the current time point to the storage unit is completed within the output completion time," which is covered by the technology of the present invention. Conversely, the case where "no" is determined in step ST322 is an example of "the case where the storage of one frame of image data related to the subject captured at the current time point to the storage unit is not completed within the output completion time," which is covered by the technology of the present invention.

[0293] In step ST324, the determination unit 62E3 determines the first image data as the digital image data 69B of the output object, and then the output processing is transferred to step ST328.

[0294] In step ST326, the determination unit 62E3 determines the second image data as the digital image data 69B to be output, and then the output processing is transferred to step ST328.

[0295] In the next step ST328, the control circuit 62E determines whether the output horizontal synchronization signal has been received by the second receiver I / F 63B. If the second receiver I / F 63B does not receive the output horizontal synchronization signal in step ST328, the determination is "No," and step ST328 is repeated. If the second receiver I / F 63B receives the output horizontal synchronization signal in step ST328, the determination is "Yes," and the output processing proceeds to step ST330. When the determination is "Yes" in step ST328, the output horizontal synchronization signal is transmitted from the second receiver I / F 63B to the control circuit 62E via the output circuit 62D.

[0296] In the next step ST330, when the first image data is determined to be digital image data 69B as the output target in step ST324, the control circuit 62E acquires the digital image data 69B of the Nth horizontal row included in the first image data. Furthermore, when the second image data is determined to be digital image data 69B as the output target in step ST326, the control circuit 62E acquires the digital image data 69B of the Nth horizontal row included in the second image data. The acquired digital image data 69B of the Nth horizontal row is then transmitted by the control circuit 62E to the output circuit 62D.

[0297] In the next step ST332, the output circuit 62D outputs the digital image data 69B of the horizontal row of the Nth row input from the control circuit 62E from the output I / F 62D to the signal processing unit 50 via the communication line 53.

[0298] In the next step ST334, the control circuit 62E determines whether the processing of steps ST328 to ST332 has been completed up to the horizontal line of the final row. If, in step ST334, the processing of steps ST328 to ST332 has been completed up to the horizontal line of the final row, the determination is "yes," and the output processing proceeds to step ST338. If, in step ST334, the processing of steps ST328 to ST332 has not been completed up to the horizontal line of the final row, the determination is "no," and the output processing proceeds to step ST336.

[0299] In step ST336, the control circuit 62E increments N by 1, and then the output processing is transferred to step ST328.

[0300] In step ST338, similarly to the first embodiment described above, the control circuit 62E determines whether the output processing end condition is met. In step ST338, if the output processing end condition is not met, the determination is "No," and the output processing proceeds to step ST300. In step ST338, if the output processing end condition is met, the determination is "Yes," and the output processing ends.

[0301] As explained above, in the camera device 10, if the image data reading completion time is shorter than the image data output completion time, the first image data is output. Furthermore, if the image data reading completion time is longer than the image data output completion time, the second image data is output. Therefore, according to the camera device 10, both real-time performance between shooting and output can be balanced, and output stagnation of digital image data can be avoided.

[0302] Furthermore, in the camera device 10, the second image data is digital image data 69B that was stored in memory 64 one frame earlier than the digital image data 69B currently stored in memory 64. Therefore, according to the camera device 10, compared to the case where the output is digital image data 69B that was stored in memory 64 several frames earlier than the digital image data 69B currently stored in memory 64, real-time performance between shooting and output can be ensured.

[0303] Furthermore, in the imaging device 10, the image data output completion time is a time predicted based on the period of the output horizontal synchronization signal received by the second receiving I / F63B. Therefore, according to the imaging device 10, the image data output completion time can be accurately determined compared to the case where the image data output completion time is determined using a signal other than the output horizontal synchronization signal.

[0304] Furthermore, in the fourth embodiment described above, the second image data exemplified is digital image data 69B that was stored in memory 64 one frame earlier than the digital image data 69B currently stored in memory 64. However, the technology of the present invention is not limited to this. Even if the second image data is digital image data 69B that was stored in memory 64 multiple frames earlier than the digital image data 69B currently stored in memory 64, the technology of the present invention still applies.

[0305] [Fifth Implementation]

[0306] In the first to fourth embodiments described above, an example was given in which only the signal processing unit 50 was connected to the processing circuit 62 of the imaging element 44. In this fifth embodiment, an example in which two signal processing units are connected to the processing circuit 62 of the imaging element 44 will be described. In addition, in this fifth embodiment, the same reference numerals are used for the same components as in the first to fourth embodiments, and their descriptions are omitted. Hereinafter, the parts that are different from those in the first to fourth embodiments will be described. Furthermore, in the description of this fifth embodiment, for ease of explanation, the imaging device 10 involved in this fifth embodiment will be simply referred to as "imaging device 10".

[0307] As an example, such as Figure 26 As shown, a first signal processing unit 50A is connected to the processing circuit 62 via communication lines 53A and 55A. A second signal processing unit 50B is connected to the processing circuit 62 via communication lines 53B and 55B. The first signal processing unit 50A is connected to the controller 46 via communication line 60A. Furthermore, the second signal processing unit 50B is connected to the controller 46 via communication line 60B.

[0308] As an example, such as Figure 27 As shown, the first signal processing unit 50A is connected to the second receiving I / F 63B via communication line 55A and to the output I / F 62D1 via communication line 53A. Furthermore, the first signal processing unit 50A is connected to the controller 46 via communication line 60A. The first signal processing unit 50A is a device equivalent to the signal processing unit 50 used in the first to fourth embodiments described above. Therefore, the same input / output as that between the output circuit 62D and the first signal processing unit 50A is performed between them. Similarly, the same input / output as that between the signal processing unit 50 and the output circuit 62D is performed between the first signal processing unit 50A and the controller 46.

[0309] The processing circuit 62 includes a third receiving I / F 63C. The second signal processing unit 50B is connected to the third receiving I / F 63C via a communication line 55B. Furthermore, the output circuit 62D is connected to the third receiving I / F 63C. In addition, the second signal processing unit 50B, like the signal processing unit 50, outputs an output synchronization signal to the third receiving I / F 63C via the communication line 55B. The output synchronization signal output to the third receiving I / F 63C includes an output vertical synchronization signal and an output horizontal synchronization signal.

[0310] The output circuit 62D includes an output I / F 62D2. The output I / F 62D2 is connected to the second signal processing unit 50B via a communication line 53B. Furthermore, the second signal processing unit 50B is connected to the controller 46 via a communication line 60B.

[0311] The second signal processing unit 50B is a device that has the same functions as the first signal processing unit 50A. Therefore, the same input / output occurs between the output circuit 62D and the second signal processing unit 50B as it does between the output circuit 62D and the first signal processing unit 50A. Furthermore, the same input / output occurs between the second signal processing unit 50B and the controller 46 as it does between the first signal processing unit 50A and the controller 46.

[0312] Furthermore, in this fifth embodiment, outputs I / F 62D1 and 62D2 are examples of "multiple output interfaces" according to the technology of this invention. The output synchronization signal output from the first signal processing unit 50A and the output synchronization signal output from the second signal processing unit 50B are examples of "multiple output synchronization signals corresponding to each of the multiple output interfaces" according to the technology of this invention.

[0313] As explained above, in the camera device 10, the output synchronization signal output from the first signal processing unit 50A is received by the output I / F 62D1, and the output synchronization signal output from the second signal processing unit 50B is received by the output I / F 62D2. Therefore, according to the camera device 10, even if there are multiple output destinations for digital image data 69B, the digital image data 69B can be processed according to the status of each output destination (the first signal processing unit 50A and the second signal processing unit 50B).

[0314] Furthermore, in the fifth embodiment described above, two signal processing units, a first signal processing unit 50A and a second signal processing unit 50B, are exemplified as the downstream circuit of the imaging element 44. However, the technology of the present invention is not limited to this. For example, three or more signal processing units may be connected to the processing circuit 62. In this case, the output circuit 62D has an output I / F corresponding to the number of signal processing units, and each output I / F is connected to a corresponding signal processing unit. Also, similar to the case where the third receiving I / F 63C is connected to the output circuit 62D and the second signal processing unit 50B, it is set to correspond to the third receiving I / F 63C (see reference). Figure 27 The receiving I / F is connected to the third and subsequent signal processing units. Furthermore, it is assumed that the third and subsequent signal processing units are also connected to the controller 46.

[0315] Furthermore, in the fifth embodiment described above, a wired communication method was used between the imaging element 44 and the first signal processing unit 50A via communication lines 53A and 55A. However, the technology of the present invention is not limited to this. For example, communication between the imaging element 44 and the first signal processing unit 50A may also be wireless. Similarly, communication between the imaging element 44 and the second signal processing unit 50B may also be wireless. Furthermore, communication between the imaging element 44 and the controller 46 may also be wireless. Furthermore, communication between the first signal processing unit 50A and the controller 46 may also be wireless. In addition, communication between the second signal processing unit 50B and the controller 46 may also be wireless.

[0316] Furthermore, in the above embodiments, examples of the processing circuit 62 being implemented using ASIC and FPGA have been described, but the technology of the present invention is not limited thereto. For example, the above-described camera processing can also be implemented using computer software architecture.

[0317] In this case, for example, such as Figure 28As shown, various programs are stored in the storage medium 900, and these programs are used to enable the computer 852 built into the imaging element 44 to perform the above-mentioned read-out storage processing, output processing, camera system drive processing, output system drive processing, and rewrite processing.

[0318] The various programs refer to the read-store program 902, the output program 904, the camera system driver 906, the output system driver 908, and the rewrite program 910. The read-store program 902 is used to cause the computer 852 to perform the aforementioned read-store processing. The output program 904 is used to cause the computer 852 to perform the aforementioned output processing. The camera system driver 906 is used to cause the computer 852 to perform the aforementioned camera system driver processing. The output system driver 908 is used to cause the computer 852 to perform the aforementioned output system driver processing. The rewrite program 910 is used to cause the computer 852 to perform the aforementioned rewrite processing.

[0319] As an example, such as Figure 28 As shown, the computer 852 includes a CPU 852A, a ROM 852B, and a RAM 852C. Various programs stored in the storage medium 900 are installed in the computer 852. The CPU 852A executes the read-store process according to the read-store program 902. The CPU 852A executes the output process according to the output program 904. The CPU 852A executes the camera system driver process according to the camera system driver program 906. The CPU 852A executes the output system driver process according to the output system driver program 908. Furthermore, the CPU 852A executes the rewrite process according to the rewrite program 910.

[0320] Here, a single CPU is illustrated as CPU 852A, but the technology of the present invention is not limited thereto, and multiple CPUs may be used instead of CPU 852A. Furthermore, storage medium 900 is a non-transitory storage medium. Examples of storage medium 900 include any portable storage medium such as an SSD or USB flash drive.

[0321] exist Figure 28 In the example shown, various programs are stored in the storage medium 900, but the technology of the present invention is not limited thereto. For example, various programs may be pre-stored in ROM 852B, and CPU 852A may read the various programs from ROM 852B, expand them into RAM 852C, and execute the expanded programs.

[0322] Furthermore, various programs can be stored in storage units such as other computers or server devices connected to computer 852 via a communication network (not shown), and various programs can be downloaded to computer 852 upon request from camera device 10. In this case, the downloaded programs are executed by CPU 852A of computer 852.

[0323] Furthermore, the computer 852 can be located outside the imaging element 44. In this case, the computer 852 can control the processing circuit 62 according to the program.

[0324] As hardware resources for performing the read-to-store processing, output processing, camera system drive processing, output system drive processing, and rewrite processing (hereinafter referred to as "various processes") described in the above embodiments, various processors as shown below can be used. For example, a general-purpose processor, i.e., a CPU, can be cited as a processor, which, as described above, functions as a hardware resource for performing various processes by executing software, i.e., a program. Furthermore, as processors, for example, dedicated circuits can be cited, which have circuit structures specifically designed for performing specific processes, such as FPGAs, PLDs, or ASICs.

[0325] The hardware resources for performing various processes can consist of one of these various processors, or they can consist of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware resources for performing various processes can be a single processor.

[0326] As examples of processors, firstly, there exists a configuration where a single processor, such as a client or server computer, is composed of a combination of one or more CPUs and software, functioning as the hardware resource for executing processing within the imaging element. Secondly, there exists a configuration where a single IC chip, such as a System-on-a-chip (SoC), is used to implement the functions of the entire system, including multiple hardware resources performing various processes. In this way, processing within the imaging element is achieved by using one or more of these processors as hardware resources.

[0327] Furthermore, as the hardware structure of these various processors, more specifically, circuits that combine circuit elements such as semiconductor elements can be used.

[0328] Furthermore, while interchangeable-lens cameras are exemplified as imaging devices 10 in the above embodiments, the technology of the present invention is not limited thereto. For example, the technology of the present invention can be applied to... Figure 29 The smart device 950 is shown as an example. Figure 29The smart device 950 shown is an example of a camera device according to the technology of the present invention. The smart device 950 incorporates the imaging element 44 described in the above embodiments. Even with this configuration, the smart device 950 can achieve the same function and effect as the camera device 10 described in the above embodiments. Furthermore, the technology of the present invention is not limited to the smart device 950, but can also be applied to personal computers or wearable terminal devices.

[0329] Furthermore, while the first display 32 and the second display 86 are exemplified in the above embodiments, the technology of the present invention is not limited thereto. For example, a separate display attached to the camera device body 12 can be used as the "display unit (display)" involved in the technology of the present invention.

[0330] Furthermore, the above-described processes are merely examples. Therefore, without departing from the main point, unnecessary steps can certainly be deleted, new steps can be added, or the processing order can be changed.

[0331] The above description and illustrations are detailed explanations of the technical aspects of this invention and are merely one example of the invention. For instance, the descriptions relating to the above structure, function, effect, and effect are examples of the structure, function, effect, and effect of the technical aspects of this invention. Therefore, without departing from the spirit of this invention, unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above description and illustrations. Furthermore, to avoid complexity and to facilitate understanding of the technical aspects of this invention, descriptions related to common technical knowledge that do not require special explanation in aspects enabling the implementation of this invention have been omitted from the above description and illustrations.

[0332] In this specification, "A and / or B" has the same meaning as "at least one of A and B". That is, "A and / or B" means that it can be just A, just B, or a combination of A and B. Furthermore, in this specification, when "and / or" is added to represent more than three items, the same concept as "A and / or B" can also be applied.

[0333] All documents, patent applications and technical standards described in this specification are referenced in this specification to the same extent that each document, patent application and technical standard is specifically and separately described and referenced by reference.

Claims

1. An imaging element comprising: The receiving interface receives from outside the imaging element a camera synchronization signal related to the time of shooting and at least one output synchronization signal related to the time of outputting image data obtained by shooting; The memory stores the image data obtained by shooting at a first frame rate according to the camera synchronization signal received by the receiving interface; and The output circuit outputs the image data stored in the memory at a second frame rate according to the output synchronization signal received by the receiving interface. The first frame rate is greater than or equal to the second frame rate. The receiving interface has a first receiving interface and a second receiving interface. The first receiving interface receives the camera synchronization signal from the outside. The second receiving interface receives the output synchronization signal from the external source. The camera synchronization signal is a signal that includes at least one of a vertical synchronization signal and a horizontal synchronization signal for camera use.

2. The imaging element according to claim 1, wherein, The output synchronization signal is a signal that includes at least one of an output vertical synchronization signal and an output horizontal synchronization signal.

3. The imaging element according to claim 1, wherein, The output circuit has multiple output interfaces. The receiving interface receives a plurality of output synchronization signals corresponding to each of the plurality of output interfaces.

4. The imaging element according to claim 1, wherein, The output circuit outputs the latest image data stored in the memory at the time point when the output synchronization signal is received by the receiving interface.

5. The imaging element according to claim 1, wherein, The camera synchronization signal is a signal that includes a vertical synchronization signal for camera operation. The imaging element further includes: The camera system holding circuit holds camera system drive mode indication information that indicates the drive mode of the camera system for the imaging element; and When the camera system control circuit receives the vertical synchronization signal for the camera from the receiving interface, it controls the camera system to drive the camera system in the driving mode indicated by the camera system driving mode indication information held by the camera system holding circuit.

6. The imaging element according to claim 5, wherein, The receiving interface receives camera system rewrite content information, which represents the rewrite content of the camera system drive mode indication information. When the camera system rewrite content information is received by the receiving interface, the camera system drive mode indication information held by the camera system holding circuit is rewritten to the content represented by the camera system rewrite content information.

7. The imaging element according to claim 6, wherein, The camera system drive mode indication information includes at least one of the following: information related to the camera area, information related to pixel interval elimination, information related to pixel addition operation method, information related to exposure time, information related to conversion gain switching, information related to analog gain, and information related to A / D conversion accuracy.

8. The imaging element according to claim 1, wherein, The output synchronization signal is a signal that includes the vertical synchronization signal for output. The imaging element further includes: The output system holding circuit holds the output system drive mode indication information that indicates the drive mode of the imaging element's output system; and The output system control circuit, upon receiving the vertical synchronization signal for the output from the receiving interface, controls the output system to drive in the manner indicated by the output system drive mode indication information held by the output system holding circuit.

9. The imaging element according to claim 8, wherein, The receiving interface receives output system rewrite content information, which represents the rewrite content of the output system drive mode indication information. When the output system rewrite content information is received by the receiving interface, the output system drive mode indication information held by the output system holding circuit is rewritten to the content represented by the output system rewrite content information.

10. The imaging element according to claim 8, wherein, The output system drive mode indication information includes at least one of the following: information related to the output destination, information related to digital interval elimination, information related to the digital addition operation method, information related to the average number of output frames, information related to digital gain, information related to the number of A / D output bits, and information related to the filling method of low-order idle bits.

11. The imaging element according to claim 1, wherein it is formed by at least a single chip comprising a photoelectric conversion element and the memory.

12. The imaging element according to claim 11, wherein, The imaging element is a stacked imaging element in which the memory is stacked on the photoelectric conversion element.

13. An imaging element comprising: The receiving interface receives from outside the imaging element a camera synchronization signal related to the time of shooting and at least one output synchronization signal related to the time of outputting image data obtained by shooting; The memory stores the image data obtained by shooting at a first frame rate according to the camera synchronization signal received by the receiving interface; and The output circuit outputs the image data stored in the memory at a second frame rate according to the output synchronization signal received by the receiving interface. The first frame rate is greater than or equal to the second frame rate. The output circuit outputs the latest image data as the latest image data among the plurality of image data. If the storage of one frame of image data related to the subject captured at the current time point to the memory is completed within the predicted output completion time, which is the time when the output of one frame of image data point to the memory is completed, then the latest image data is one frame of image data related to the subject captured at the current time point. If the storage of one frame of image data related to the subject captured at the current time point to the memory is not completed within the output completion time, then the latest image data is one frame of image data already stored in the memory.

14. The imaging element according to claim 13, wherein, If the storage of the latest image data in the memory is not completed within the output completion time, the latest image data is the latest image data stored in the memory.

15. The imaging element according to claim 13 or 14, wherein, The output synchronization signal is a signal that includes the horizontal synchronization signal for output. The output completion time is a predicted time based on the period of the horizontal synchronization signal received by the receiving interface.

16. The imaging element according to claim 13 or 14, wherein, The camera synchronization signal is a signal that includes a vertical synchronization signal for camera operation. The imaging element further includes: The camera system holding circuit holds camera system drive mode indication information that indicates the drive mode of the camera system for the imaging element; and When the camera system control circuit receives the vertical synchronization signal for the camera from the receiving interface, it controls the camera system to drive the camera system in the driving mode indicated by the camera system driving mode indication information held by the camera system holding circuit.

17. The imaging element according to claim 16, wherein, The receiving interface receives camera system rewrite content information, which represents the rewrite content of the camera system drive mode indication information. When the camera system rewrite content information is received by the receiving interface, the camera system drive mode indication information held by the camera system holding circuit is rewritten to the content represented by the camera system rewrite content information.

18. The imaging element according to claim 17, wherein, The camera system drive mode indication information includes at least one of the following: information related to the camera area, information related to pixel interval elimination, information related to pixel addition operation method, information related to exposure time, information related to conversion gain switching, information related to analog gain, and information related to A / D conversion accuracy.

19. The imaging element according to claim 13 or 14, wherein, The output synchronization signal is a signal that includes the vertical synchronization signal for output. The imaging element further includes: The output system holding circuit holds the output system drive mode indication information that indicates the drive mode of the imaging element's output system; and The output system control circuit, upon receiving the vertical synchronization signal for the output from the receiving interface, controls the output system to drive in the manner indicated by the output system drive mode indication information held by the output system holding circuit.

20. The imaging element according to claim 19, wherein, The receiving interface receives output system rewrite content information, which represents the rewrite content of the output system drive mode indication information. When the output system rewrite content information is received by the receiving interface, the output system drive mode indication information held by the output system holding circuit is rewritten to the content represented by the output system rewrite content information.

21. The imaging element according to claim 19, wherein, The output system drive mode indication information includes at least one of the following: information related to the output destination, information related to digital interval elimination, information related to the digital addition operation method, information related to the average number of output frames, information related to digital gain, information related to the number of A / D output bits, and information related to the filling method of low-order idle bits.

22. The imaging element according to claim 13 or 14, wherein at least the photoelectric conversion element and the memory are formed on a single chip.

23. The imaging element according to claim 22, wherein, The imaging element is a stacked imaging element in which the memory is stacked on the photoelectric conversion element.

24. A camera device comprising: The imaging element according to any one of claims 1 to 23; and The display processor controls the display to show an image based on the image data output by the output circuit.

25. A method for operating an imaging element, the imaging element comprising a receiving interface, a memory, and an output circuit, the method comprising the following steps: The receiving interface receives from outside the imaging element a camera synchronization signal related to the time of shooting and at least one output synchronization signal related to the time of outputting image data obtained through shooting. The memory stores the image data obtained by shooting at a first frame rate according to the camera synchronization signal received by the receiving interface. The output circuit outputs the image data stored in the memory at a second frame rate according to the output synchronization signal received by the receiving interface. Set the first frame rate to a rate greater than or equal to the second frame rate. The receiving interface has a first receiving interface and a second receiving interface. This includes the step of the first receiving interface receiving the camera synchronization signal from the outside. This includes the step of the second receiving interface receiving the output synchronization signal from the external source. The camera synchronization signal is a signal that includes at least one of a vertical synchronization signal and a horizontal synchronization signal for camera use.

26. A computer-readable storage medium storing a program for enabling a computer to function as a receiving interface and output circuitry included in an imaging element, the imaging element including the receiving interface, a memory, and the output circuitry. The receiving interface receives from outside the imaging element a camera synchronization signal related to the time of shooting and at least one output synchronization signal related to the time of outputting image data obtained through shooting. The memory stores the image data obtained by shooting at a first frame rate according to the camera synchronization signal received by the receiving interface. The output circuit outputs the image data stored in the memory at a second frame rate according to the output synchronization signal received by the receiving interface. The first frame rate is greater than or equal to the second frame rate. The receiving interface has a first receiving interface and a second receiving interface. The first receiving interface receives the camera synchronization signal from the outside. The second receiving interface receives the output synchronization signal from the external source. The camera synchronization signal includes at least one of a vertical synchronization signal and a horizontal synchronization signal for camera use.

Images