Imaging element, imaging device, method of operating imaging element, and program

Abstract

Provided are an imaging element, an imaging device, a method of operating an imaging element, and a program capable of selectively acquiring necessary image data from the imaging element.SOLUTION: The imaging element includes a storage unit that stores image data obtained by imaging and is built in the imaging element, a control unit that controls storage of the image data in the storage unit, causes the storage unit to store attribute information of the image data, and is built in the imaging element, an output unit that outputs the image data stored in the storage unit and is built in the imaging element, and a reception unit that receives an instruction regarding the attribute information, and the output unit outputs the attribute information according to the instruction received by the reception unit.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The technology of the present disclosure relates to an imaging element, an imaging device, an operation method for an imaging element, and a program. [Background technology]

[0002] International Publication No. 2014 / 007004 discloses a solid-state imaging device that includes an AD converter that digitizes analog pixel signals read out to a signal line from each pixel in a pixel array section, and is equipped with a signal processing section that transfers the digitized pixel data at a first speed that is faster than the frame rate, a memory section that holds the pixel data transferred from the signal processing section, a data processing section that reads out the pixel data from the memory section at a second speed that is slower than the first speed, and a control section that controls to stop the operation of a current source connected to the signal line and the operation of at least the AD converter in the signal processing section when reading out pixel data from the memory section.

[0003] In the solid-state imaging device described in WO 2014 / 007004, the signal processing unit, memory unit, data processing unit, and control unit are formed on at least one chip different from the chip on which the pixel array unit is formed, and the chip on which the pixel array unit is formed and the other at least one chip are stacked. Also, in the solid-state imaging device described in WO 2014 / 007004, the data processing unit has a decoder that specifies a column address for the memory unit and a sense amplifier that reads pixel data at the specified address, and reads the pixel data from the memory unit through the sense amplifier and the decoder.

[0004] International Publication No. 2013 / 145765 discloses an imaging unit that includes an imaging section that includes a first group that includes one or more pixels and a second group that includes one or more pixels different from the pixels that make up the first group, and a control section that, during a period in which the first group is caused to perform one charge accumulation, causes the second group to perform charge accumulation a different number of times than the first group, and outputs each pixel signal.

[0005] In the imaging unit described in WO 2013 / 145765, an imaging chip including an imaging section and a signal processing chip including a processing circuit for processing pixel signals are electrically connected in a stacked structure. A memory chip including a pixel memory for storing pixel signals is also electrically connected in a stacked structure. In the imaging unit described in WO 2013 / 145765, a control unit reads pixel signals of a designated group from the pixel memory and transfers them to an image processing unit in response to a transfer request from an external circuit for a designated group among groups including the second group. The imaging unit described in WO 2013 / 145765 also includes a data transfer interface that transmits pixel signals in response to the transfer request. The data transfer interface employs at least one of a double data rate system, an addressing system, a burst transfer system, a bus system, and a serial system for transmitting pixel signals. In the imaging unit described in WO 2013 / 145765, if pixel signals corresponding to multiple charge accumulations are stored in the pixel memory, the control unit transfers each pixel signal to the image processing unit. Summary of the Invention

[0006] One embodiment of the technique of the present disclosure provides an imaging element, an imaging apparatus, an operating method for an imaging element, and a program that can selectively acquire required image data from the imaging element. [Means for solving the problem]

[0007] A first aspect of the technology of the present disclosure is an imaging element that includes: a memory unit that stores image data obtained by imaging and is built into the imaging element; a control unit that controls the storage of image data in the memory unit and stores attribute information of the image data in the memory unit and is built into the imaging element; an output unit that outputs image data stored in the memory unit and is built into the imaging element; and a reception unit that receives instructions regarding the attribute information, wherein the output unit is an imaging element that outputs the attribute information in accordance with the instructions received by the reception unit.

[0008] A second aspect according to the technique of the present disclosure is the imaging device according to the first aspect, in which the output unit outputs the attribute information at a timing when the instruction is accepted by the accepting unit.

[0009] A third aspect according to the technique of the present disclosure is the image sensor according to the first or second aspect, in which the instruction is an external frame synchronization signal.

[0010] A fourth aspect of the technology of the present disclosure is an imaging element according to any one of the first to third aspects, wherein the output unit outputs attribute information relating to the latest image data among the image data stored in the memory unit.

[0011] A fifth aspect according to the technique of the present disclosure is the image sensor according to the fourth aspect, wherein the output unit outputs attribute information related to the latest image data at the timing when the instruction is accepted by the accepting unit.

[0012] A sixth aspect of the technology of the present disclosure is an imaging element according to any one of the first to fifth aspects, in which the output unit is capable of outputting attribute information for each of multiple image data, and the attribute information is output by the output unit in the order of imaging.

[0013] A seventh aspect of the technology of the present disclosure is an imaging element according to any one of the first to sixth aspects, in which the attribute information is information including at least one of an address, an image size, an imaging time, and imaging conditions.

[0014] An eighth aspect of the technology of the present disclosure is an imaging element according to any one of the first to seventh aspects, in which, when image data is erased from the memory unit upon the next imaging, the attribute information output from the output unit is information including erasure information indicating that the image data will be erased from the memory unit upon the next imaging.

[0015] A ninth aspect of the technology of the present disclosure is an imaging element according to the eighth aspect, in which, when image data is erased from the storage unit, the control unit erases attribute information related to the image data to be erased, which is image data in the storage unit that has been designated for erasure, from the storage unit, and erases the image data to be erased from the storage unit.

[0016] A tenth aspect of the technology of the present disclosure is an imaging element according to any one of the first to ninth aspects, which includes a compression circuit that compresses image data, and a control unit that stores image data for a predetermined number of frames in a memory unit, stores compressed image data obtained by compressing the image data by the compression circuit in the memory unit, and associates attribute information regarding the compressed image data and compression identification information that can identify that the image data has been compressed with the compressed image data.

[0017] An eleventh aspect of the technology of the present disclosure is an imaging element according to the tenth aspect, in which, when compressed image data is stored in the memory unit, the attribute information output from the output unit is information including compression plan information indicating that compressed image data is scheduled to be stored in the memory unit.

[0018] A twelfth aspect of the technology of the present disclosure is an imaging element according to the tenth or eleventh aspect, wherein the attribute information output from the output unit is information including information capable of identifying the compression format of the compressed image data.

[0019] A thirteenth aspect of the technology of the present disclosure is an imaging element according to any one of the first to twelfth aspects, in which the instructions include an output amount of attribute information by the output unit, and the output unit outputs the attribute information at the output amount.

[0020] A fourteenth aspect according to the technique of the present disclosure is the image sensor according to the thirteenth aspect, in which the output amount is defined by the number of frames of image data.

[0021] A 15th aspect of the technology disclosed herein is an imaging element according to any one of the first to fourteenth aspects, further including a receiving unit that receives attribute information transmitted from a downstream circuit located downstream of the imaging element, and a control unit that acquires image data from a memory unit according to the attribute information received by the receiving unit and causes an output unit to output the acquired image data to the downstream circuit.

[0022] A 16th aspect of the technology of the present disclosure is an imaging element according to any one of the first to fifteenth aspects, wherein the output unit has a first output unit and a second output unit, the first output unit outputs image data, and the second output unit outputs attribute information.

[0023] A seventeenth aspect of the technique of the present disclosure is the imaging device according to any one of the first to sixteenth aspects, in which at least the photoelectric conversion element and the storage unit are integrated into a single chip.

[0024] An eighteenth aspect according to the technique of the present disclosure is the imaging element according to the seventeenth aspect, in which the imaging element is a stacked imaging element in which a storage unit is stacked on a photoelectric conversion element.

[0025] A 19th aspect of the technology of the present disclosure is an imaging device including an imaging element according to any one of the first to eighteenth aspects, and a control device that performs at least one of control to display an image based on image data output by the output unit on a display unit, and control to store the image data output by the output unit in a storage device.

[0026] A twentieth aspect of the technology of the present disclosure is a method for operating an imaging element having a built-in memory unit that stores image data obtained by capturing an image, the method including controlling the storage of image data in the memory unit, storing attribute information of the image data in the memory unit, outputting the image data stored in the memory unit, accepting instructions regarding the attribute information, and outputting the attribute information in accordance with the accepted instructions.

[0027] A 21st aspect of the technology of the present disclosure is a program for causing a computer applied to an imaging element having a built-in memory unit for storing image data obtained by capturing an image to execute processing including controlling the storage of image data in the memory unit, storing attribute information of the image data in the memory unit, outputting the image data stored in the memory unit, accepting instructions regarding the attribute information, and outputting the attribute information in accordance with the accepted instructions. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 1 is a perspective view illustrating an example of the appearance of an imaging device according to an embodiment. [Figure 2] 2 is a rear view showing an example of the appearance of the rear side of the imaging device shown in FIG. 1. FIG. [Figure 3] 1 is a block diagram showing an example of a configuration of an imaging apparatus according to an embodiment. [Figure 4] FIG. 2 is a block diagram showing an example of a hardware configuration of an electrical system of an imaging device main body included in the imaging device according to the embodiment. [Figure 5] 3A and 3B are conceptual diagrams illustrating the frame rate of an image sensor included in the image pickup device according to the embodiment. [Figure 6] FIG. 2 is a conceptual diagram illustrating an example of a layered structure of an imaging element according to an embodiment. [Figure 7] FIG. 2 is a block diagram showing an example of the structure of a memory included in the image sensor according to the embodiment. [Figure 8] FIG. 1 is a block diagram showing an example of the electrical hardware configuration of an imaging element included in an imaging device according to an embodiment, and is a block diagram used to explain an example in which imaging is performed by the imaging element and attribute information is output from the imaging element. [Figure 9] 3A and 3B are conceptual diagrams showing an example of the contents of an imaging process and an output process performed by the imaging element according to the embodiment. [Figure 10] 10 is a time chart showing an example of processing content in an image sensor included in the image pickup device according to the embodiment. [Figure 11A]FIG. 10 is a conceptual diagram showing an example of the contents of attribute information output from the output I / F of the imaging element for the first to fourth frames. [Figure 11B] FIG. 10 is a conceptual diagram showing an example of the contents of attribute information output from the output I / F of the imaging element in the fifth and sixth frames. [Figure 12] FIG. 2 is a block diagram showing an example of the hardware configuration of an electrical system of an image sensor included in an imaging apparatus according to an embodiment, and is a block diagram used to explain an example in which digital image data is output from the image sensor. [Figure 13] 10 is a flowchart showing an example of the flow of attribute information output processing according to the embodiment. [Figure 14] 10 is a flowchart illustrating an example of the flow of a data erasure process according to the embodiment. [Figure 15] 10 is a time chart showing an example of processing content in which digital image data and compressed image data are stored in a memory when compression processing is performed on digital image data. [Figure 16A] FIG. 10 is a conceptual diagram showing an example of the contents of attribute information output from the output I / F of the imaging element for the first to fourth frames when compression processing is performed on digital image data for the fourth frame and thereafter. [Figure 16B] FIG. 10 is a conceptual diagram showing an example of the contents of attribute information output from the output I / F of the image sensor in the fifth and sixth frames when compression processing is performed on digital image data from the fourth frame onwards. [Figure 17A] This is a conceptual diagram showing an example of the content of attribute information that includes a compression plan flag indicating whether one frame of digital image data is scheduled to be compressed when the next image is captured, and is output from the output I / F of the imaging element for the first to fourth frames. [Figure 17B] This is a conceptual diagram showing an example of the content of attribute information that includes a compression plan flag indicating whether one frame of digital image data is scheduled to be compressed when the next image is captured, and is output from the output I / F of the image sensor for the fifth and sixth frames. [Figure 18A]This is a conceptual diagram showing an example of the content of attribute information that includes a compression plan flag indicating whether one frame of digital image data is scheduled to be compressed when the next image is captured, and also includes compression format specification information, and is output from the output I / F of the image sensor for the fourth and fifth frames. [Figure 18B] This is a conceptual diagram showing an example of the content of attribute information that includes a compression plan flag indicating whether one frame of digital image data is scheduled to be compressed when the next image is captured, and also includes compression format specification information, and is output from the output I / F of the image sensor in the sixth frame. [Figure 19] This is a time chart showing an example of the processing content in the image sensor when attribute information is output by the output I / F of the image sensor at the timing when an instruction regarding attribute information (e.g., a vertical synchronization signal) is received by the reception I / F of the image sensor. [Figure 20] FIG. 10 is a conceptual diagram showing an example of the contents of attribute information output for the first to sixth frames when the attribute information output for each frame is attribute information relating to one frame of digital image data. [Figure 21] FIG. 10 is a conceptual diagram showing an example of the content of attribute information that is output for the first to sixth frames when the attribute information output for each frame is attribute information related to one frame of digital image data, and does not include an erasure flag or a compression planned flag. [Figure 22] 10 is a flowchart showing a modified example of the flow of the attribute information output process according to the embodiment. [Figure 23] FIG. 10 is a block diagram showing a modified example of the hardware configuration of the electrical system of the image sensor included in the image pickup apparatus according to the embodiment. [Figure 24] FIG. 2 is a rear perspective view showing an example of the appearance of the rear side of the smart device according to the embodiment. [Figure 25] FIG. 25 is a front perspective view showing an example of the appearance of the front side of the smart device shown in FIG. 24. [Figure 26]10 is a conceptual diagram showing an example of how an image sensor-side program according to an embodiment is installed from a storage medium storing the image sensor-side program to a computer within the image sensor. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0029] An example of an embodiment of an imaging device according to the technique of the present disclosure will now be described with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". RAM is an abbreviation for "Random Access Memory". ROM is an abbreviation for "Read Only Memory". DRAM is an abbreviation for "Dynamic Random Access Memory". SRAM is an abbreviation for "Static Random Access Memory". LSI is an abbreviation for "Large-Scale Integrated circuit". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory." CCD is an abbreviation for "Charge Coupled Device." CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor." EL stands for "Electro-Luminescence". A / D stands for "Analog / Digital". I / F stands for "Interface". UI stands for "User Interface". LVDS stands for "Low Voltage Differential Signaling". PCI-e stands for "Peripheral Component Interconnect Express". SATA stands for "Serial Advanced Technology SLVS-EC is an abbreviation for "Scalable Low Signaling with Embedded Clock". MIPI is an abbreviation for "Mobile Industry Processor Interface". fps is an abbreviation for "frames per second". FIFO is an abbreviation for "First In, First Out". MPEG is an abbreviation for "Moving Picture Experts Group". JPEG is an abbreviation for "Joint Photographic Experts Group". TIFF is an abbreviation for "Tagged Image File Format". BMP is an abbreviation for "Bitmap". PNG is an abbreviation for "Portable Network Graphics". GIF is an abbreviation for "Graphics Interchange Format".

[0032] 1, as an example, imaging device 10 is a digital camera with an interchangeable lens and no reflex mirror. Imaging device 10 includes imaging device body 12 and an interchangeable lens 14 that is interchangeably attached to imaging device body 12. Note that, although a digital camera with an interchangeable lens and no reflex mirror is given here as an example of imaging device 10, the technology of the present disclosure is not limited to this, and imaging device 10 may be other types of digital cameras, such as a fixed lens type.

[0033] The imaging device body 12 is provided with an imaging element 38. When the interchangeable lens 14 is attached to the imaging device body 12, subject light representing the subject passes through the interchangeable lens 14 and is focused on the imaging element 38, and image data representing the image of the subject (see, for example, FIGS. 4 and 5) is generated by the imaging element 38.

[0034] The imaging device body 12 is provided with a Hybrid Finder (registered trademark) 16. The hybrid finder 16 here refers to a finder that selectively uses, for example, an optical viewfinder (hereinafter referred to as "OVF") and an electronic viewfinder (hereinafter referred to as "EVF"). Note that OVF is an abbreviation for "optical viewfinder." Also, EVF is an abbreviation for "electronic viewfinder."

[0035] A viewfinder switch lever 18 is provided on the front of the imaging device body 12. Rotating the viewfinder switch lever 18 in the direction of the arrow SW switches between an optical image visible in the OVF and a live view image, which is an electronic image visible in the EVF. The term "live view image" used here refers to a moving image for display based on image data obtained by capturing an image using the imaging element 38. A live view image is also commonly referred to as a through image. A release button 20 and a dial 23 are provided on the top surface of the imaging device body 12. The dial 23 is operated to set the operating mode of the imaging system and the operating mode of the playback system, and thereby the imaging device 10 is selectively set to an imaging mode or a playback mode as its operating mode.

[0036] The release button 20 functions as an image capture preparation instruction section and an image capture instruction section, and is capable of detecting two stages of pressing operation: an image capture preparation instruction state and an image capture instruction state. The image capture preparation instruction state refers to a state in which the button is pressed, for example, from a standby position to an intermediate position (half-pressed position), and the image capture instruction state refers to a state in which the button is pressed beyond the intermediate position to a final pressed position (fully pressed position). Note that, hereinafter, the "state in which the button is pressed from the standby position to the half-pressed position" will be referred to as the "half-pressed state," and the "state in which the button is pressed from the standby position to the fully pressed position" will be referred to as the "fully pressed state."

[0037] As an example, as shown in FIG. 2, a touch panel display 24, instruction keys 27, and a finder eyepiece 30 are provided on the rear surface of the imaging device body 12.

[0038] The touch panel display 24 includes a display 26 and a touch panel 28 (see also FIG. 4). An example of the display 26 is a liquid crystal display. The display 26 may be other types of display, such as an organic EL display or an inorganic EL display, instead of an LCD display. The display 26 and the EVF are examples of a "display unit (display)" according to the technology of the present disclosure. The display of the EVF is the same as the display of the display 26, so further description will be omitted, but in this specification, the display on the display 26 can be interpreted as the display on the EVF.

[0039] The display 26 displays images, text information, etc. The display 26 is used to display live view images obtained by continuous image capture when the imaging device 10 is in imaging mode. The display 26 is also used to display still images obtained by capturing images when an instruction to capture a still image is given. The display 26 is also used to display playback images and menu screens, etc. when the imaging device 10 is in playback mode.

[0040] The touch panel 28 is a transmissive touch panel, and is placed on the surface of the display area of ​​the display 26. The touch panel 28 receives instructions from the user by detecting contact with a pointing object such as a finger or a stylus pen.

[0041] The instruction keys 27 accept various instructions. The "various instructions" referred to here refer to, for example, an instruction to display a menu screen from which various menus can be selected, an instruction to select one or more menus, an instruction to confirm the selected content, an instruction to erase the selected content, an instruction to zoom in, zoom out, and frame-by-frame advance, etc.

[0042] As an example, as shown in FIG. 3, the interchangeable lens 14 has an imaging lens 40. The imaging lens 40 includes an objective lens 40A, a focus lens 40B, and an aperture 40C. The objective lens 40A, the focus lens 40B, and the aperture 40C are arranged in this order along the optical axis L1 from the subject side (object side) to the imaging device body 12 side (image side). The focus lens 40B and the aperture 40C are operated by receiving power from a drive source (not shown) such as a motor. That is, the focus lens 40B and the aperture 40C move along the optical axis L1 in response to the applied power. The aperture 40C also adjusts exposure by operating in response to the applied power.

[0043] The imaging device main body 12 includes a post-circuit 13, a UI device 17, a mechanical shutter 41, and an imaging element 38. The post-circuit 13 is a circuit located after the imaging element 38. The post-circuit 13 includes a controller 15 and a signal processing circuit 34. The controller 15 is connected to the UI device 17, the signal processing circuit 34, and the imaging element 38, and controls the entire electrical system of the imaging device 10.

[0044] The imaging element 38 includes a photoelectric conversion element 42 having a light receiving surface 42A. In this embodiment, the imaging element 38 is a CMOS image sensor. Although a CMOS image sensor is exemplified as the imaging element 38 here, the technology of the present disclosure is not limited thereto, and the technology of the present disclosure also applies even if the imaging element 38 is another type of image sensor, such as a CCD image sensor.

[0045] The mechanical shutter 41 operates by receiving power from a drive source (not shown) such as a motor. When the interchangeable lens 14 is attached to the imaging device body 12, subject light representing the subject passes through the imaging lens 40 and is imaged on the light receiving surface 42A via the mechanical shutter 41.

[0046] The UI device 17 is a device that presents information to a user and receives instructions from the user. The controller 15 acquires various information from the UI device 17 and controls the UI device 17.

[0047] The image sensor 38 is connected to the controller 15, and under the control of the controller 15, captures an image of a subject to generate image data representing the image of the subject.

[0048] The imaging element 38 is connected to the signal processing circuit 34. The signal processing circuit 34 is an LSI, and more specifically, a device including an ASIC and an FPGA. The controller 15 acquires various information from the signal processing circuit 34 and controls the imaging element 38. Under the control of the controller 15, the imaging element 38 outputs image data generated by the photoelectric conversion element 42 to the signal processing circuit 34.

[0049] The signal processing circuit 34 performs various types of signal processing on the image data input from the image sensor 38. The various types of signal processing performed by the signal processing circuit 34 include known signal processing such as white balance adjustment, sharpness adjustment, gamma correction, color space conversion processing, and color difference correction.

[0050] The various signal processing operations performed by the signal processing circuit 34 may be distributed between the signal processing circuit 34 and the image sensor 38. In other words, at least a part of the various signal processing operations performed by the signal processing circuit 34 may be performed by the processing circuit 110 of the image sensor 38.

[0051] In this embodiment, a device including an ASIC and an FPGA is exemplified as the signal processing circuit 34, but the technology of the present disclosure is not limited to this, and the signal processing circuit 34 may be a device including an ASIC, an FPGA, and / or a PLD.

[0052] The signal processing circuit 34 may also be a computer including a CPU, storage, and memory. The term "storage" as used herein refers to a non-volatile storage device such as an SSD or HDD, and the term "memory" as used herein refers to a volatile storage device such as a DRAM or SRAM. The computer may include a single CPU or multiple CPUs. A GPU may also be used instead of a CPU. The signal processing circuit 34 may also be realized by a combination of hardware and software configurations.

[0053] 4, the controller 15 includes a CPU 15A, a storage 15B, a memory 15C, an output I / F 15D, and a reception I / F 15E. The CPU 15A, the storage 15B, the memory 15C, the output I / F 15D, and the reception I / F 15E are connected via a bus line 100. In the example shown in FIG. 4, for convenience of illustration, one bus line is shown as the bus line 100, but the bus line 100 also includes a data bus, an address bus, a control bus, and the like.

[0054] The storage 15B stores various parameters and various programs. The storage 15B is a non-volatile storage device. Here, an EEPROM is used as an example of the storage 15B, but the storage 15B is not limited to this and may be a mask ROM, HDD, SSD, or the like. The memory 15C is a volatile storage device. Various information is temporarily stored in the memory 15C. The memory 15C is used as a work memory by the CPU 15A. Here, a DRAM is used as an example of the memory 15C, but the storage 15B is not limited to this and may be another type of volatile storage device such as an SRAM. The CPU 15A is an example of a "control device" according to the technology of the present disclosure, and the storage 15B is a non-volatile storage device according to the technology of the present disclosure. This is an example of a "memory device."

[0055] Various programs are stored in storage 15B. CPU 15A reads the various programs from storage 15B and loads the read programs into memory 15C. CPU 15A controls the entire imaging device 10 in accordance with the various programs loaded into memory 15C.

[0056] The output I / F 15D is connected to the imaging element 38. The CPU 15A controls the imaging element 38 via the output I / F 15D. For example, the CPU 15A controls the timing of imaging performed by the imaging element 38 by supplying an imaging timing signal that defines the timing of imaging to the imaging element 38 via the output I / F 15D.

[0057] The reception I / F 15E is connected to the signal processing circuit 34. The CPU 15A exchanges various information with the signal processing circuit 34 via the reception I / F 15E.

[0058] Image data is input to the signal processing circuit 34 from the imaging element 38. The signal processing circuit 34 performs various signal processes (described in detail later) on the image data input from the imaging element 38. The signal processing circuit 34 outputs the image data that has undergone the various signal processes to the reception I / F 15E. The reception I / F 15E receives the image data from the signal processing circuit 34 and transfers the received image data to the CPU 15A.

[0059] An external I / F 104 is connected to the bus line 100. The external I / F 104 is a communication device configured with a circuit. Note that although a device configured with a circuit is employed as the external I / F 104 here, this is merely an example. The external I / F 104 may be a device including an ASIC, an FPGA, and / or a PLD. Furthermore, the external I / F 104 may be realized by a combination of a hardware configuration and a software configuration.

[0060] An example of the external I / F 104 is a USB interface, to which external devices (not shown) such as a memory card controller, a smart device, a personal computer, a server, a USB memory, and / or a memory card can be connected. The external I / F 104 controls the exchange of various information between the CPU 15A and the external devices. Note that the external devices directly or indirectly connected to the external I / F 104, i.e., the smart device, a personal computer, a server, a USB memory, and / or a memory card, are examples of "storage devices" according to the technology of the present disclosure.

[0061] The UI device 17 includes a touch panel display 24 and a reception device 84. The display 26 and the touch panel 28 are connected to a bus line 100. Therefore, the CPU 15A causes the display 26 to display various information and operates in accordance with various instructions received by the touch panel 28.

[0062] The accepting device 84 includes a hard key unit 25. The hard key unit 25 is a plurality of hard keys, and includes a release button 20 (see FIG. 1), a dial 23 (see FIGS. 1 and 2), and instruction keys 27 (see FIG. 2). The hard key unit 25 is connected to the bus line 100, and the CPU 15A acquires instructions accepted by the hard key unit 25 and operates in accordance with the acquired instructions.

[0063] As an example, as shown in FIG. 5, an imaging timing signal is input from the controller 15 to the imaging element 38. The imaging timing signal includes a vertical synchronization signal and a horizontal synchronization signal. The vertical synchronization signal is used to synchronize the reading of image data from the photoelectric conversion element 42 for each frame. The vertical synchronization signal is a synchronization signal that specifies the start timing of reading image data for each horizontal line from the photoelectric conversion element 42. The image sensor 38 reads image data from the photoelectric conversion element 42 in accordance with a frame rate determined by the vertical synchronization signal input from the controller 15. The vertical synchronization signal is an example of an "instruction related to attribute information" and an "external frame synchronization signal" according to the technology of the present disclosure, and the controller 15 is an example of "external" according to the technology of the present disclosure.

[0064] 5, the frame rate of the image sensor 38 is set to a frame rate at which eight frames are read out from the photoelectric conversion element 42 within the period T. A specific example of the frame rate is 120 fps, but the frame rate is not limited to this and may be a frame rate greater than 120 fps (e.g., 240 fps) or less than 120 fps (e.g., 60 fps).

[0065] 6, an image sensor 38 includes a photoelectric conversion element 42, a processing circuit 110, and a memory 112. The image sensor 38 is an image sensor in which the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single chip. That is, the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single package. In the image sensor 38, the processing circuit 110 and the memory 112 are stacked on the photoelectric conversion element 42. Specifically, the photoelectric conversion element 42 and the processing circuit 110 are electrically connected to each other by conductive bumps (not shown) made of copper or the like, and the processing circuit 110 and the memory 112 are electrically connected to each other by conductive bumps (not shown) made of copper or the like.

[0066] The processing circuit 110 is, for example, an LSI. The memory 112 is a memory with different write and read timings. Here, a DRAM is used as an example of the memory 112.

[0067] The processing circuit 110 is a device including an ASIC and an FPGA, and controls the entire image sensor 38 according to instructions from the controller 15. While the example described here is one in which the processing circuit 110 is implemented by a device including an ASIC and an FPGA, the technology of the present disclosure is not limited thereto and may be implemented by a device including an ASIC, an FPGA, and / or a PLD. The processing circuit 110 may also be a computer including a CPU, a non-volatile storage device such as an EEPROM, and a volatile storage device such as a RAM. The computer may include a single or multiple CPUs. A GPU may be used instead of the CPU. The processing circuit 110 may also be implemented by a combination of hardware and software.

[0068] The photoelectric conversion element 42 has a plurality of photodiodes arranged in a matrix. An example of the plurality of photodiodes is photodiodes for "4896 x 3265" pixels.

[0069] A color filter is disposed on each photodiode included in the photoelectric conversion element 42. The color filters include a G filter corresponding to G (green), which contributes most to obtaining a luminance signal, an R filter corresponding to R (red), and a B filter corresponding to B (blue).

[0070] The photoelectric conversion element 42 has R pixels, G pixels, and B pixels. The R pixels correspond to photodiodes on which R filters are arranged, the G pixels correspond to photodiodes on which G filters are arranged, and the B pixels correspond to photodiodes on which B filters are arranged. The R pixels, G pixels, and B pixels are arranged with a predetermined periodicity in the row direction (horizontal direction) and column direction (vertical direction). In this embodiment, the R pixels, The G pixels and B pixels are arranged with a periodicity corresponding to the X-Trans (registered trademark) array. Note that although the X-Trans array is exemplified here, the technology of the present disclosure is not limited to this, and the array of the R pixels, G pixels, and B pixels may be a Bayer array, a honeycomb array, or the like.

[0071] The image sensor 38 has a so-called electronic shutter function, and by operating the electronic shutter function under the control of the controller 15, the charge accumulation time of each photodiode in the photoelectric conversion element 42 is controlled. The charge accumulation time refers to the so-called shutter speed.

[0072] The image sensor 38 selectively captures still images and live view images using a rolling shutter system. Still image capture is achieved by activating the electronic shutter function and operating the mechanical shutter (not shown), while live view image capture is achieved by activating the electronic shutter function without operating the mechanical shutter. While the rolling shutter system is exemplified here, the technology of the present disclosure is not limited to this, and a global shutter system may be applied instead of the rolling shutter system.

[0073] The memory 112 is an example of a "storage unit (memory)" according to the technology of the present disclosure. In this embodiment, a DRAM is used as the memory 112, but the technology of the present disclosure is also applicable when the memory 112 is another type of memory. The imaging element 38 is an example of a "stacked imaging element" according to the technology of the present disclosure.

[0074] 7, the memory 112 has a first storage area 112A, a second storage area 112B, a third storage area 112C, a fourth storage area 112D, a fifth storage area 112E, and a sixth storage area 112F. Note that, for convenience of explanation, six storage areas are shown here, but this is merely an example, and the memory 112 may have multiple storage areas capable of storing multiple frames of image data.

[0075] In the example shown in FIG. 7, addresses within the memory 112 are assigned to each of the first storage area 112A, the second storage area 112B, the third storage area 112C, the fourth storage area 112D, the fifth storage area 112E, and the sixth storage area 112F. The address "0X00000000" is assigned to the first storage area 112A. The address "0X00100000" is assigned to the second storage area 112B. The address "0X00200000" is assigned to the third storage area 112C. The address "0X00300000" is assigned to the fourth storage area 112D. The address "0X00400000" is assigned to the fifth storage area 112E. The sixth memory area 112F is assigned an address of "0X00500000".

[0076] 8, the processing circuit 110 includes a reception I / F 110D1 and an output I / F 110D2. Here, the reception I / F 110D1 is an example of a "reception unit" and a "reception unit" according to the technology of the present disclosure, and the output I / F 110D2 is an example of an "output unit (output interface)" according to the technology of the present disclosure. Note that the reception unit refers to, for example, an acceptor, and the reception unit refers to, for example, a receiver.

[0077] The output I / F 15D of the controller 15 is connected to the reception I / F 110D1 of the processing circuit 110, and outputs an imaging timing signal to the reception I / F 110D1. The reception I / F 110D1 receives the imaging timing signal output from the output I / F 15D.

[0078] The signal processing circuit 34 includes a reception I / F 34A and an output I / F 34B. The output I / F 34A is connected to the output I / F 110D2 of the image sensor 38. The output I / F 110D2 of the processing circuit 110 outputs various information such as image data (hereinafter also simply referred to as "various information") to the reception I / F 34A of the signal processing circuit 34, and the reception I / F 34A receives the various information output from the output I / F 110D2. The signal processing circuit 34 performs signal processing as necessary on the various information received by the reception I / F 34A. The output I / F 34B is connected to the reception I / F 15E of the controller 15 and outputs the various information to the reception I / F 15E of the controller 15. The reception I / F 15E receives the various information output from the output I / F 34B.

[0079] In the image sensor 38, the processing circuit 110 includes a reception I / F 110D1 and an output I / F 110D2, as well as a readout circuit 110A, a digital processing circuit 110B, a control circuit 110C, and an image processing circuit 110E. The control circuit 110C is an example of a "control unit (control circuit)" according to the technology of the present disclosure.

[0080] The readout circuit 110A is connected to each of the photoelectric conversion element 42, the digital processing circuit 110B, and the control circuit 110C. The digital processing circuit 110B is connected to the control circuit 110C. The control circuit 110C is connected to each of the memory 112, the reception I / F 110D1, the output I / F 110D2, and the image processing circuit 110E.

[0081] The image data described above is roughly divided into analog image data 70A and digital image data 70B, as shown in Fig. 8. For ease of explanation, when there is no need to distinguish between the analog image data 70A and the digital image data 70B, they will be referred to as "image data" without being assigned a reference symbol.

[0082] Each of the reception I / F 110D1 and output I / F 110D2 of the processing circuit 110 is a communication device having an FPGA. Each of the output I / F 15D and reception I / F 15E of the controller 15 is also a communication device having an FPGA. Each of the reception I / F 34A and output I / F 34B of the signal processing circuit 34 is also a communication device having an FPGA.

[0083] The receiving I / F 110D1 of the processing circuit 110 and the output I / F 15D of the controller 15 are connected in accordance with the PCI-e connection standard. The output I / F 110D2 of the processing circuit 110 and the receiving I / F 34A of the signal processing circuit 34 are also connected in accordance with the PCI-e connection standard. The output I / F 34B of the signal processing circuit 34 and the receiving I / F 15E of the controller 15 are also connected in accordance with the PCI-e connection standard. In the following, when it is not necessary to distinguish between the receiving I / F 110D1, the output I / F 110D2, the receiving I / F 34A, the output I / F 34B, the receiving I / F 15E, and the output I / F 15D, they will be referred to as "communication I / Fs" without reference numerals.

[0084] Here, a communication device configured with circuits (such as ASIC, FPGA, and / or PLD) is used as the communication I / F, but this is merely one example. The communication I / F may be a computer including a CPU, storage such as EEPROM, and memory such as RAM. In this case, the computer may include a single CPU or multiple CPUs. A GPU may be used instead of a CPU. The communication I / F may also be realized by a combination of hardware and software configurations.

[0085] The reception I / F 110D1 receives the imaging timing signal output from the output I / F 15D of the controller 15, and transfers the received imaging timing signal to the control circuit 110C.

[0086] The readout circuit 110A controls the photoelectric conversion element 42 under the control of the control circuit 110C. The analog image data 70A is read out from the conversion element 42. The analog image data 70A is read out from the photoelectric conversion element 42 in accordance with an imaging timing signal input from the controller 15 to the processing circuit 110.

[0087] Specifically, first, the receiving I / F 110D1 receives an imaging timing signal from the controller 15 and transfers the received imaging timing signal to the control circuit 110C. Next, the control circuit 110C transfers the imaging timing signal transferred from the receiving I / F 110D1 to the readout circuit 110A. That is, a vertical synchronization signal and a horizontal synchronization signal are transferred to the readout circuit 110A. Then, the readout circuit 110A starts reading out the analog image data 70A from the photoelectric conversion element 42 in frame units in accordance with the vertical synchronization signal transferred from the control circuit 110C. Furthermore, the readout circuit 110A starts reading out the analog image data 70A in horizontal line units in accordance with the horizontal synchronization signal transferred from the control circuit 110C.

[0088] The readout circuit 110A performs analog signal processing on the analog image data 70A read out from the photoelectric conversion element 42. The analog signal processing includes well-known processes such as noise cancellation processing and analog gain processing. The noise cancellation processing is processing to cancel noise caused by variations in characteristics between pixels included in the photoelectric conversion element 42. The analog gain processing is processing to apply gain to the analog image data 70A. The readout circuit 110A also performs correlated double sampling on the analog image data 70A. After the readout circuit 110A performs correlated double sampling on the analog image data 70A, the analog image data 70A is output to the digital processing circuit 110B.

[0089] The digital processing circuit 110B includes an A / D converter 110B1, which performs A / D conversion on the analog image data 70A.

[0090] The digital processing circuit 110B performs digital signal processing on the analog image data 70A input from the readout circuit 110A. The digital signal processing includes, for example, correlated double sampling, A / D conversion by an A / D converter 110B1, and digital gain processing.

[0091] The analog image data 70A input from the readout circuit 110A is subjected to A / D conversion by the A / D converter 110B1, thereby digitizing the analog image data 70A and obtaining digital image data 70B as RAW data. The digital image data 70B is then subjected to digital gain processing by the digital processing circuit 110B. Digital gain processing refers to processing that applies a gain to the digital image data 70B. The digital image data 70B obtained by this digital signal processing is output to the control circuit 110C by the digital processing circuit 110B.

[0092] The control circuit 110C outputs the digital image data 70B input from the digital processing circuit 110B to the image processing circuit 110E. The image processing circuit 110E performs image processing on the digital image data 70B input from the control circuit 110C, and outputs the processed digital image data 70B to the control circuit 110C. The "image processing" referred to here includes, for example, demosaic processing and / or digital thinning processing.

[0093] Demosaicing is a process of calculating all color information for each pixel from a mosaic image corresponding to the arrangement of color filters. For example, if the image sensor 38 is an image sensor to which three color filters of RGB are applied, all color information of RGB is calculated for each pixel from the RGB mosaic image. Digital thinning is a process of thinning out pixels included in image data in line units. The line unit refers to, for example, a horizontal line unit and / or a vertical line unit.

[0094] The memory 112 is a memory capable of storing digital image data for multiple frames. The memory 112 has a storage area for each pixel (see FIG. 7), and the digital image data 70B is stored in a corresponding storage area of ​​the memory 112 for each pixel by the control circuit 110C. The control circuit 110C stores the digital image data 70B input from the image processing circuit 110E in the memory 112.

[0095] The control circuit 110C is capable of randomly accessing the memory 112. In response to an instruction from the controller 15, the control circuit 110C acquires attribute information of the digital image data 70B stored in the memory 112. The attribute information is information that indicates the attributes of the digital image data 70B stored in the memory 112. Here, the attribute information refers to information including, for example, an address, an image size, an image capture time, an exposure time, an image sensor sensitivity, and an erase flag.

[0096] The address refers to information (recording address) that identifies the storage location of digital image data 70B in frame units in memory 112. The image size refers to the size of one frame's worth of digital image data 70B in the horizontal line direction (horizontal size) and the size of one frame's worth of digital image data 70B in the vertical line direction (vertical size). The imaging time refers to the time (e.g., year, month, day, hour, minute, and second) when imaging is performed by image sensor 38. Here, the "time when imaging is performed" refers to, for example, the time when one frame's worth of digital image data 70B is stored in memory 112. However, this is merely an example, and the imaging time may also be the time when exposure for one frame is completed, or the time when A / D conversion for one frame is completed, etc.

[0097] The exposure time refers to the exposure time required to obtain one frame of digital image data 70B. The image sensor sensitivity refers to, for example, the sensitivity (sensor gain) of the photoelectric conversion element 42. The deletion flag refers to a flag that indicates whether or not the digital image data 70B will be deleted from the memory 112 when the next image is captured. If the deletion flag is off, the digital image data 70B will not be deleted from the memory 112 when the next image is captured, and if the deletion flag is on, the digital image data 70B will be deleted from the memory 112 when the next image is captured. An example of the digital image data 70B that is deleted from the memory 112 is one frame of digital image data 70B that is deleted from the memory 112.

[0098] The exposure time and the image sensor sensitivity are examples of "imaging conditions" according to the technology of the present disclosure, and the erase flag in an ON state is an example of "erasure information" according to the technology of the present disclosure.

[0099] Here, exposure time and image sensor sensitivity are given as examples of "imaging conditions" according to the technology of the present disclosure, but the technology of the present disclosure is not limited to these, and the type of imaging lens 40, subject distance, focal length, angle of view, and / or the presence or absence of shake correction, etc. may be used instead of or in addition to the exposure time and / or image sensor sensitivity.

[0100] The control circuit 110C generates attribute information in response to an instruction regarding the attribute information from the controller 15, and outputs the generated attribute information to the output I / F 110D2. An instruction regarding the attribute information refers to, for example, an instruction from the controller 15 requesting the processing circuit 110 to output the attribute information. In this embodiment, a vertical synchronization signal is used as an example of an instruction regarding the attribute information. The output I / F 110D2 outputs the attribute information input from the control circuit 110C to the signal processing circuit 34.

[0101] Specifically, first, the reception I / F 110D1 receives information about the attribute information from the controller 15. A vertical synchronization signal is received as an instruction. Next, the control circuit 110C generates attribute information relating to the digital image data 70B stored least recently among the digital image data 70B stored in the memory 112 as attribute information corresponding to the vertical synchronization signal received by the reception I / F 110D1. Then, the output I / F 110D2 outputs the attribute information generated by the control circuit 110C to the signal processing circuit 34.

[0102] As an example, as shown in FIG. 9, the process performed by the image sensor 38 includes an image capturing process and an output process.

[0103] In the imaging process, exposure, reading of analog image data 70A, resetting of the photoelectric conversion element 42, analog signal processing, digital signal processing, first storage, acquisition of digital image data 70B, image processing, and second storage are performed in this order.

[0104] Exposure is performed by the photoelectric conversion element 42. Readout of the analog image data 70A, resetting of the photoelectric conversion element 42, and analog signal processing are performed by the readout circuit 110A. Readout of the analog image data 70A begins on condition that the vertical synchronization signal is received by the reception I / F 110D1. The period during which exposure is performed by the photoelectric conversion element 42 is the period during which reading of the analog image data 70A and resetting of the photoelectric conversion element 42 are not performed. Resetting of the photoelectric conversion element 42 refers to the operation of erasing residual charge of each pixel in the photoelectric conversion element 42. Exposure by the photoelectric conversion element 42 is performed between the previous reset of the photoelectric conversion element 42 by the readout circuit 110A and readout.

[0105] Digital signal processing is performed by digital processing circuit 110B. "First storage" refers to storing digital image data 70B obtained by performing digital signal processing in memory 112. "Acquisition of digital image data 70B" refers to acquiring digital image data 70B from memory 112. The first storage and acquisition of digital image data 70B are performed by control circuit 110C. Image processing is performed by image processing circuit 110E on digital image data 70B acquired by control circuit 110C. "Second storage" refers to storing digital image data 70B that has undergone image processing in memory 112. The second storage is performed by control circuit 110C.

[0106] In the output process, attribute information is generated and output. The generation of attribute information refers to the generation of attribute information for each frame regarding digital image data 70B stored in memory 112. The generation of attribute information is performed by control circuit 110C. The output of attribute information refers to the output of attribute information generated by control circuit 110C by output I / F 110D2.

[0107] 10 shows an example of the flow from reading image data of the first frame (1F) to the sixth frame (6F) to storing it in memory 112, and the output timing of attribute information for each frame. The "F" in 1F to 6F shown in FIG. 10 stands for "Frame." For ease of explanation, the example shown in FIG. 10 shows a state in which one frame of digital image data 70B is stored in each of the first storage area 112A, second storage area 112B, third storage area 112C, fourth storage area 112D, fifth storage area 112E, and sixth storage area 112F of memory 112.

[0108] 10, each time a vertical synchronization signal is received by the reception I / F 110D1, reading of one frame of analog image data 70A from the photoelectric conversion element 42 is started. In the example shown in FIG. 10, the first to sixth vertical synchronization signals are sequentially received by the reception I / F 110D1, and the analog image data of the first to sixth frames are read out. Then, reading of the data 70A starts.

[0109] The analog image data 70A is read out line by line from the first horizontal line to the last horizontal line of the photoelectric conversion element 42 in accordance with a horizontal synchronization signal, and when readout for each line is completed, each pixel of the horizontal line for which readout has been completed is reset. The analog image data 70A is converted into digital image data 70B, and the digital image data 70B is stored in the memory 112 in the order of the frames of the read analog image data 70A (image capture order) so that each frame can be distinguished and in a FIFO manner. Specifically, the latest digital image data 70B is overwritten and saved frame by frame in the first storage area 112A, second storage area 112B, third storage area 112C, fourth storage area 112D, fifth storage area 112E, and sixth storage area 112F in sequence. The overwriting and saving is repeatedly performed in the order of the first storage area 112A, the second storage area 112B, the third storage area 112C, the fourth storage area 112D, the fifth storage area 112E, and the sixth storage area 112F.

[0110] The output I / F 110D2 outputs attribute information related to the latest digital image data 70B of the digital image data 70B stored in the memory 112 to the signal processing circuit 34. In this case, first, the control circuit 110C generates attribute information related to the latest digital image data 70B of the digital image data 70B stored in the memory 112 each time storage of one frame of the latest digital image data 70B in the memory 112 is completed at the timing when the vertical synchronization signal is accepted by the acceptance I / F 110D1. Then, the output I / F 110D2 outputs the attribute information generated by the control circuit 110C to the signal processing circuit 34.

[0111] 11A and 11B, the attribute information of each frame (the first to sixth frames in the example shown in FIGS. 11A and 11B) is output by the output I / F 110D2 to the signal processing circuit 34. Then, the signal processing circuit 34 outputs the input attribute information to the controller 15.

[0112] 11A, the attribute information output for the first frame is attribute information related to digital image data 70B stored in the first storage area 112A. In the example shown in FIG. 11A, examples of the attribute information related to digital image data 70B stored in the first storage area 112A include the address of the first storage area 112A, the horizontal size of the digital image data 70B in the first storage area 112A, the vertical size of the digital image data 70B in the first storage area 112A, the image capture time related to the digital image data 70B in the first storage area 112A, the exposure time related to the digital image data 70B in the first storage area 112A, the image sensor sensitivity related to the digital image data 70B in the first storage area 112A, and an erase flag "off." Here, "erase flag "off" refers to information indicating that the erase flag is off.

[0113] 11A, examples of the attribute information related to the digital image data 70B stored in the second storage area 112B include the address of the second storage area 112B, the horizontal size of the digital image data 70B in the second storage area 112B, the vertical size of the digital image data 70B in the second storage area 112B, the image capture time related to the digital image data 70B in the second storage area 112B, the exposure time related to the digital image data 70B in the second storage area 112B, the image sensor sensitivity related to the digital image data 70B in the second storage area 112B, and the erase flag_off.

[0114] 11A, examples of the attribute information related to the digital image data 70B stored in the third storage area 112C include the address of the third storage area 112C, the horizontal size of the digital image data 70B in the third storage area 112C, the vertical size of the digital image data 70B in the third storage area 112C, the image capture time of the digital image data 70B in the third storage area 112C, the exposure time of the digital image data 70B in the third storage area 112C, the image sensor sensitivity of the digital image data 70B in the third storage area 112C, and the erase flag "off."

[0115] 11A, examples of the attribute information related to the digital image data 70B stored in the fourth storage area 112D include the address of the fourth storage area 112D, the horizontal size of the digital image data 70B in the fourth storage area 112D, the vertical size of the digital image data 70B in the fourth storage area 112D, the imaging time of the digital image data 70B in the fourth storage area 112D, the exposure time of the digital image data 70B in the fourth storage area 112D, the imaging element sensitivity of the digital image data 70B in the fourth storage area 112D, and the erase flag "off." Here, the "erasure flag_off" in the attribute information for the digital image data 70B stored in the first storage area 112A and included in the attribute information output for the fourth frame is changed to "erasure flag_on" by the control circuit 110C. "erasure flag_on" refers to information indicating that the erasure flag is on.

[0116] In this way, the "delete flag_off" in the attribute information related to the digital image data 70B stored in the first storage area 112A is changed to "delete flag_on" by the control circuit 110C. This means that the attribute information output from the output I / F 110D2 for the fourth frame includes information indicating that the digital image data 70B in the first storage area 112A will be deleted from the memory 112 when the next image is captured.

[0117] If the attribute information output for the fourth frame includes "deletion flag_on," the control circuit 110C erases the attribute information for the image data to be deleted, which is the digital image data 70B in the memory 112 that has been selected for erasure (e.g., at the start of imaging for the fifth frame, or during the period from the current time until imaging for the fifth frame starts), i.e., the attribute information for the first storage area 112A. The control circuit 110C also erases the digital image data 70B from the first storage area 112A. In this case, for example, the control circuit 110C first erases the attribute information for the first storage area 112A, and then erases the digital image data 70B from the first storage area 112A. Note that while an example in which the digital image data 70B and the attribute information are erased has been given here, the technology of the present disclosure is not limited to this. "Erasing" includes not only the actual erasure of the digital image data 70B and the attribute information, but also the process of freeing up the area and making it overwritable.

[0118] As an example, as shown in FIG. 11B, the attribute information output for the fifth frame includes attribute information related to the digital image data 70B stored in the second storage area 112B, attribute information related to the digital image data 70B stored in the third storage area 112C, attribute information related to the digital image data 70B stored in the fourth storage area 112D, and attribute information related to the digital image data 70B stored in the fifth storage area 112C. 11B, the attribute information for the digital image data 70B stored in the fifth storage area 112E includes information such as the address of the fifth storage area 112E, the horizontal size of the digital image data 70B in the fifth storage area 112E, the vertical size of the digital image data 70B in the fifth storage area 112E, the image capture time for the digital image data 70B in the fifth storage area 112E, the exposure time for the digital image data 70B in the fifth storage area 112E, the image sensor sensitivity for the digital image data 70B in the fifth storage area 112E, and an erase flag "off." Here, the attribute information for the digital image data 70B stored in the second storage area 112B, which is included in the attribute information to be output for the fifth frame, has its "erase flag "off" changed to "erase flag "on" by the control circuit 110C.

[0119] In this way, when the control circuit 110C changes the "deletion flag_off" in the attribute information related to the digital image data 70B stored in the second storage area 112B to "deletion flag_on," the control circuit 110C erases the attribute information related to the image data to be deleted, which is the digital image data 70B to be deleted from the memory 112, that is, the attribute information related to the second storage area 112B, upon the next imaging (for example, at the time when imaging of the sixth frame starts, or during the period from the present time until imaging of the sixth frame starts). The control circuit 110C also erases the digital image data 70B from the second storage area 112B. In this case, for example, upon the next imaging, first, the attribute information related to the second storage area 112B is erased by the control circuit 110C, and then the digital image data 70B is erased from the second storage area 112B by the control circuit 110C.

[0120] 11B, examples of the attribute information related to the digital image data 70B stored in the sixth storage area 112F include the address of the sixth storage area 112F, the horizontal size of the digital image data 70B in the sixth storage area 112F, the vertical size of the digital image data 70B in the sixth storage area 112F, the imaging time of the digital image data 70B in the sixth storage area 112F, the exposure time of the digital image data 70B in the sixth storage area 112F, the imaging element sensitivity of the digital image data 70B in the sixth storage area 112F, and the erase flag "off." Here, the "deletion flag_off" in the attribute information for the digital image data 70B stored in the third storage area 112C, which is included in the attribute information output for the sixth frame, is changed to "deletion flag_on" by the control circuit 110C.

[0121] In this way, when the control circuit 110C changes the "deletion flag_off" in the attribute information related to the digital image data 70B stored in the third storage area 112C to the "deletion flag_on," the control circuit 110C erases the attribute information related to the image data to be deleted, which is the digital image data 70B to be deleted from the memory 112, i.e., the attribute information related to the third storage area 112C, upon the next imaging (for example, at the time when imaging of the seventh frame starts, or during the period from the present time until imaging of the seventh frame starts). The control circuit 110C also erases the digital image data 70B from the third storage area 112C. In this case, for example, upon the next imaging, first, the attribute information related to the third storage area 112C is erased by the control circuit 110C, and then the digital image data 70B is erased from the third storage area 112C by the control circuit 110C.

[0122] In this way, the attribute information for each frame is transmitted to the signal processing circuit 34 by the output I / F 110D2. When the attribute information is transmitted to the controller 15, the signal processing circuit 34 transfers the attribute information to the controller 15. The attribute information transferred from the signal processing circuit 34 to the controller 15 is received by the reception I / F 15E of the controller 15. The controller 15 acquires an address as necessary from the attribute information received by the reception I / F 15E.

[0123] 12, the controller 15 outputs the address acquired from the attribute information to the reception I / F 110D1 of the imaging element 38 via the output I / F 15D. The reception I / F 110D1 of the imaging element 38 receives the address from the controller 15. The control circuit 110C acquires digital image data 70B from the memory 112 in accordance with the address received by the reception I / F 110D1, and causes the output I / F 110D2 to output the acquired digital image data 70B to the signal processing circuit 34.

[0124] Note that, although an example in which the address is output from the output I / F 15D of the controller 15 to the imaging element 38 has been described here, the technology of the present disclosure is not limited to this. For example, the controller 15 may further include a transmission I / F, and the address may be transmitted to the imaging element 38 from the transmission I / F.

[0125] Although the embodiment has been described above with reference to an example in which the address is received by the reception I / F 110D1, the technology of the present disclosure is not limited to this. For example, the image sensor 38 may further include a reception I / F, and the address transmitted from the controller 15 may be received by the reception I / F of the image sensor 38.

[0126] Next, the operation of the imaging device 10 will be described.

[0127] First, the flow of the attribute information output process executed by the processing circuit 110 of the image sensor 38 will be described with reference to FIG.

[0128] 13, first, in step ST10, the control circuit 110C determines whether or not a vertical synchronization signal has been received by the reception I / F 110D1. If the vertical synchronization signal has not been received by the reception I / F 110D1 in step ST10, the determination is negative, and the attribute information output process proceeds to step ST22. If the vertical synchronization signal has been received by the reception I / F 110D1 in step ST10, the determination is positive, and the attribute information output process proceeds to step ST12.

[0129] In step ST12, the control circuit 110C stores the digital image data 70B obtained by capturing an image in the memory 112, and then the attribute information output process proceeds to step ST14.

[0130] In step ST14, the control circuit 110C generates attribute information for the digital image data 70B stored in the memory 112 in step ST12, and stores the generated attribute information, after which the attribute information output process proceeds to step ST16.

[0131] In step ST16, the control circuit 110C determines whether the number of frames of digital image data 70B that can be stored in the memory 112 will reach the upper limit when the next image is captured. In this embodiment, the memory 112 has six storage areas: a first storage area 112A, a second storage area 112B, a third storage area 112C, a fourth storage area 112D, a fifth storage area 112E, and a sixth storage area 112F, so the "upper limit" referred to here is "6." Note that the upper limit is not limited to this and may be any number of frames that can be stored in the memory 112.

[0132] In step ST16, if the number of frames of digital image data 70B that can be stored in memory 112 due to the next imaging does not reach the upper limit, the determination is negative and the attribute information output processing proceeds to step ST20. In step ST16, if the number of frames of digital image data 70B that can be stored in memory 112 due to the next imaging reaches the upper limit, the determination is positive and the attribute information output processing proceeds to step ST18.

[0133] In step ST18, the control circuit 110C turns on the deletion flag included in the attribute information for the one frame of digital image data 70B that was stored in memory 112 the earliest out of all frames of digital image data 70B stored in memory 112, and then the attribute information output processing proceeds to step ST20.

[0134] In step ST20, the control circuit 110C causes the output I / F 110D2 to output the currently held attribute information to the signal processing circuit 34, and then the attribute information output process proceeds to step ST22.

[0135] In step ST22, the control circuit 110C determines whether or not a condition for terminating the attribute information output process (hereinafter referred to as the "attribute information output process termination condition") has been satisfied. One example of the attribute information output process termination condition is that an instruction to terminate the attribute information output process has been accepted by the acceptance device 84 (see FIG. 4). In step ST22, if the attribute information output process termination condition is not satisfied, the determination is negative, and the attribute information output process proceeds to step ST10. In step ST22, if the attribute information output process termination condition is satisfied, the determination is positive, and the attribute information output process ends.

[0136] Next, the flow of the data erasure process executed by the processing circuit 110 of the image sensor 38 will be described with reference to FIG.

[0137] 14, first, in step ST50, control circuit 110C determines whether attribute information has been output by output I / F 110D2. If attribute information has not been output by output I / F 110D2 in step ST50, the determination is negative, and the data erasure process proceeds to step ST60. If attribute information has been output by output I / F 110D2 in step ST50, the determination is positive, and the data erasure process proceeds to step ST52.

[0138] In step ST52, the control circuit 110C determines whether or not a vertical synchronization signal has been received by the reception I / F 110D1. If the vertical synchronization signal has not been received by the reception I / F 110D1 in step ST52, the determination is negative, and the determination of step ST52 is made again. If the vertical synchronization signal has been received by the reception I / F 110D1 in step ST52, the determination is positive, and the data erasure process proceeds to step ST54.

[0139] In step ST54, the control circuit 110C determines whether or not attribute information with an erase flag set to on is held. If attribute information with an erase flag set to on is not held in step ST54, the determination is negative, and the data erasure process proceeds to step ST60. If attribute information with an erase flag set to on is held in step ST54, the determination is positive, and the data erasure process proceeds to step ST56.

[0140] In step ST56, the control circuit 110C first identifies the digital image data 70B from the address included in the attribute information whose deletion flag is set to ON among all the attribute information currently held. Of these, the control circuit 110C erases the attribute information whose erasure flag is set to ON. Then, the control circuit 110C erases the digital image data 70B identified from the address from the memory 112, and then the data erasure process proceeds to step ST60.

[0141] In step ST60, the control circuit 110C determines whether or not a condition for terminating the data erasure process (hereinafter referred to as the "data erasure process termination condition") has been satisfied. One example of the data erasure process termination condition is that an instruction to terminate the data erasure process has been accepted by the accepting device 84 (see FIG. 4). If the data erasure process termination condition has not been satisfied in step ST60, the determination is negative, and the data erasure process proceeds to step ST50. If the data erasure process termination condition has been satisfied in step ST60, the determination is positive, and the data erasure process ends.

[0142] As described above, in the imaging device 10, instructions regarding attribute information are received by the reception I / F 110D1, and attribute information corresponding to the received instructions is output by the output I / F 110D2 to the subsequent circuit 13. This allows the subsequent circuit 13 to request the image sensor 38 to provide digital image data 70B by using the attribute information. Therefore, the subsequent circuit 13 can selectively obtain necessary digital image data 70B from the image sensor 38. Furthermore, the power consumption required for output by the image sensor 38 can be reduced compared to when both the digital image data 70B and at least a portion of the attribute information are output simultaneously.

[0143] Furthermore, in the imaging device 10, the control circuit 110C generates attribute information according to the vertical synchronization signal from the controller 15, and the generated attribute information is output by the output I / F 110D2 to the signal processing circuit 34. Therefore, the subsequent circuit 13 can acquire the attribute information from the imaging element 38 without newly generating and using a signal different from the vertical synchronization signal.

[0144] Furthermore, in the imaging device 10, the output I / F 110D2 outputs attribute information relating to the latest digital image data 70B among all the digital image data 70B stored in the memory 112. Therefore, the subsequent circuit 13 can acquire the attribute information relating to the latest digital image data 70B.

[0145] Furthermore, in the imaging device 10, attribute information related to the latest digital image data 70B at the timing when the vertical synchronization signal is received by the reception I / F 110D1 is output by the output I / F 110D2. Therefore, the subsequent circuit 13 can acquire attribute information related to the latest digital image data 70B at the timing when the vertical synchronization signal is received by the reception I / F 110D1.

[0146] Furthermore, in the imaging device 10, the output I / F 110D2 can output attribute information for each of multiple frames of digital image data 70B, and the attribute information is output by the output I / F 110D2 in the order in which the images were captured. Therefore, the subsequent circuit 13 can obtain the digital image data 70B from the imaging element 38 in the order in which the images were captured.

[0147] Furthermore, the image capturing device 10 employs information including the address, image size, image capturing time, and image capturing conditions as the attribute information, which allows for a wider range of uses for the digital image data 70B acquired by the subsequent circuit 13 from the image capturing element 38, compared to when the attribute information does not include any of the address, image size, image capturing time, and image capturing conditions.

[0148] In addition, in the imaging device 10, when the digital image data 70B is erased from the memory 112 upon the next imaging, the attribute information output from the output I / F 110D2 includes an erase flag that is set to ON. It is possible to prevent the output of the data 70B from being instructed by the subsequent circuit 13.

[0149] Furthermore, in imaging device 10, when digital image data 70B is erased from memory 112 following the next imaging, attribute information whose erase flag is set is erased from memory 112 by control circuit 110C, and digital image data 70B identified from the address included in the attribute information whose erase flag is set is erased from memory 112 by control circuit 110C. Therefore, both the attribute information and digital image data 70B related to the attribute information can be erased at different times.

[0150] Furthermore, in the imaging device 10, the attribute information transmitted from the controller 15 is received by the reception I / F 110D1. In the control circuit 110C, the digital image data 70B is acquired from the memory 112 in accordance with the attribute information received by the reception I / F 110D1, and the acquired digital image data 70B is output to the signal processing circuit 34 by the output I / F 110D2. Therefore, the imaging element 38 can output the digital image data 70B requested by the subsequent circuit 13 to the subsequent circuit 13.

[0151] Furthermore, the imaging device 10 employs an imaging element in which the photoelectric conversion element 42, processing circuit 110, and memory 112 are integrated into a single chip as the imaging element 38. This increases the portability of the imaging element 38 compared to imaging elements in which the photoelectric conversion element 42, processing circuit 110, and memory 112 are not integrated into a single chip. This also increases the degree of freedom in design compared to imaging elements in which the photoelectric conversion element 42, processing circuit 110, and memory 112 are not integrated into a single chip. Furthermore, this contributes to the miniaturization of the imaging device 10 compared to imaging elements in which the photoelectric conversion element 42, processing circuit 110, and memory 112 are not integrated into a single chip.

[0152] 6, a stacked imaging element in which a memory 112 is stacked on a photoelectric conversion element 42 is employed as the imaging element 38. This allows for a faster transfer rate of image data from the photoelectric conversion element 42 to the memory 112 compared to when the photoelectric conversion element 42 and the memory 112 are not stacked. The improved transfer rate also contributes to faster processing in the processing circuit 110 as a whole. Furthermore, this allows for greater design freedom compared to when the photoelectric conversion element 42 and the memory 112 are not stacked. Furthermore, this also contributes to a more compact imaging device 10 compared to when the photoelectric conversion element 42 and the memory 112 are not stacked.

[0153] Furthermore, in the imaging device 10, a live view image or the like based on the digital image data 70B output by the output I / F 110D2 is displayed by the CPU 15A on the display 26. This allows the user to visually recognize the live view image or the like based on the digital image data 70B output by the output I / F 110D2.

[0154] Furthermore, in the imaging device 10, the digital image data 70B output by the output I / F 110D2 is stored by the CPU 15A in the storage 15B, a smart device, a personal computer, a server, a USB memory, and / or a memory card, etc. This allows the digital image data 70B output by the output I / F 110D2 to be managed.

[0155] In the above embodiment, an example in which digital image data 70B for multiple frames is stored in memory 112 has been described, but the technology of the present disclosure is not limited to this. For example, compressed image data may be stored in memory 112 in addition to digital image data 70B for multiple frames. The compressed image data is obtained by a compression process performed by image processing circuit 110E. Compression refers to a process of compressing digital image data 70B. In other words, in addition to digital image data 70B for multiple frames, the memory 112 may store compressed image data by image processing circuit 110E, which is an example of a "compression circuit" according to the technology of the present disclosure. Alternatively, compressed image data obtained by compressing the image data 70B may be stored.

[0156] Specifically, the control circuit 110C stores a predetermined number of frames (four frames in the example shown in FIG. 15) of digital image data 70B in the memory 112. The control circuit 110C also stores compressed image data obtained by compressing the digital image data by the image processing circuit 110E in the memory 112. Furthermore, the control circuit 110C associates attribute information related to the compressed image data and compression specification information that can specify that the image data has been compressed with the compressed image data.

[0157] In this case, as shown in Fig. 15 as an example, the memory 112 has addresses "0X00540000," "0X00580000," and "0X005C0000" in addition to the addresses described in the above embodiment, with each address having a storage area for one frame. In the example shown in Fig. 15, when digital image data 70B for the fourth frame is stored in the fourth storage area 112D (the storage area with the address "0X00300000"), the image processing circuit 110E compresses the digital image data 70B for the fourth frame. The compressed image data ("compressed data" in the example shown in Fig. 15) obtained by compressing the digital image data 70B for the fourth frame is stored in the fifth storage area 112E (the storage area with the address "0X00400000") by the control circuit 110C.

[0158] 15, upon completion of imaging of the fourth frame, digital image data 70B for the first frame is deleted from first storage area 112A (storage area with address "0X00000000") by control circuit 110C, and digital image data 70B for the fifth frame is stored in first storage area 112A by control circuit 110C. Then, when digital image data 70B for the fifth frame is stored in first storage area 112A, image processing circuit 110E compresses digital image data 70B for the fifth frame. Compressed image data obtained by compressing digital image data 70B for the fifth frame is stored in sixth storage area 112F (storage area with address "0X00500000") by control circuit 110C.

[0159] Furthermore, in the example shown in FIG. 15, upon completion of imaging of the fifth frame, digital image data 70B for the second frame is erased from second storage area 112B (storage area with address "0X00100000") by control circuit 110C, and digital image data 70B for the sixth frame is stored in second storage area 112B by control circuit 110C. Then, when digital image data 70B for the sixth frame is stored in second storage area 112B, image processing circuit 110E compresses digital image data 70B for the sixth frame. Compressed image data obtained by compressing digital image data 70B for the sixth frame is stored in seventh storage area 112G (storage area with address "0X00540000") by control circuit 110C.

[0160] In this way, the digital image data 70B for the fourth frame and thereafter is compressed by the image processing circuit 110E, and the compressed image data obtained by the compression is stored in an available storage area of ​​the memory 112 by the control circuit 110C.

[0161] Furthermore, when the digital image data 70B from the fourth frame onwards is compressed and stored in the memory 112 as compressed image data, attribute information is generated by the control circuit 110C as shown in Figs. 16A and 16B as an example.

[0162] As an example, as shown in FIG. 16A, the attribute information output in the first to third frames is the same as that in FIG. 11A. 16A, the attribute information for the fourth frame digital image data 70B includes an address of "0X00300000" that can identify the fourth storage area 112D, an image size, an image capture time, image capture conditions, and an erase flag of "OFF." The erase flag for the attribute information for the first frame included in the attribute information output for the fourth frame is set to "ON." As a result, the digital image data 70B for the first frame is erased from the first storage area 112A when the fifth frame is captured. The attribute information output for the fourth frame also includes attribute information for the compressed image data of the digital image data 70B for the fifth frame. In the example shown in FIG. 16A, the attribute information for the compressed image data of the digital image data 70B for the fifth frame includes an address of "0X00400000" that can identify the fifth storage area 112E, an image size, an image capture time, image capture conditions, and compression specification information.

[0163] As an example, as shown in FIG. 16B, the attribute information output for the fifth frame includes attribute information related to digital image data 70B for the fifth frame. In the example shown in FIG. 16B, the attribute information related to digital image data 70B for the fifth frame includes an address of "0X00000000" that can identify the first storage area 112A, an image size, an image capture time, image capture conditions, and an erase flag of "off." The erase flag for the attribute information for the second frame included in the attribute information output for the fifth frame is set to "on." As a result, digital image data 70B for the second frame is erased from the second storage area 112B when the sixth frame is captured. The attribute information output for the fifth frame also includes attribute information related to the compressed image data of digital image data 70B for the fifth frame and attribute information related to the compressed image data of digital image data 70B for the sixth frame. In the example shown in Figure 16B, the attribute information for the compressed image data of the sixth frame of digital image data 70B includes the address "0X00500000" that can identify the sixth memory area 112F, the image size, the shooting time, the shooting conditions, and the compression specification information.

[0164] As an example, as shown in FIG. 16B, the attribute information output for the sixth frame includes attribute information related to digital image data 70B for the sixth frame. In the example shown in FIG. 16B, the attribute information related to digital image data 70B for the sixth frame includes an address of "0X01000000" that can identify the second storage area 112B, an image size, an image capture time, image capture conditions, and an erase flag of "off." The erase flag for the attribute information for the third frame included in the attribute information output for the sixth frame is set to "on." As a result, digital image data 70B for the third frame is erased from the third storage area 112C upon capturing the seventh frame. The attribute information output for the fifth frame also includes attribute information related to the compressed image data of digital image data 70B for the fifth frame, attribute information related to the compressed image data of digital image data 70B for the sixth frame, and attribute information related to the compressed image data of digital image data 70B for the seventh frame. In the example shown in FIG. 16B, the attribute information relating to the compressed image data of the seventh frame of digital image data 70B includes an address of "0X00540000", an image size, an image capturing time, image capturing conditions, and compression specification information.

[0165] In this way, control circuit 110C stores digital image data 70B for a predetermined number of frames in memory 112, control circuit 110C also stores compressed image data in memory 112, and control circuit 110C associates attribute information related to the compressed image data with compression specification information (in the example shown in FIGS. 16A and 16B, the compression specification information is included in the attribute information). Therefore, even when digital image data 70B and compressed image data are stored in memory 112, the digital image data 70B and the compressed image data can be identified in a distinguishable manner.

[0166] In the example shown in FIGS. 16A and 16B, the attribute information regarding the digital image data 70B includes the following: Although an example embodiment has been shown in which the address, image size, image capture time, image capture conditions, and deletion flag are included, the technology of the present disclosure is not limited to this. For example, when compressed image data is stored in memory 112, as shown in Figures 17A and 17B, the attribute information for digital image data 70B may include, in addition to the address, image size, image capture time, image capture conditions, and deletion flag, a compression schedule flag indicating whether the compressed image data is scheduled to be stored in memory 112. A compression schedule flag that is turned on is an example of "compression schedule information" according to the technology of the present disclosure, and indicates that compressed image data is scheduled to be stored in memory 112 with the next image capture. A compression schedule flag that is turned off indicates that compressed image data is not scheduled to be stored in memory 112 with the next image capture.

[0167] When the compression-to-be-compressed flag is turned on, the digital image data 70B associated with the attribute information for which the compression-to-be-compressed flag is turned on is compressed by the image processing circuit 110E in association with the next imaging. For example, as shown in FIG. 17A, the attribute information for the first frame of digital image data 70B included in the attribute information output for the third frame includes "compression-to-be-compressed flag_on," indicating that the compression-to-be-compressed flag is on. In this case, the digital image data 70B for the first frame is compressed by the image processing circuit 110E in association with the next imaging (imaging of the fourth frame), and the compressed image data obtained by the compression is stored in the fifth storage area 112E. Accordingly, the attribute information output for the fourth frame includes attribute information associated with the compressed image data stored in the fifth storage area 112E (the storage area with the address "0X00400000"). The attribute information associated with the compressed image data stored in the fifth storage area 112E includes compression identification information, which enables the subsequent circuit 13 to identify that compressed image data is stored in the fifth storage area 112E.

[0168] 17A and 17B, when the erasure flag is turned on by the control circuit 110C, the compression to be flag is also turned on by the control circuit 110C. As a result, even if the digital image data 70B is erased from the memory 112, the compressed image data obtained by compressing the digital image data 70B before erasure is held in the memory 112.

[0169] In this way, when compressed image data is stored in memory 112, the attribute information output from output I / F 110D2 includes an on-set compression flag, thereby giving the output destination of the compressed image data (e.g., subsequent circuit 13) time to prepare for accepting the compressed image data.

[0170] Note that examples of the compression format when digital image data 70B is compressed into compressed image data include MPEG, JPEG, TIFF, PNG, GIF, and BMP. When the compressed image data is stored in memory 112, as shown in FIGS. 18A and 18B, the attribute information output from output I / F 110D2 may include compression format specification information that specifies the compression format of the compressed image data. In the example shown in FIGS. 18A and 18B, the compression format specification information is included in the attribute information together with the compression specification information. This allows the output destination of the compressed image data (e.g., subsequent circuit 13) to perform processing according to the compression format.

[0171] Furthermore, in the above embodiment, an example was given in which attribute information is output by the output I / F 110D2 after one frame of digital image data 70B is stored in the memory 112, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 19, attribute information may be output by the output I / F 110D2 at the timing when an instruction regarding attribute information (for example, a vertical synchronization signal) is received by the reception I / F 110D1. This allows the attribute information to be provided to the subsequent circuit 13 in real time in response to an instruction regarding the attribute information.

[0172] In the above embodiment, an example was described in which the attribute information output for each frame includes attribute information for multiple frames, but the technology of the present disclosure is not limited to this, and for example, the attribute information output for each frame may be only attribute information related to the digital image data 70B of the corresponding frame, as shown in Fig. 20. In this case, the amount of attribute information output to the signal processing circuit 34 for each frame can be reduced compared to when the attribute information output for each frame includes attribute information for multiple frames.

[0173] 20, the attribute information output for each frame includes an erasure flag and a compression planned flag, but the technology of the present disclosure is not limited to this, and for example, the attribute information output for each frame may not include an erasure flag and a compression planned flag, as shown in Fig. 21. Furthermore, the attribute information output for each frame may not include an erasure flag, but may include a compression planned flag.

[0174] Furthermore, in the above embodiment, an example form was described in which attribute information for each of all frames is output by output I / F 110D2, but the technology of the present disclosure is not limited to this, and the downstream circuit 13 may instruct the image sensor 38 on how many frames of attribute information to output to output I / F 110D2.

[0175] In this case, for example, instead of the attribute information output process shown in FIG. 13, the attribute information output process shown in FIG. 22 is executed by the processing circuit 110 of the image sensor 38.

[0176] The attribute information output process shown in FIG. 22 differs from the attribute information output process shown in FIG. 13 in that it has step ST10A instead of step ST10, step ST18A instead of step ST18, and step ST20A instead of step ST20. The controller 15 outputs an attribute information request instruction to the image sensor 38 via the output I / F 15D. The attribute information request instruction includes a vertical synchronization signal and output amount information. The output amount information refers to information indicating the amount of attribute information output by the output I / F 110D2 of the image sensor 38. The "output amount" here is defined, for example, in terms of the number of frames.

[0177] 22, in step ST10A, the control circuit 110C determines whether the attribute information request instruction output from the controller 15 has been accepted by the reception I / F 110D1. If the attribute information request instruction has not been accepted by the reception I / F 110D1 in step ST10A, the determination is negative, and the attribute information output processing proceeds to step ST22. If the attribute information request instruction has been accepted by the reception I / F 110D1 in step ST10A, the determination is positive, and the attribute information output processing proceeds to step ST12.

[0178] In step ST18A, control circuit 110C selects, from the attribute information currently held, attribute information to be output to signal processing circuit 34 via output I / F 110D2, in accordance with the output amount indicated by the output amount information included in the attribute information request instruction. For example, if the output amount indicated by the output amount information is one frame, control circuit 110C selects attribute information related to one frame of the latest digital image data 70B, and if the output amount indicated by the output amount information is two frames, control circuit 110C selects attribute information related to two frames of the latest digital image data 70B.

[0179] In step ST20A, the control circuit 110C causes the output I / F 110D2 to output the attribute information selected in step ST18A to the signal processing circuit 34. This prevents the I / F 110D2 from outputting more attribute information than necessary. In addition, since the output amount is specified in terms of the number of frames, the subsequent circuit 13 can process the attribute information in units of frames.

[0180] Furthermore, in the above embodiment, an example has been described in which the attribute information and digital image data 70B are output to the signal processing circuit 34 by the output I / F 110D2, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 23, the attribute information and digital image data 70B may be output to the signal processing circuit 34 by the output I / F 110D2 and the output I / F 110D3. In the example shown in FIG. 23, the processing circuit 110 has the output I / F 110D2 and the output I / F 110D3. The output I / F 110D3 is an example of a "first output unit" according to the technology of the present disclosure, and the output I / F 110D2 is an example of a "second output unit" according to the technology of the present disclosure.

[0181] The output I / F 110D3 is connected to the control circuit 110C. The signal processing circuit 34 also includes a reception I / F 34C. The output I / F 110D2 outputs attribute information to the signal processing circuit 34. The attribute information output by the output I / F 110D2 is received by the reception I / F 34A of the signal processing circuit 34. The output I / F 110D3 also outputs digital image data 70B to the signal processing circuit 34 under the control of the control circuit 110C. The digital image data 70B output by the output I / F 110D3 is received by the reception I / F 34C of the signal processing circuit 34.

[0182] According to the configuration shown in FIG. 23, the attribute information is output by the output I / F 110D2, and the digital image data 70B is output by the output I / F 110D3, so that the attribute information and the digital image data 70B can be output in parallel.

[0183] While the above embodiment illustrates an interchangeable-lens imaging device 10, the technology of the present disclosure is not limited to this. For example, as shown in Fig. 24, a smart device 200 may be equipped with an imaging device main body 214 having a configuration and functions equivalent to those of the imaging device main body 12 described in the above embodiment.

[0184] 24, a smart device 200 includes a housing 212 that houses an imaging device main body 214. An example of the smart device 200 is a smartphone. Here, examples of the smart device 200 include a smartphone or a tablet terminal, which is an electronic device with an imaging function.

[0185] An imaging lens 216 is attached to the housing 212. In the example shown in FIG. 24 , the imaging lens 216 is exposed from the rear surface 212A of the housing 212 at the upper left of the rear surface 212A when the smart device 200 is placed vertically. The center of the imaging lens 216 is located on the optical axis L1. An imaging element 38 is built into the imaging device main body 214. The imaging device main body 214 captures subject light through the imaging lens 216. The subject light captured in the imaging device main body 214 is focused on the imaging element 38, and an image is captured as described in the above embodiment.

[0186] As an example, as shown in FIG. 25, command keys 222 and a touch panel display 224 are provided on the front surface 212B of the housing 212. When the smart device 200 is placed vertically, the command keys 222 are arranged on the lower part of the front surface 212B, and the touch panel display 224 is arranged above the command keys 222. The touch panel display 224 includes a display 226 and a touch panel 228, and the touch panel 228 is superimposed on the display 226. The touch panel display 224 is provided with a display 226 and a touch panel 228. The touch panel display 224 and the instruction keys 222 have the same functions as the UI device 17 described in the above embodiment. That is, the touch panel display 224 and the instruction keys 222 present information to the user and receive instructions from the user. Instructions from the user are realized, for example, by touching soft keys displayed on the touch panel display 224 (for example, soft keys corresponding to the release button 20, etc.).

[0187] The display 226 is an example of a "display unit (display)" according to the technology of the present disclosure. The smart device 200 is an example of an "imaging device" according to the technology of the present disclosure. In addition, the example shown in FIG. 24 illustrates an example in which only the imaging device main body 214 is built into the smart device 200, but the technology of the present disclosure is not limited to this. The smart device 200 may have multiple digital cameras built in, and in this case, it is sufficient that at least one digital camera is equipped with the imaging device main body 214.

[0188] In the above embodiment, an image sensor in which the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single chip is exemplified as the image sensor 38, but the technology of the present disclosure is not limited to this. For example, it is sufficient if at least the photoelectric conversion element 42 and the memory 112 out of the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single chip.

[0189] In addition, in the above embodiment, the attribute information is exemplified as information including an address, an image size, an image capture time, and image capture conditions, but the technology of the present disclosure is not limited to this. For example, the attribute information may be information including an address among the address, image size, image capture time, and image capture conditions, or may be information including an address, an image size, an image capture time, and / or image capture conditions.

[0190] In the above embodiment, the communication interfaces are connected in accordance with the PCI-e connection standard, but the technology of the present disclosure is not limited to this. Instead of the PCI-e connection standard, other connection standards such as LVDS, SATA, SLVS-EC, or MIPI may be adopted.

[0191] In the above embodiment, the communication between the imaging element 38 and the signal processing circuit 34, the communication between the controller 15 and the imaging element 38, and the communication between the signal processing circuit 34 and the controller 15 are all wired communications. However, the technology of the present disclosure is not limited to this. The communication between the imaging element 38 and the signal processing circuit 34, the communication between the controller 15 and the imaging element 38, and / or the communication between the signal processing circuit 34 and the controller 15 may be wireless communications.

[0192] Furthermore, in the above embodiment, an example in which the UI device 17 is incorporated into the imaging device main body 12 has been described, but at least some of the components included in the UI device 17 may be externally attached to the imaging device main body 12. Furthermore, at least some of the components included in the UI device 17 may be used by being connected to the external I / F 104 as separate entities.

[0193] Furthermore, in the above embodiment, 120 fps is exemplified as the frame rate, but the technology of the present disclosure is not limited to this, and the frame rate for imaging (for example, the frame rate applied in the imaging process shown in FIG. 9) and the frame rate for output (for example, the frame rate applied in the output process shown in FIG. 9 and / or the frame rate for outputting the digital image data 70B) may be different frame rates. The frame rate for imaging and / or the frame rate for output may be a fixed frame rate or a variable frame rate. In the case of a variable frame rate, for example, a predetermined condition (for example, a condition that an instruction to change the frame rate has been accepted by the accepting device 84 and / or a condition that a predetermined timing for changing the frame rate has arrived) may be satisfied. The frame rate may be changed when the condition is satisfied. In the case of a variable frame rate, the specific value of the frame rate may be changed, for example, according to an instruction received by the receiving device 84, or according to the operating rate of the subsequent circuit 13 and / or the image sensor 38.

[0194] Furthermore, in the above embodiment, the signal processing circuit 34 is exemplified, but the technology of the present disclosure is not limited to this, and one or more signal processing circuits may be used in addition to the signal processing circuit 34. In this case, the image sensor 38 may be directly connected to each of the multiple signal processing circuits.

[0195] In the above embodiment, the processing circuit 110 is implemented by a device including an ASIC and an FPGA, but the technology of the present disclosure is not limited to this. For example, at least the control circuit 110C among the multiple devices included in the processing circuit 110 may be implemented by a software configuration using a computer.

[0196] 26 , for example, the image sensor 38 has a built-in computer 852, and an attribute information output program 902 for causing the computer 852 to execute the attribute information output process according to the embodiment, and a data erasure program 904 for causing the computer 852 to execute the data erasure process according to the embodiment are stored in a storage medium 900. An example of the storage medium 900 is any portable storage medium such as an SSD or USB memory, which is a non-transitory storage medium. Hereinafter, for convenience of explanation, the attribute information output process and the data erasure process will be referred to as "image sensor-side processing" when there is no need to distinguish between them, and the attribute information output program 902 and the data erasure program 904 will be referred to as "image sensor-side programs" when there is no need to distinguish between them.

[0197] The computer 852 includes a CPU 852A, a storage 852B, and a memory 852C. The storage 852B is a non-volatile storage device such as an EEPROM, and the memory 852C is a volatile storage device such as a RAM. The image sensor-side program stored in the storage medium 900 is installed in the computer 852. The CPU 852A executes image sensor-side processing in accordance with the image sensor-side program.

[0198] The imaging element-side program may be stored in storage 852B instead of storage medium 900. In this case, CPU 852A reads out the imaging element-side program from storage 852B and loads the read imaging element-side program into memory 852C. Then, CPU 852A executes imaging element-side processing in accordance with the imaging element-side program loaded into memory 852C.

[0199] Further, although an example is given here in which the attribute information output program 902 and the data erasure program 904 are stored in storage 852B, this is not limiting, and one of the attribute information output program 902 and the data erasure program 904 may be stored in storage 852B, and the other may be stored in storage medium 900.

[0200] In addition, the imaging element side program may be stored in a memory unit of another computer or server device connected to the computer 852 via a communication network (not shown), and the imaging element side program may be downloaded and installed in the computer 852 in response to a request from the imaging device 10 or smart device 200 described above.

[0201] It is not necessary to store all of the image sensor-side programs in the memory of another computer or server device connected to the computer 852; only a portion of the image sensor-side programs (for example, the attribute information output program 902 or the data erasure program 904) may be stored.

[0202] In the example shown in Figure 26, an example is shown in which the computer 852 is built into the imaging element 38, but the technology of the present disclosure is not limited to this, and for example, the computer 852 may be provided outside the imaging element 38.

[0203] 26, the CPU 852A is a single CPU, but it may be a plurality of CPUs. Also, a GPU may be applied instead of the CPU 852A.

[0204] 26 illustrates a computer 852, the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 852. Furthermore, a combination of a hardware configuration and a software configuration may be used instead of the computer 852.

[0205] The hardware resources for executing the image sensor-side processing described in the above embodiments can be various processors, as listed below. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource for executing image sensor-side processing by executing software, i.e., a program, as described above. Examples of processors include dedicated electrical circuits, such as FPGAs, PLDs, or ASICs, which are processors with circuit configurations designed specifically for executing specific processing. Each processor has built-in or connected memory, and each processor uses the memory to execute image sensor-side processing.

[0206] The hardware resource that executes the image sensor-side processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the image sensor-side processing may be a single processor.

[0207] Examples of a single processor include: first, a form in which one processor is configured by combining one or more CPUs and software, as typified by computers such as client and server, and this processor functions as a hardware resource that executes image sensor-side processing; second, a form in which a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute image sensor-side processing, on a single IC chip, as typified by SoC; thus, image sensor-side processing is realized using one or more of the above-mentioned various processors as hardware resources.

[0208] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0209] 1 illustrates the image capture device 10, and the example shown in Fig. 24 illustrates the smart device 200, the technology of the present disclosure is not limited to this. That is, the technology of the present disclosure can be applied to various electronic devices (for example, fixed-lens cameras, personal computers, wearable terminal devices, etc.) that incorporate an image capture device having a configuration and functions equivalent to the image capture device main body 12 described in the above embodiment, and these electronic devices can also obtain the same effects and advantages as the image capture device 10 and the smart device 200.

[0210] In addition, although the above embodiment illustrates the displays 26 and 226, the technology of the present disclosure is not limited to this. For example, a separate display attached to the imaging device may be used as the "display unit" according to the technology of the present disclosure.

[0211] Furthermore, the various processes described above are merely examples, and it goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed, without departing from the spirit of the invention.

[0212] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0213] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0214] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0215] The following additional notes are provided regarding the above-described embodiments.

[0216] (Appendix 1) a processor; a memory that stores image data obtained by capturing an image and is built into the image capturing element, The processor is Controlling the storage of the image data in the memory; storing attribute information of the image data in the memory; outputting the image data stored in the memory; Accept instructions regarding the above attribute information, Output the attribute information according to the received instruction. Image sensor.

Claims

1. a memory that stores image data obtained by capturing an image at a first frame rate; a control circuit that controls the storage of the image data in the memory and stores attribute information of the image data in the memory; an output interface that outputs the image data stored in the memory at a second frame rate different from the first frame rate; a reception interface that receives instructions regarding the attribute information, The output interface outputs the attribute information according to the instruction received by the reception interface. Image sensor.

2. The imaging device according to claim 1 , wherein the output interface outputs the attribute information at the timing when the instruction is accepted by the acceptance interface.

3. 3. The imaging device according to claim 1, wherein the instruction is an external frame synchronization signal.

4. 4. The imaging device according to claim 1, wherein the output interface outputs the attribute information relating to the latest image data among the image data stored in the memory.

5. The imaging device according to claim 4 , wherein the output interface outputs the attribute information relating to the latest image data at the timing when the instruction is accepted by the acceptance interface.

6. the output interface is capable of outputting attribute information for each of the plurality of image data; The imaging device according to claim 1 , wherein the attribute information is output by the output interface in the order in which the images are captured.

7. The imaging device according to claim 1 , wherein the attribute information includes at least one of an address, an image size, an image capturing time, and an image capturing condition.

8. 8. The imaging element according to claim 1, wherein, when the image data is erased from the memory upon the next image capture, the attribute information output from the output interface is information including erasure information indicating that the image data will be erased from the memory upon the next image capture.

9. 9. The imaging element according to claim 8, wherein when the image data is erased from the memory, the control circuit erases from the memory the attribute information relating to the image data to be erased, which is the image data in the memory that has been selected for erasure, and erases the image data to be erased from the memory.

10. a compression circuit for compressing the image data; The control circuit stores the image data for a predetermined number of frames in the memory, stores compressed image data obtained by compressing the image data by the compression circuit in the memory, and associates the attribute information regarding the compressed image data and compression identification information that can identify that the image data has been compressed with the compressed image data.

11. The imaging element according to claim 10, wherein when the compressed image data is stored in the memory, the attribute information output from the output interface is information including compression plan information indicating that the compressed image data is scheduled to be stored in the memory.

12. 12. The image sensor according to claim 10, wherein the attribute information output from the output interface includes information capable of identifying a compression format of the compressed image data.

13. the instruction includes an output amount of the attribute information by the output interface; The imaging device according to claim 1 , wherein the output interface outputs the attribute information at the output amount.

14. The image sensor according to claim 13 , wherein the output amount is defined by the number of frames of the image data.

15. a receiver for receiving the attribute information transmitted from a subsequent circuit located subsequent to the imaging element; 15. The imaging element according to claim 1, wherein the control circuit acquires the image data from the memory in accordance with the attribute information received by the receiver, and outputs the acquired image data to the subsequent circuit via the output interface.

16. the output interface includes a first output interface and a second output interface; the first output interface outputs the image data; The imaging device according to claim 1 , wherein the second output interface outputs the attribute information.

17. 17. The imaging device according to claim 1, wherein at least the photoelectric conversion element and the memory are integrated into a single chip.

18. 18. The imaging device according to claim 17, wherein the imaging device is a stacked type imaging device in which the memory is stacked on the photoelectric conversion element.

19. The imaging device according to any one of claims 1 to 18; a control device that performs at least one of control to display an image based on the image data output by the output interface on a display and control to store the image data output by the output interface in a storage device; and An imaging device comprising:

20. A method of operating an imaging device, comprising: controlling storage of image data in a memory that stores image data obtained by capturing an image at a first frame rate; storing attribute information of the image data in the memory; outputting the image data stored in the memory at a second frame rate different from the first frame rate; receiving an instruction regarding the attribute information; and and outputting the attribute information in accordance with the received instruction. How the image sensor works.

21. A computer applied to an imaging device, controlling storage of image data in a memory that stores image data obtained by capturing an image at a first frame rate; storing attribute information of the image data in the memory; outputting the image data stored in the memory at a second frame rate different from the first frame rate; receiving an instruction regarding the attribute information; and A program for executing a process including outputting the attribute information in accordance with the received instruction.

Images