Imaging element, imaging device, actuation method for imaging element, and program

The imaging device integrates a storage unit to manage and output image data with attribute information, addressing the need for selective data acquisition and efficient power use, enhancing data management and transfer speeds.

JP2024156957AActive Publication Date: 2024-11-06FUJIFILM CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024133234
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2019-07-26
Filing Date
2024-08-08
Publication Date
2024-11-06
Estimated Expiration
2040-07-01

AI Technical Summary

Technical Problem

Existing imaging devices lack the ability to selectively acquire necessary image data and efficiently manage and output attribute information associated with image data, leading to increased power consumption and limited flexibility in data usage.

Method used

An imaging device with a built-in storage unit that controls the storage and output of image data, including attribute information such as address, size, time, and erasure flags, allowing selective acquisition and management of image data based on external instructions.

Benefits of technology

Enables efficient power usage, selective data acquisition, and expanded usage options for image data by managing attribute information, while allowing for miniaturization and improved data transfer speeds through integrated chip design.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024156957000001_ABST
    Figure 2024156957000001_ABST
Patent Text Reader

Abstract

To selectively acquire required digital image data from an imaging element.SOLUTION: An imaging element includes: a storage part which stores image data acquired by imaging and is built in the imaging element; a control part which controls the storing of the image data into the storage part and also allows the storage part to store attribute information of the image data, and is built in the imaging element; an output part which outputs the image data stored in the storage part, and is built in the imaging element; and a reception part which receives directions associated with the attribute information. The control part acquires the image data from the storage part according to the received attribute information, and outputs the acquired image data to the output part.SELECTED DRAWING: Figure 8
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The technology disclosed herein relates to an imaging element, an imaging device, an operation method for an imaging element, and a program. [Background technology]

[0002] International Publication No. 2014 / 007004 discloses a solid-state imaging element that includes an AD converter that digitizes analog pixel signals read out to a signal line from each pixel in a pixel array section, and is equipped with a signal processing section that transfers the digitized pixel data at a first speed faster than a frame rate, a memory section that holds the pixel data transferred from the signal processing section, a data processing section that reads out the pixel data from the memory section at a second speed slower than the first speed, and a control section that controls to stop the operation of a current source connected to the signal line and the operation of at least the AD converter of the signal processing section when reading out pixel data from the memory section.

[0003] In the solid-state imaging device described in WO 2014 / 007004, the signal processing unit, memory unit, data processing unit, and control unit are formed in at least one chip different from the chip in which the pixel array unit is formed, and the chip in which the pixel array unit is formed and at least one other chip are stacked. Also, in the solid-state imaging device described in WO 2014 / 007004, the data processing unit has a decoder that specifies a column address for the memory unit and a sense amplifier that reads out pixel data of the specified address, and reads out the pixel data from the memory unit through the sense amplifier and the decoder.

[0004] International Publication No. 2013 / 145765 discloses an imaging unit comprising an imaging section including a first group including one or more pixels and a second group including one or more pixels different from the pixels constituting the first group, and a control section that, during a period in which the first group is caused to perform one charge accumulation, causes the second group to perform a different number of charge accumulations than the first group, and outputs each pixel signal.

[0005] In the imaging unit described in WO 2013 / 145765, an imaging chip including an imaging section and a signal processing chip including a processing circuit for processing pixel signals are electrically connected by a stacked structure. A memory chip including a pixel memory for storing pixel signals is also electrically connected by a stacked structure. In the imaging unit described in WO 2013 / 145765, the control unit reads out pixel signals of a designated group from the pixel memory and transfers them to an image processing unit in accordance with a transfer request from an external circuit for a designated group among groups including the second group. In addition, the imaging unit described in WO 2013 / 145765 includes a data transfer interface that transmits pixel signals in accordance with the transfer request. The data transfer interface employs at least one of a double data rate method, an addressing method, a burst transfer method, a bus method, and a serial method for transmitting pixel signals. In addition, in the imaging unit described in WO 2013 / 145765, when each pixel signal for a plurality of charge accumulations is stored in the pixel memory, the control unit transfers each pixel signal to an image processing unit. Summary of the Invention

[0006] One embodiment of the technique disclosed herein provides an image sensor, an imaging device, an operation method for the image sensor, and a program that can selectively acquire required image data from the image sensor. [Means for solving the problem]

[0007] A first aspect of the technology of the present disclosure is an imaging element that includes: a memory unit that stores image data obtained by imaging and is built into the imaging element; a control unit that controls the storage of image data in the memory unit and stores attribute information of the image data in the memory unit and is built into the imaging element; an output unit that outputs image data stored in the memory unit and is built into the imaging element; and a reception unit that receives instructions regarding the attribute information, wherein the output unit is an imaging element that outputs the attribute information in accordance with the instructions received by the reception unit.

[0008] A second aspect according to the technique of the present disclosure is the imaging element according to the first aspect, in which the output unit outputs the attribute information at a timing when the instruction is accepted by the acceptance unit.

[0009] A third aspect according to the technique of the present disclosure is the image sensor according to the first or second aspect, in which the instruction is an external frame synchronization signal.

[0010] A fourth aspect of the technology of the present disclosure is an imaging element according to any one of the first to third aspects, wherein the output unit outputs attribute information relating to the latest image data among the image data stored in the memory unit.

[0011] A fifth aspect according to the technique of the present disclosure is the imaging element according to the fourth aspect, in which the output unit outputs attribute information related to the latest image data at the timing when the instruction is accepted by the acceptance unit.

[0012] A sixth aspect of the technology of the present disclosure is an imaging element relating to any one of the first to fifth aspects, wherein the output unit is capable of outputting attribute information for each of a plurality of image data, and the attribute information is output by the output unit in the order of imaging.

[0013] A seventh aspect of the technology of the present disclosure is an imaging element according to any one of the first to sixth aspects, wherein the attribute information is information including at least one of an address, an image size, an imaging time, and imaging conditions.

[0014] An eighth aspect of the technology of the present disclosure is an imaging element according to any one of the first to seventh aspects, in which, when image data is erased from the memory unit upon the next imaging, the attribute information output from the output unit is information including erasure information indicating that the image data will be erased from the memory unit upon the next imaging.

[0015] A ninth aspect of the technology of the present disclosure is an imaging element according to the eighth aspect, in which, when image data is erased from the memory unit, the control unit erases attribute information relating to the image data to be erased, which is the image data in the memory unit that has been erased, from the memory unit, and erases the image data to be erased from the memory unit.

[0016] A tenth aspect of the technology of the present disclosure is an imaging element relating to any one of the first to ninth aspects, which includes a compression circuit that compresses image data, and a control unit that stores image data for a predetermined number of frames in a memory unit, stores compressed image data obtained by compressing the image data by the compression circuit in the memory unit, and associates attribute information regarding the compressed image data and compression identification information that can identify that the image data has been compressed with the compressed image data.

[0017] An eleventh aspect of the technology of the present disclosure is an imaging element according to the tenth aspect, in which, when compressed image data is stored in the memory unit, the attribute information output from the output unit is information including compression plan information indicating that the compressed image data is scheduled to be stored in the memory unit.

[0018] A twelfth aspect of the technology of the present disclosure is an imaging element according to the tenth or eleventh aspect, wherein the attribute information output from the output unit is information including information capable of identifying the compression format of the compressed image data.

[0019] A thirteenth aspect of the technology of the present disclosure is an imaging element relating to any one of the first to twelfth aspects, in which the instructions include an output amount of attribute information by the output unit, and the output unit outputs the attribute information at the output amount.

[0020] A fourteenth aspect according to the technique of the present disclosure is an image sensor according to the thirteenth aspect, in which the output amount is defined by the number of frames of image data.

[0021] A fifteenth aspect of the technology disclosed herein is an imaging element relating to any one of the first to fourteenth aspects, further including a receiving unit that receives attribute information transmitted from a downstream circuit located downstream of the imaging element, and a control unit that acquires image data from the memory unit in accordance with the attribute information received by the receiving unit, and causes an output unit to output the acquired image data to the downstream circuit.

[0022] A 16th aspect of the technology of the present disclosure is an imaging element relating to any one of the first to fifteenth aspects, wherein the output section has a first output section and a second output section, the first output section outputs image data, and the second output section outputs attribute information.

[0023] A seventeenth aspect according to the technique of the present disclosure is an imaging element according to any one of the first to sixteenth aspects, in which at least a photoelectric conversion element and a storage unit are integrated into a single chip.

[0024] An 18th aspect according to the technique of the present disclosure is the imaging element according to the 17th aspect, in which the imaging element is a stacked type imaging element in which a storage unit is stacked on a photoelectric conversion element.

[0025] A 19th aspect of the technology of the present disclosure is an imaging device including an imaging element relating to any one of the first to eighteenth aspects, and a control device that performs at least one of control to cause a display unit to display an image based on image data output by the output unit, and control to store the image data output by the output unit in a storage device.

[0026] A twentieth aspect of the technology of the present disclosure is a method for operating an imaging element having a built-in memory unit that stores image data obtained by imaging, the method including controlling the storage of image data in the memory unit, storing attribute information of the image data in the memory unit, outputting the image data stored in the memory unit, accepting instructions related to the attribute information, and outputting the attribute information in accordance with the accepted instructions.

[0027] A 21st aspect of the technology of the present disclosure is a program for causing a computer applied to an imaging element having a built-in memory unit for storing image data obtained by imaging to execute processes including controlling the storage of image data in the memory unit, storing attribute information of the image data in the memory unit, outputting the image data stored in the memory unit, accepting instructions related to the attribute information, and outputting the attribute information in accordance with the accepted instructions. [Brief description of the drawings]

[0028] [Figure 1] 1 is a perspective view illustrating an example of the appearance of an imaging device according to an embodiment. [Diagram 2] 2 is a rear view showing an example of the appearance of the rear side of the imaging device shown in FIG. [Diagram 3] 1 is a block diagram showing an example of a configuration of an imaging device according to an embodiment. [Figure 4] 2 is a block diagram showing an example of a hardware configuration of an electrical system of an imaging device main body included in the imaging device according to the embodiment. FIG. [Diagram 5] 4 is a conceptual diagram illustrating a frame rate of an imaging element included in the imaging device according to the embodiment. FIG. [Figure 6] 1 is a conceptual diagram showing an example of a layered structure of an imaging element according to an embodiment. [Figure 7] 2 is a block diagram showing an example of a structure of a memory included in an image sensor according to an embodiment. FIG. [Figure 8] FIG. 1 is a block diagram showing an example of the electrical hardware configuration of an imaging element included in an imaging device according to an embodiment, and is a block diagram used to explain an example form in which imaging is performed by the imaging element and attribute information is output from the imaging element. [Figure 9] 4A and 4B are conceptual diagrams showing an example of the contents of an imaging process and an output process performed by the imaging element according to the embodiment. [Figure 10] 5 is a time chart showing an example of processing content in an imaging element included in the imaging device according to the embodiment. [Figure 11A]FIG. 13 is a conceptual diagram showing an example of the contents of attribute information output from an output I / F of an imaging element for the first to fourth frames. [Figure 11B] 13 is a conceptual diagram showing an example of the contents of attribute information output from an output I / F of the imaging element in the fifth and sixth frames. FIG. [Figure 12] FIG. 2 is a block diagram showing an example of the hardware configuration of an electrical system of an image sensor included in an imaging device according to an embodiment, and is a block diagram provided for explaining an example of a form in which digital image data is output from the image sensor. [Figure 13] 10 is a flowchart showing an example of the flow of attribute information output processing according to the embodiment. [Figure 14] 10 is a flowchart showing an example of the flow of a data erasure process according to the embodiment. [Figure 15] 10 is a time chart showing an example of processing contents in which digital image data and compressed image data are stored in a memory when compression processing is performed on digital image data. [Figure 16A] FIG. 13 is a conceptual diagram showing an example of the contents of attribute information output from an output I / F of an image sensor for the first to fourth frames when compression processing is performed on digital image data for the fourth frame and onward. [Figure 16B] 13 is a conceptual diagram showing an example of the contents of attribute information output from the output I / F of the image sensor in the fifth and sixth frames when compression processing is performed on digital image data from the fourth frame onwards. FIG. [Figure 17A] This is a conceptual diagram showing an example of the content of attribute information that includes a compression plan flag indicating whether one frame's worth of digital image data is scheduled to be compressed with the next imaging, and is output from the output I / F of the imaging element for the first to fourth frames. [Figure 17B] This is a conceptual diagram showing an example of the content of attribute information that includes a compression plan flag indicating whether one frame of digital image data is scheduled to be compressed with the next image capture, and is output from the output I / F of the image capture element for the fifth and sixth frames. [Figure 18A]This is a conceptual diagram showing an example of the content of attribute information that includes a compression plan flag indicating whether or not one frame of digital image data is scheduled to be compressed with the next image capture, and also includes compression format specification information, and is output from the output I / F of the image capture element for the fourth and fifth frames. [Figure 18B] This is a conceptual diagram showing an example of the content of attribute information that includes a compression plan flag indicating whether or not one frame of digital image data is scheduled to be compressed with the next image capture, and also includes compression format specification information, and is output from the output I / F of the image capture element in the sixth frame. [Figure 19] A time chart showing an example of the processing content in an imaging element when attribute information is output by an output I / F of the imaging element at the timing when an instruction regarding attribute information (e.g., a vertical synchronization signal) is received by the reception I / F of the imaging element. [Figure 20] FIG. 13 is a conceptual diagram showing an example of the contents of attribute information output for the first to sixth frames in a case where the attribute information output for each frame is attribute information related to digital image data for one frame. [Figure 21] FIG. 13 is a conceptual diagram showing an example of the contents of attribute information that is output for the first to sixth frames when the attribute information output for each frame is attribute information related to one frame's worth of digital image data, and does not include an erasure flag or a compression planned flag. [Figure 22] 13 is a flowchart showing a modified example of the flow of the attribute information output process according to the embodiment. [Diagram 23] 13 is a block diagram showing a modified example of the hardware configuration of the electrical system of the image sensor included in the image pickup apparatus according to the embodiment. FIG. [Figure 24] FIG. 2 is a rear perspective view showing an example of the appearance of the rear side of the smart device according to the embodiment. [Diagram 25] FIG. 25 is a front perspective view showing an example of the appearance of the front side of the smart device shown in FIG. 24. [Figure 26]11 is a conceptual diagram showing an example of a manner in which an imaging element-side program according to an embodiment is installed from a storage medium storing the imaging element-side program to a computer in the imaging element; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Hereinafter, an example of an embodiment of an imaging device according to the technique of the present disclosure will be described with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". RAM is an abbreviation for "Random Access Memory". ROM is an abbreviation for "Read Only Memory". DRAM is an abbreviation for "Dynamic Random Access Memory". SRAM is an abbreviation for "Static Random Access Memory". LSI is an abbreviation for "Large-Scale Integrated circuit". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EEPROM is an abbreviation of "Electrically Erasable and Programmable Read Only Memory". CCD is an abbreviation of "Charge Coupled Device". CMOS is an abbreviation of "Complementary Metal Oxide Semiconductor". EL stands for "Electro-Luminescence". A / D stands for "Analog / Digital". I / F stands for "Interface". UI stands for "User Interface". LVDS stands for "Low Voltage Differential Signaling". PCI-e stands for "Peripheral Component Interconnect Express". SATA stands for "Serial Advanced Technology". "Standard" is an abbreviation for "Scalable Low Signaling with Embedded Clock". "SLVS-EC" is an abbreviation for "Scalable Low Signaling with Embedded Clock". "MIPI" is an abbreviation for "Mobile Industry Processor Interface". "fps" is an abbreviation for "frame per second". "FIFO" is an abbreviation for "First In, First Out". "MPEG" is an abbreviation for "Moving Picture Experts Group". "JPEG" is an abbreviation for "Joint Photographic Experts Group". "TIFF" is an abbreviation for "Tagged Image File Format". "BMP" is an abbreviation for "Bitmap". "PNG" is an abbreviation for "Portable Network Graphics". "GIF" is an abbreviation for "Graphics Interchange Format".

[0032] 1, as an example, the imaging device 10 is a digital camera with interchangeable lenses and no reflex mirror. The imaging device 10 includes an imaging device body 12 and an interchangeable lens 14 that is replaceably attached to the imaging device body 12. Note that, although a digital camera with interchangeable lenses and no reflex mirror is given here as an example of the imaging device 10, the technology of the present disclosure is not limited thereto, and the imaging device 10 may be other types of digital cameras, such as a fixed lens type.

[0033] The imaging device body 12 is provided with an imaging element 38. When the interchangeable lens 14 is attached to the imaging device body 12, subject light indicative of a subject passes through the interchangeable lens 14 and is focused on the imaging element 38, and image data indicative of an image of the subject (for example, see FIGS. 4 and 5) is generated by the imaging element 38.

[0034] The imaging device body 12 is provided with a hybrid finder (registered trademark) 16. The hybrid finder 16 here refers to a finder in which, for example, an optical viewfinder (hereinafter referred to as "OVF") and an electronic viewfinder (hereinafter referred to as "EVF") are selectively used. Note that OVF is an abbreviation for "optical viewfinder." Also, EVF is an abbreviation for "electronic viewfinder."

[0035] A viewfinder switching lever 18 is provided on the front of the imaging device body 12. An optical image visible in the OVF and a live view image, which is an electronic image visible in the EVF, are switched by rotating the viewfinder switching lever 18 in the direction of the arrow SW. The "live view image" referred to here refers to a moving image for display based on image data obtained by capturing an image using the imaging element 38. A live view image is also generally called a through image. A release button 20 and a dial 23 are provided on the top surface of the imaging device body 12. The dial 23 is operated when setting an operation mode of the imaging system and an operation mode of the playback system, and thereby, in the imaging device 10, an imaging mode and a playback mode are selectively set as the operation mode.

[0036] The release button 20 functions as an imaging preparation instruction section and an imaging instruction section, and is capable of detecting two stages of pressing operation: an imaging preparation instruction state and an imaging instruction state. The imaging preparation instruction state refers to a state in which the button is pressed from the standby position to an intermediate position (half-pressed position), for example, and the imaging instruction state refers to a state in which the button is pressed beyond the intermediate position to a final pressed position (fully-pressed position). In the following, the "state in which the button is pressed from the standby position to the half-pressed position" is referred to as the "half-pressed state", and the "state in which the button is pressed from the standby position to the fully-pressed position" is referred to as the "fully-pressed state".

[0037] As an example, as shown in FIG. 2, a touch panel display 24, instruction keys 27, and a viewfinder eyepiece 30 are provided on the rear surface of the imaging device body 12.

[0038] The touch panel display 24 includes a display 26 and a touch panel 28 (see also FIG. 4). An example of the display 26 is a liquid crystal display. The display 26 may be other types of displays, such as an organic EL display or an inorganic EL display, instead of a liquid crystal display. The display 26 and the EVF are examples of a "display unit (display)" according to the technology of the present disclosure. The display of the EVF is the same as the display of the display 26, so further description will be omitted, but in this specification, the display on the display 26 can be read as the display on the EVF.

[0039] The display 26 displays images, text information, etc. The display 26 is used to display live view images obtained by continuous image capture when the imaging device 10 is in imaging mode. The display 26 is also used to display still images obtained by capturing images when an instruction to capture still images is given. Furthermore, the display 26 is also used to display playback images and menu screens, etc. when the imaging device 10 is in playback mode.

[0040] The touch panel 28 is a transmissive touch panel, and is overlaid on the surface of the display area of ​​the display 26. The touch panel 28 receives instructions from a user by detecting contact with a pointing object such as a finger or a stylus pen.

[0041] The instruction keys 27 accept various instructions. The "various instructions" referred to here refer to, for example, an instruction to display a menu screen from which various menus can be selected, an instruction to select one or more menus, an instruction to confirm the selected content, an instruction to erase the selected content, an instruction to zoom in, zoom out, frame-by-frame advance, and various other instructions.

[0042] As an example, as shown in FIG. 3, the interchangeable lens 14 has an imaging lens 40. The imaging lens 40 includes an objective lens 40A, a focus lens 40B, and an aperture 40C. The objective lens 40A, the focus lens 40B, and the aperture 40C are arranged in this order along the optical axis L1 from the subject side (object side) to the imaging device body 12 side (image side). The focus lens 40B and the aperture 40C are operated by receiving power from a driving source (not shown) such as a motor. That is, the focus lens 40B and the aperture 40C move along the optical axis L1 according to the applied power. The aperture 40C adjusts the exposure by operating according to the applied power.

[0043] The imaging device main body 12 includes a subsequent circuit 13, a UI device 17, a mechanical shutter 41, and an imaging element 38. The subsequent circuit 13 is a circuit located subsequent to the imaging element 38. The subsequent circuit 13 includes a controller 15 and a signal processing circuit 34. The controller 15 is connected to the UI device 17, the signal processing circuit 34, and the imaging element 38, and controls the entire electrical system of the imaging device 10.

[0044] The imaging element 38 includes a photoelectric conversion element 42 having a light receiving surface 42A. In this embodiment, the imaging element 38 is a CMOS image sensor. Although a CMOS image sensor is exemplified as the imaging element 38 here, the technology of the present disclosure is not limited thereto, and the technology of the present disclosure is also applicable even if the imaging element 38 is another type of image sensor such as a CCD image sensor.

[0045] The mechanical shutter 41 operates by receiving power from a drive source (not shown) such as a motor. When the interchangeable lens 14 is attached to the imaging device body 12, subject light indicative of a subject passes through the imaging lens 40 and is imaged on the light receiving surface 42A via the mechanical shutter 41.

[0046] The UI-based device 17 is a device that presents information to a user and receives instructions from the user. The controller 15 acquires various information from the UI-based device 17 and controls the UI-based device 17.

[0047] The imaging element 38 is connected to the controller 15, and under the control of the controller 15, captures an image of a subject, thereby generating image data representing the image of the subject.

[0048] The imaging element 38 is connected to the signal processing circuit 34. The signal processing circuit 34 is an LSI, specifically, a device including an ASIC and an FPGA. The controller 15 acquires various information from the signal processing circuit 34 and controls the imaging element 38. Under the control of the controller 15, the imaging element 38 outputs image data generated by the photoelectric conversion element 42 to the signal processing circuit 34.

[0049] The signal processing circuit 34 performs various signal processing on the image data input from the imaging element 38. The various signal processing performed by the signal processing circuit 34 includes known signal processing such as white balance adjustment, sharpness adjustment, gamma correction, color space conversion processing, and color difference correction.

[0050] The various signal processes performed by the signal processing circuit 34 may be distributed between the signal processing circuit 34 and the image sensor 38. In other words, at least a part of the various signal processes performed by the signal processing circuit 34 may be assigned to the processing circuit 110 of the image sensor 38.

[0051] In this embodiment, a device including an ASIC and an FPGA is exemplified as the signal processing circuit 34, but the technology of the present disclosure is not limited to this, and the signal processing circuit 34 may be a device including an ASIC, an FPGA, and / or a PLD.

[0052] The signal processing circuit 34 may be a computer including a CPU, storage, and memory. The term "storage" as used herein refers to a non-volatile storage device such as an SSD or HDD, and the term "memory" as used herein refers to a volatile storage device such as a DRAM or SRAM. The computer may include a single CPU or multiple CPUs. A GPU may be used instead of a CPU. The signal processing circuit 34 may be realized by a combination of a hardware configuration and a software configuration.

[0053] As an example, as shown in Fig. 4, the controller 15 includes a CPU 15A, a storage 15B, a memory 15C, an output I / F 15D, and a reception I / F 15E. The CPU 15A, the storage 15B, the memory 15C, the output I / F 15D, and the reception I / F 15E are connected via a bus line 100. In the example shown in Fig. 4, for convenience of illustration, one bus line is shown as the bus line 100, but the bus line 100 includes a data bus, an address bus, a control bus, and the like.

[0054] The storage 15B stores various parameters and various programs. The storage 15B is a non-volatile storage device. Here, an EEPROM is used as an example of the storage 15B, but the storage 15B is not limited to this and may be a mask ROM, HDD, SSD, or the like. The memory 15C is a volatile storage device. Various information is temporarily stored in the memory 15C. The memory 15C is used as a work memory by the CPU 15A. Here, a DRAM is used as an example of the memory 15C, but the storage 15B is not limited to this and may be another type of volatile storage device such as an SRAM. The CPU 15A is an example of a "control device" according to the technology of the present disclosure, and the storage 15B is a non-volatile storage device according to the technology of the present disclosure. This is an example of a "memory device."

[0055] Various programs are stored in the storage 15B. The CPU 15A reads out the various programs from the storage 15B and loads the read out various programs in the memory 15C. The CPU 15A controls the entire imaging device 10 in accordance with the various programs loaded in the memory 15C.

[0056] The output I / F 15D is connected to the imaging element 38. The CPU 15A controls the imaging element 38 via the output I / F 15D. For example, the CPU 15A controls the timing of imaging performed by the imaging element 38 by supplying an imaging timing signal that specifies the timing of imaging to the imaging element 38 via the output I / F 15D.

[0057] The reception I / F 15E is connected to the signal processing circuit 34. The CPU 15A transmits and receives various types of information to and from the signal processing circuit 34 via the reception I / F 15E.

[0058] Image data is input to the signal processing circuit 34 from the imaging element 38. The signal processing circuit 34 performs various signal processing (described in detail later) on the image data input from the imaging element 38. The signal processing circuit 34 outputs the image data that has been subjected to various signal processing to the reception I / F 15E. The reception I / F 15E receives the image data from the signal processing circuit 34 and transfers the received image data to the CPU 15A.

[0059] An external I / F 104 is connected to the bus line 100. The external I / F 104 is a communication device configured with a circuit. Note that, although a device configured with a circuit is adopted as the external I / F 104 here, this is merely one example. The external I / F 104 may be a device including an ASIC, an FPGA, and / or a PLD. Furthermore, the external I / F 104 may be realized by a combination of a hardware configuration and a software configuration.

[0060] An example of the external I / F 104 is a USB interface, to which external devices (not shown) such as a memory card controller, a smart device, a personal computer, a server, a USB memory, and / or a memory card can be connected. The external I / F 104 controls the transmission and reception of various information between the CPU 15A and the external devices. Note that the external devices directly or indirectly connected to the external I / F 104, i.e., the smart device, a personal computer, a server, a USB memory, and / or a memory card, are examples of "storage devices" according to the technology of the present disclosure.

[0061] The UI device 17 includes a touch panel display 24 and a reception device 84. The display 26 and the touch panel 28 are connected to a bus line 100. Therefore, the CPU 15A causes the display 26 to display various information and operates according to various instructions received by the touch panel 28.

[0062] The accepting device 84 includes a hard key unit 25. The hard key unit 25 is a plurality of hard keys, and includes a release button 20 (see FIG. 1), a dial 23 (see FIGS. 1 and 2), and instruction keys 27 (see FIG. 2). The hard key unit 25 is connected to the bus line 100, and the CPU 15A acquires instructions accepted by the hard key unit 25 and operates according to the acquired instructions.

[0063] As an example, as shown in FIG. 5, an imaging timing signal is input from the controller 15 to the imaging element 38. The imaging timing signal includes a vertical synchronization signal and a horizontal synchronization signal. The vertical synchronization signal is a signal that is used to synchronize the reading of image data from the photoelectric conversion element 42 for each frame. The vertical synchronization signal is a synchronization signal that specifies the start timing of reading out image data for each horizontal line from the photoelectric conversion element 42. The image sensor 38 reads out image data from the photoelectric conversion element 42 in accordance with a frame rate determined according to the vertical synchronization signal input from the controller 15. Note that the vertical synchronization signal is an example of an "instruction related to attribute information" and an "external frame synchronization signal" according to the technology of the present disclosure, and the controller 15 is an example of the "external" according to the technology of the present disclosure.

[0064] 5, the frame rate of the imaging element 38 is set to a frame rate at which eight frames are read out from the photoelectric conversion element 42 within a period T. A specific example of the frame rate is 120 fps, but is not limited thereto and may be a frame rate exceeding 120 fps (e.g., 240 fps) or less than 120 fps (e.g., 60 fps).

[0065] 6, the image sensor 38 includes a photoelectric conversion element 42, a processing circuit 110, and a memory 112. The image sensor 38 is an image sensor in which the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single chip. That is, the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single package. In the image sensor 38, the processing circuit 110 and the memory 112 are stacked on the photoelectric conversion element 42. Specifically, the photoelectric conversion element 42 and the processing circuit 110 are electrically connected to each other by conductive bumps (not shown) such as copper, and the processing circuit 110 and the memory 112 are electrically connected to each other by conductive bumps (not shown) such as copper.

[0066] The processing circuit 110 is, for example, an LSI. The memory 112 is a memory in which the write timing and the read timing are different. Here, a DRAM is used as an example of the memory 112.

[0067] The processing circuit 110 is a device including an ASIC and an FPGA, and controls the entire image sensor 38 according to instructions from the controller 15. Here, an example is given in which the processing circuit 110 is realized by a device including an ASIC and an FPGA, but the technology of the present disclosure is not limited to this, and may be, for example, a device including an ASIC, an FPGA, and / or a PLD. In addition, a computer including a CPU, a storage such as an EEPROM that is a non-volatile storage device, and a memory such as a RAM that is a volatile storage device may be adopted as the processing circuit 110. The computer may include a single CPU or multiple CPUs. A GPU may be used instead of the CPU. In addition, the processing circuit 110 may be realized by a combination of a hardware configuration and a software configuration.

[0068] The photoelectric conversion element 42 has a plurality of photodiodes arranged in a matrix. One example of the plurality of photodiodes is photodiodes for "4896×3265" pixels.

[0069] A color filter is disposed on each photodiode included in the photoelectric conversion element 42. The color filters include a G filter corresponding to G (green), which contributes most to obtaining a luminance signal, an R filter corresponding to R (red), and a B filter corresponding to B (blue).

[0070] The photoelectric conversion element 42 has R pixels, G pixels, and B pixels. The R pixels correspond to photodiodes in which an R filter is arranged, the G pixels correspond to photodiodes in which a G filter is arranged, and the B pixels correspond to photodiodes in which a B filter is arranged. The R pixels, G pixels, and B pixels are arranged with a predetermined periodicity in the row direction (horizontal direction) and column direction (vertical direction). In this embodiment, the R pixels, The G pixels and the B pixels are arranged with a periodicity corresponding to the X-Trans (registered trademark) array. Note that although the X-Trans array is illustrated here as an example, the technology of the present disclosure is not limited to this, and the array of the R pixels, G pixels, and B pixels may be a Bayer array, a honeycomb array, or the like.

[0071] The imaging element 38 has a so-called electronic shutter function, and by operating the electronic shutter function under the control of the controller 15, the charge accumulation time of each photodiode in the photoelectric conversion element 42 is controlled. The charge accumulation time refers to the so-called shutter speed.

[0072] The image sensor 38 selectively captures still images and live view images using a rolling shutter system. Still image capture is achieved by activating the electronic shutter function and operating the mechanical shutter (not shown), while live view image capture is achieved by activating the electronic shutter function without operating the mechanical shutter. Note that although the rolling shutter system is exemplified here, the technology of the present disclosure is not limited to this, and a global shutter system may be applied instead of the rolling shutter system.

[0073] The memory 112 is an example of a "storage unit (memory)" according to the technology of the present disclosure. In this embodiment, a DRAM is used as the memory 112, but the technology of the present disclosure is valid even if the memory 112 is another type of memory. Also, the imaging element 38 is an example of a "stacked type imaging element" according to the technology of the present disclosure.

[0074] 7, the memory 112 has a first storage area 112A, a second storage area 112B, a third storage area 112C, a fourth storage area 112D, a fifth storage area 112E, and a sixth storage area 112F. Note that, for convenience of explanation, six storage areas are illustrated here, but this is merely an example, and the memory 112 may have multiple storage areas capable of storing multiple frames of image data.

[0075] In the example shown in FIG. 7, addresses in the memory 112 are assigned to each of the first storage area 112A, the second storage area 112B, the third storage area 112C, the fourth storage area 112D, the fifth storage area 112E, and the sixth storage area 112F. The address "0X00000000" is assigned to the first storage area 112A. The address "0X00100000" is assigned to the second storage area 112B. The address "0X00200000" is assigned to the third storage area 112C. The address "0X00300000" is assigned to the fourth storage area 112D. The address "0X00400000" is assigned to the fifth storage area 112E. The sixth memory area 112F is assigned an address of "0X00500000".

[0076] 8, the processing circuit 110 includes a reception I / F 110D1 and an output I / F 110D2. Here, the reception I / F 110D1 is an example of a "reception unit" and a "reception unit" according to the technology of the present disclosure, and the output I / F 110D2 is an example of an "output unit (output interface)" according to the technology of the present disclosure. Note that the reception unit means, for example, an acceptor, and the reception unit means, for example, a receiver.

[0077] The output I / F 15D of the controller 15 is connected to the reception I / F 110D1 of the processing circuit 110, and outputs an imaging timing signal to the reception I / F 110D1. The reception I / F 110D1 receives the imaging timing signal output from the output I / F 15D.

[0078] The signal processing circuit 34 includes a reception I / F 34A and an output I / F 34B. The output I / F 34A is connected to the output I / F 110D2 of the image sensor 38. The output I / F 110D2 of the processing circuit 110 outputs various information such as image data (hereinafter, also simply referred to as "various information") to the reception I / F 34A of the signal processing circuit 34, and the reception I / F 34A receives the various information output from the output I / F 110D2. The signal processing circuit 34 performs signal processing on the various information received by the reception I / F 34A as necessary. The output I / F 34B is connected to the reception I / F 15E of the controller 15, and outputs the various information to the reception I / F 15E of the controller 15. The reception I / F 15E receives the various information output from the output I / F 34B.

[0079] In the image sensor 38, the processing circuit 110 includes a reading circuit 110A, a digital processing circuit 110B, a control circuit 110C, and an image processing circuit 110E in addition to a reception I / F 110D1 and an output I / F 110D2. The control circuit 110C is an example of a "control unit (control circuit)" according to the technology of the present disclosure.

[0080] The readout circuit 110A is connected to each of the photoelectric conversion element 42, the digital processing circuit 110B, and the control circuit 110C. The digital processing circuit 110B is connected to the control circuit 110C. The control circuit 110C is connected to each of the memory 112, the reception I / F 110D1, the output I / F 110D2, and the image processing circuit 110E.

[0081] The image data described above is roughly divided into analog image data 70A and digital image data 70B, as shown in Fig. 8. For convenience of explanation, when there is no need to distinguish between the analog image data 70A and the digital image data 70B, they will be referred to as "image data" without being given a reference symbol.

[0082] Each of the reception I / F 110D1 and the output I / F 110D2 of the processing circuit 110 is a communication device having an FPGA. In addition, each of the output I / F 15D and the reception I / F 15E of the controller 15 is also a communication device having an FPGA. Furthermore, each of the reception I / F 34A and the output I / F 34B of the signal processing circuit 34 is also a communication device having an FPGA.

[0083] The reception I / F 110D1 of the processing circuit 110 and the output I / F 15D of the controller 15 are connected in accordance with the PCI-e connection standard. The output I / F 110D2 of the processing circuit 110 and the reception I / F 34A of the signal processing circuit 34 are also connected in accordance with the PCI-e connection standard. The output I / F 34B of the signal processing circuit 34 and the reception I / F 15E of the controller 15 are also connected in accordance with the PCI-e connection standard. In the following, when it is not necessary to distinguish between the reception I / F 110D1, the output I / F 110D2, the reception I / F 34A, the output I / F 34B, the reception I / F 15E, and the output I / F 15D, they will be referred to as "communication I / F" without reference numerals.

[0084] Here, a communication device composed of circuits (ASIC, FPGA, and / or PLD, etc.) is used as the communication I / F, but this is merely one example. The communication I / F may be a computer including a CPU, storage such as EEPROM, and memory such as RAM. In this case, the computer may include a single CPU or multiple CPUs. A GPU may be used instead of a CPU. The communication I / F may also be realized by a combination of hardware and software configurations.

[0085] The reception I / F 110D1 receives an imaging timing signal output from the output I / F 15D of the controller 15, and transfers the received imaging timing signal to the control circuit 110C.

[0086] The readout circuit 110A controls the photoelectric conversion element 42 under the control of the control circuit 110C. The analog image data 70A is read out from the conversion element 42. The analog image data 70A is read out from the photoelectric conversion element 42 in accordance with an imaging timing signal input from the controller 15 to the processing circuit 110.

[0087] Specifically, first, the reception I / F 110D1 receives an imaging timing signal from the controller 15 and transfers the received imaging timing signal to the control circuit 110C. Next, the control circuit 110C transfers the imaging timing signal transferred from the reception I / F 110D1 to the readout circuit 110A. That is, a vertical synchronization signal and a horizontal synchronization signal are transferred to the readout circuit 110A. Then, the readout circuit 110A starts reading out the analog image data 70A in units of frames from the photoelectric conversion element 42 in accordance with the vertical synchronization signal transferred from the control circuit 110C. Also, the readout circuit 110A starts reading out the analog image data 70A in units of horizontal lines in accordance with the horizontal synchronization signal transferred from the control circuit 110C.

[0088] The readout circuit 110A performs analog signal processing on the analog image data 70A read out from the photoelectric conversion element 42. The analog signal processing includes known processes such as noise cancellation processing and analog gain processing. The noise cancellation processing is a process for canceling noise caused by variations in characteristics between pixels included in the photoelectric conversion element 42. The analog gain processing is a process for applying a gain to the analog image data 70A. The readout circuit 110A also performs correlated double sampling on the analog image data 70A. After the readout circuit 110A performs correlated double sampling on the analog image data 70A, the analog image data 70A is output to the digital processing circuit 110B.

[0089] The digital processing circuit 110B includes an A / D converter 110B1. The A / D converter 110B1 performs A / D conversion on the analog image data 70A.

[0090] The digital processing circuit 110B performs digital signal processing on the analog image data 70A input from the readout circuit 110A. The digital signal processing includes, for example, correlated double sampling, A / D conversion by an A / D converter 110B1, and digital gain processing.

[0091] The analog image data 70A input from the readout circuit 110A is A / D converted by the A / D converter 110B1, whereby the analog image data 70A is digitized to obtain digital image data 70B as RAW data. The digital image data 70B is then subjected to digital gain processing by the digital processing circuit 110B. The digital gain processing refers to processing that applies gain to the digital image data 70B. The digital image data 70B obtained by performing this digital signal processing is output to the control circuit 110C by the digital processing circuit 110B.

[0092] The control circuit 110C outputs the digital image data 70B input from the digital processing circuit 110B to the image processing circuit 110E. The image processing circuit 110E performs image processing on the digital image data 70B input from the control circuit 110C, and outputs the image-processed digital image data 70B to the control circuit 110C. The "image processing" referred to here includes, for example, demosaic processing and / or digital thinning processing.

[0093] Demosaic processing is a process of calculating all color information for each pixel from a mosaic image corresponding to the arrangement of color filters. For example, if the imaging element 38 is an imaging element to which three color filters of RGB are applied, all RGB color information is calculated for each pixel from the RGB mosaic image. Digital thinning processing is a process of thinning out pixels included in image data on a line-by-line basis. The line unit refers to, for example, a horizontal line unit and / or a vertical line unit.

[0094] The memory 112 is a memory capable of storing digital image data of a plurality of frames. The memory 112 has a storage area in pixel units (see FIG. 7), and the digital image data 70B is stored in a corresponding storage area of ​​the memory 112 in pixel units by the control circuit 110C. The control circuit 110C stores the digital image data 70B input from the image processing circuit 110E in the memory 112.

[0095] The control circuit 110C is capable of randomly accessing the memory 112. The control circuit 110C acquires attribute information of the digital image data 70B stored in the memory 112 in response to an instruction from the controller 15. The attribute information is information indicating the attributes of the digital image data 70B stored in the memory 112. Here, the attribute information refers to information including, for example, an address, an image size, an imaging time, an exposure time, an image sensor sensitivity, and an erase flag.

[0096] The address refers to information (recording address) that specifies the storage location of the digital image data 70B in frame units in the memory 112. The image size refers to the size of one frame's worth of digital image data 70B in the horizontal line direction (horizontal size) and the size of one frame's worth of digital image data 70B in the vertical line direction (vertical size). The imaging time refers to the time (e.g., year, month, day, hour, minute, and second) when imaging is performed by the imaging element 38. The "time when imaging is performed" referred to here refers to, for example, the time when one frame's worth of digital image data 70B is stored in the memory 112. However, this is merely an example, and the time when imaging is performed may be the time when exposure for one frame is completed, or the time when A / D conversion for one frame is completed, etc.

[0097] The exposure time refers to the exposure time required to obtain one frame of digital image data 70B. The image sensor sensitivity refers to, for example, the sensitivity (sensor gain) of the photoelectric conversion element 42. The deletion flag refers to a flag indicating whether or not the digital image data 70B is deleted from the memory 112 when the next image is captured. When the deletion flag is off, the digital image data 70B is not deleted from the memory 112 when the next image is captured, and when the deletion flag is on, the digital image data 70B is deleted from the memory 112 when the next image is captured. An example of the digital image data 70B deleted from the memory 112 is one frame of digital image data 70B deleted from the memory 112.

[0098] The exposure time and the image sensor sensitivity are examples of "imaging conditions" of the technology of the present disclosure, and the erasure flag in the ON state is an example of "erasure information" according to the technology of the present disclosure.

[0099] Here, the exposure time and the image sensor sensitivity are given as examples of "imaging conditions" according to the technology of the present disclosure, but the technology of the present disclosure is not limited to this, and instead of or in addition to the exposure time and / or the image sensor sensitivity, the type of imaging lens 40, the subject distance, the focal length, the angle of view, and / or the presence or absence of shake correction, etc. may be used.

[0100] The control circuit 110C generates attribute information in response to an instruction regarding the attribute information from the controller 15, and outputs the generated attribute information to the output I / F 110D2. The instruction regarding the attribute information refers to, for example, an instruction from the controller 15 to the processing circuit 110 to output the attribute information. In this embodiment, a vertical synchronization signal is used as an example of the instruction regarding the attribute information. The output I / F 110D2 outputs the attribute information input from the control circuit 110C to the signal processing circuit 34.

[0101] Specifically, first, the reception I / F 110D1 receives information on the attribute information from the controller 15. A vertical synchronization signal is received as an instruction. Next, the control circuit 110C generates attribute information related to the oldest stored digital image data 70B among the digital image data 70B stored in the memory 112 as attribute information corresponding to the vertical synchronization signal received by the reception I / F 110D1. Then, the output I / F 110D2 outputs the attribute information generated by the control circuit 110C to the signal processing circuit 34.

[0102] As an example, as shown in FIG. 9, the process performed by the image sensor 38 includes an image capturing process and an output process.

[0103] In the imaging process, exposure, reading of analog image data 70A, resetting of photoelectric conversion element 42, analog signal processing, digital signal processing, first storage, acquisition of digital image data 70B, image processing, and second storage are performed in that order.

[0104] Exposure is performed by the photoelectric conversion element 42. Reading of the analog image data 70A, resetting of the photoelectric conversion element 42, and analog signal processing are performed by the readout circuit 110A. Reading of the analog image data 70A is started on condition that the vertical synchronization signal is received by the reception I / F 110D1. The period during which exposure is performed by the photoelectric conversion element 42 is a period during which reading of the analog image data 70A and resetting of the photoelectric conversion element 42 are not performed. Resetting of the photoelectric conversion element 42 refers to an operation of erasing residual charges of each pixel in the photoelectric conversion element 42. Exposure by the photoelectric conversion element 42 is performed during the period from the previous reset of the photoelectric conversion element 42 by the readout circuit 110A to the reading.

[0105] The digital signal processing is performed by the digital processing circuit 110B. The first storage refers to the storage in memory 112 of the digital image data 70B obtained by performing the digital signal processing. The acquisition of the digital image data 70B refers to the acquisition of the digital image data 70B from the memory 112. The first storage and acquisition of the digital image data 70B are performed by the control circuit 110C. The image processing is performed by the image processing circuit 110E on the digital image data 70B acquired by the control circuit 110C. The second storage refers to the storage in memory 112 of the digital image data 70B that has been subjected to the image processing. The second storage is performed by the control circuit 110C.

[0106] In the output step, attribute information is generated and output. The generation of attribute information refers to the generation of attribute information for each frame regarding the digital image data 70B stored in the memory 112. The generation of attribute information is performed by the control circuit 110C. The output of attribute information refers to the output of the attribute information generated by the control circuit 110C by the output I / F 110D2.

[0107] Fig. 10 shows an example of a flow from reading image data of the first frame (1F) to the sixth frame (6F) to storing it in the memory 112, and an example of output timing of attribute information for each frame. The "F" in 1F to 6F shown in Fig. 10 stands for "Frame." For convenience of explanation, the example shown in Fig. 10 shows a mode in which one frame's worth of digital image data 70B is stored in each of the first storage area 112A, the second storage area 112B, the third storage area 112C, the fourth storage area 112D, the fifth storage area 112E, and the sixth storage area 112F of the memory 112.

[0108] 10, each time a vertical synchronization signal is received by the reception I / F 110D1, reading of one frame of analog image data 70A from the photoelectric conversion element 42 is started. In the example shown in FIG. 10, the first to sixth vertical synchronization signals are sequentially received by the reception I / F 110D1, and the analog image data of the first to sixth frames are read out. Then, reading of the data 70A starts.

[0109] The analog image data 70A is read out line by line from the first horizontal line to the last horizontal line of the photoelectric conversion element 42 in accordance with a horizontal synchronization signal, and when the readout for each line is completed, each pixel of the horizontal line for which the readout has been completed is reset. The analog image data 70A is converted into digital image data 70B, and the digital image data 70B is stored in the memory 112 in a manner distinguishable by frame in the order of the frames of the read out analog image data 70A (image capture order) and in a FIFO manner. Specifically, the latest digital image data 70B is overwritten and stored by frame in the first storage area 112A, the second storage area 112B, the third storage area 112C, the fourth storage area 112D, the fifth storage area 112E, and the sixth storage area 112F in sequence. The overwriting and saving is repeatedly performed in the order of the first storage area 112A, the second storage area 112B, the third storage area 112C, the fourth storage area 112D, the fifth storage area 112E, and the sixth storage area 112F.

[0110] The output I / F 110D2 outputs attribute information related to the latest digital image data 70B of the digital image data 70B stored in the memory 112 to the signal processing circuit 34. In this case, first, the control circuit 110C generates attribute information related to the latest digital image data 70B of the digital image data 70B stored in the memory 112 every time storage of one frame of the latest digital image data 70B in the memory 112 is completed at the timing when the vertical synchronization signal is accepted by the acceptance I / F 110D1. Then, the output I / F 110D2 outputs the attribute information generated by the control circuit 110C to the signal processing circuit 34.

[0111] 11A and 11B, the attribute information of each frame (the first to sixth frames in the example shown in FIGS. 11A and 11B) is output by the output I / F 110D2 to the signal processing circuit 34. Then, the signal processing circuit 34 outputs the input attribute information to the controller 15.

[0112] As an example, as shown in Fig. 11A, the attribute information output in the first frame is attribute information related to the digital image data 70B stored in the first storage area 112A. In the example shown in Fig. 11A, examples of the attribute information related to the digital image data 70B stored in the first storage area 112A include information including the address of the first storage area 112A, the horizontal size of the digital image data 70B in the first storage area 112A, the vertical size of the digital image data 70B in the first storage area 112A, the imaging time related to the digital image data 70B in the first storage area 112A, the exposure time related to the digital image data 70B in the first storage area 112A, the image sensor sensitivity related to the digital image data 70B in the first storage area 112A, and the erase flag_off. Here, the "erase flag_off" refers to information indicating that the erase flag is off.

[0113] The attribute information output for the second frame is attribute information related to the digital image data 70B stored in the first storage area 112A and attribute information related to the digital image data 70B stored in the second storage area 112B. In the example shown in Fig. 11A, examples of the attribute information related to the digital image data 70B stored in the second storage area 112B include information including the address of the second storage area 112B, the horizontal size of the digital image data 70B in the second storage area 112B, the vertical size of the digital image data 70B in the second storage area 112B, the imaging time related to the digital image data 70B in the second storage area 112B, the exposure time related to the digital image data 70B in the second storage area 112B, the imaging element sensitivity related to the digital image data 70B in the second storage area 112B, and the erase flag_off.

[0114] The attribute information output in the third frame includes attribute information related to the digital image data 70B stored in the first storage area 112A, attribute information related to the digital image data 70B stored in the second storage area 112B, and attribute information related to the digital image data 70B stored in the third storage area 112C. In the example shown in Fig. 11A, examples of the attribute information related to the digital image data 70B stored in the third storage area 112C include information including the address of the third storage area 112C, the horizontal size of the digital image data 70B in the third storage area 112C, the vertical size of the digital image data 70B in the third storage area 112C, the imaging time related to the digital image data 70B in the third storage area 112C, the exposure time related to the digital image data 70B in the third storage area 112C, the imaging element sensitivity related to the digital image data 70B in the third storage area 112C, and the erase flag_off.

[0115] The attribute information output in the fourth frame includes attribute information related to the digital image data 70B stored in the first storage area 112A, attribute information related to the digital image data 70B stored in the second storage area 112B, attribute information related to the digital image data 70B stored in the third storage area 112C, and attribute information related to the digital image data 70B stored in the fourth storage area 112D. In the example shown in Fig. 11A, examples of the attribute information related to the digital image data 70B stored in the fourth storage area 112D include information including the address of the fourth storage area 112D, the horizontal size of the digital image data 70B in the fourth storage area 112D, the vertical size of the digital image data 70B in the fourth storage area 112D, the imaging time related to the digital image data 70B in the fourth storage area 112D, the exposure time related to the digital image data 70B in the fourth storage area 112D, the imaging element sensitivity related to the digital image data 70B in the fourth storage area 112D, and the erase flag_off. Here, the "deletion flag_off" in the attribute information related to the digital image data 70B stored in the first storage area 112A and included in the attribute information output in the fourth frame is changed to "deletion flag_on" by the control circuit 110C. "Deletion flag_on" refers to information indicating that the deletion flag is on.

[0116] In this way, the "delete flag_off" in the attribute information related to the digital image data 70B stored in the first storage area 112A is changed to "delete flag_on" by the control circuit 110C. This means that the attribute information output from the output I / F 110D2 in the fourth frame includes information indicating that the digital image data 70B in the first storage area 112A will be deleted from the memory 112 when the next image is captured.

[0117] When the attribute information output in the fourth frame includes "deletion flag_on", the attribute information on the deletion target image data, which is the digital image data 70B to be deleted in the memory 112, i.e., the attribute information on the first storage area 112A, is deleted by the control circuit 110C with the next imaging (for example, at the time when the imaging of the fifth frame is started, or during the period from the current time until the imaging of the fifth frame is started). The digital image data 70B is also deleted from the first storage area 112A by the control circuit 110C. In this case, for example, with the next imaging, first, the attribute information on the first storage area 112A is deleted by the control circuit 110C, and then, the digital image data 70B is deleted from the first storage area 112A by the control circuit 110C. Note that, although an example of a form in which the digital image data 70B and the attribute information are deleted is given here, the technology of the present disclosure is not limited to this, and "deletion" includes a process of releasing the area and making it overwritable, in addition to actually deleting the digital image data 70B and the attribute information.

[0118] As an example, as shown in FIG. 11B, the attribute information output for the fifth frame includes attribute information related to the digital image data 70B stored in the second storage area 112B, attribute information related to the digital image data 70B stored in the third storage area 112C, attribute information related to the digital image data 70B stored in the fourth storage area 112D, and attribute information related to the digital image data 70B stored in the fifth storage area 112C. 112E. In the example shown in FIG. 11B, as an example of the attribute information on the digital image data 70B stored in the fifth storage area 112E, information including the address of the fifth storage area 112E, the horizontal size of the digital image data 70B in the fifth storage area 112E, the vertical size of the digital image data 70B in the fifth storage area 112E, the imaging time on the digital image data 70B in the fifth storage area 112E, the exposure time on the digital image data 70B in the fifth storage area 112E, the image sensor sensitivity on the digital image data 70B in the fifth storage area 112E, and the erase flag_off is given. Here, the "erase flag_off" in the attribute information on the digital image data 70B stored in the second storage area 112B included in the attribute information output for the fifth frame is changed to "erase flag_on" by the control circuit 110C.

[0119] In this way, when the control circuit 110C changes the "deletion flag_off" in the attribute information related to the digital image data 70B stored in the second storage area 112B to "deletion flag_on", the attribute information related to the deletion target image data, which is the digital image data 70B to be deleted in the memory 112, that is, the attribute information related to the second storage area 112B is deleted by the control circuit 110C with the next imaging (for example, at the time when the imaging of the sixth frame is started, or during the period from the current time until the imaging of the sixth frame is started). The digital image data 70B is also deleted from the second storage area 112B by the control circuit 110C. In this case, for example, with the next imaging, first, the attribute information related to the second storage area 112B is deleted by the control circuit 110C, and then the digital image data 70B is deleted from the second storage area 112B by the control circuit 110C.

[0120] The attribute information output for the sixth frame includes attribute information on the digital image data 70B stored in the third storage area 112C, attribute information on the digital image data 70B stored in the fourth storage area 112D, attribute information on the digital image data 70B stored in the fifth storage area 112E, and attribute information on the digital image data 70B stored in the sixth storage area 112F. In the example shown in Fig. 11B, examples of the attribute information on the digital image data 70B stored in the sixth storage area 112F include information including the address of the sixth storage area 112F, the horizontal size of the digital image data 70B in the sixth storage area 112F, the vertical size of the digital image data 70B in the sixth storage area 112F, the imaging time of the digital image data 70B in the sixth storage area 112F, the exposure time of the digital image data 70B in the sixth storage area 112F, the imaging element sensitivity of the digital image data 70B in the sixth storage area 112F, and the erase flag_off. Here, the "deletion flag_off" in the attribute information for the digital image data 70B stored in the third storage area 112C, which is included in the attribute information output for the sixth frame, is changed to "deletion flag_on" by the control circuit 110C.

[0121] In this way, when the control circuit 110C changes the "deletion flag_off" in the attribute information related to the digital image data 70B stored in the third storage area 112C to "deletion flag_on", the attribute information related to the deletion target image data, which is the digital image data 70B to be deleted in the memory 112, that is, the attribute information related to the third storage area 112C, is deleted by the control circuit 110C with the next imaging (for example, at the time when the imaging of the 7th frame starts, or during the period from the current time until the imaging of the 7th frame starts). The digital image data 70B is also deleted from the third storage area 112C by the control circuit 110C. In this case, for example, with the next imaging, first, the attribute information related to the third storage area 112C is deleted by the control circuit 110C, and then the digital image data 70B is deleted from the third storage area 112C by the control circuit 110C.

[0122] In this way, attribute information is output to the signal processing circuit 34 by the output I / F 110D2 for each frame. When the attribute information is transmitted to the controller 15, the signal processing circuit 34 transfers the attribute information to the controller 15. The attribute information transferred from the signal processing circuit 34 to the controller 15 is accepted by the reception I / F 15E of the controller 15. The controller 15 acquires an address, as necessary, from the attribute information accepted by the reception I / F 15E.

[0123] 12 as an example, the controller 15 outputs an address acquired from the attribute information to the reception I / F 110D1 of the imaging element 38 via the output I / F 15D. The reception I / F 110D1 of the imaging element 38 accepts the address from the controller 15. The control circuit 110C acquires digital image data 70B from the memory 112 in accordance with the address accepted by the reception I / F 110D1, and causes the output I / F 110D2 to output the acquired digital image data 70B to the signal processing circuit 34.

[0124] Note that, although an example is described here in which the address is output from the output I / F 15D of the controller 15 to the imaging element 38, the technology of the present disclosure is not limited to this. For example, the controller 15 may further include a transmission I / F, and the address may be transmitted to the imaging element 38 from the transmission I / F.

[0125] In addition, although an example in which the address is received by the reception I / F 110D1 has been described here, the technology of the present disclosure is not limited to this. For example, the image sensor 38 may further include a reception I / F, and the address transmitted from the controller 15 may be received by the reception I / F of the image sensor 38.

[0126] Next, the operation of the imaging device 10 will be described.

[0127] First, the flow of attribute information output processing executed by the processing circuit 110 of the imaging element 38 will be described with reference to FIG.

[0128] In the attribute information output process shown in Fig. 13, first, in step ST10, the control circuit 110C determines whether or not a vertical synchronization signal has been received by the reception I / F 110D1. If the vertical synchronization signal has not been received by the reception I / F 110D1 in step ST10, the determination is negative, and the attribute information output process proceeds to step ST22. If the vertical synchronization signal has been received by the reception I / F 110D1 in step ST10, the determination is positive, and the attribute information output process proceeds to step ST12.

[0129] In step ST12, the control circuit 110C stores the digital image data 70B obtained by capturing an image in the memory 112, and thereafter, the attribute information output process proceeds to step ST14.

[0130] In step ST14, the control circuit 110C generates attribute information for the digital image data 70B stored in the memory 112 in step ST12, and holds the generated attribute information, after which the attribute information output process proceeds to step ST16.

[0131] In step ST16, the control circuit 110C determines whether the number of frames that can store the digital image data 70B in the memory 112 reaches the upper limit number due to the next imaging. In this embodiment, the memory 112 has six storage areas, namely, the first storage area 112A, the second storage area 112B, the third storage area 112C, the fourth storage area 112D, the fifth storage area 112E, and the sixth storage area 112F, so the "upper limit number" referred to here is "6." Note that the upper limit number is not limited to this and may be any number of frames that can be stored in the memory 112.

[0132] In step ST16, if the number of frames of digital image data 70B that can be stored in memory 112 does not reach the upper limit as a result of the next imaging, the determination is negative, and the attribute information output process proceeds to step ST20. In step ST16, if the number of frames of digital image data 70B that can be stored in memory 112 reaches the upper limit as a result of the next imaging, the determination is positive, and the attribute information output process proceeds to step ST18.

[0133] In step ST18, the control circuit 110C turns on the deletion flag included in the attribute information for the one frame of digital image data 70B that was stored in the memory 112 most recently out of all frames of digital image data 70B stored in the memory 112, and then the attribute information output processing proceeds to step ST20.

[0134] In step ST20, the control circuit 110C causes the output I / F 110D2 to output the currently held attribute information to the signal processing circuit 34, and then the attribute information output process proceeds to step ST22.

[0135] In step ST22, the control circuit 110C determines whether or not a condition for terminating the attribute information output process (hereinafter referred to as the "attribute information output process termination condition") is satisfied. One example of the attribute information output process termination condition is that an instruction to terminate the attribute information output process is accepted by the acceptance device 84 (see FIG. 4). In step ST22, if the attribute information output process termination condition is not satisfied, the determination is negative, and the attribute information output process proceeds to step ST10. In step ST22, if the attribute information output process termination condition is satisfied, the determination is positive, and the attribute information output process ends.

[0136] Next, the flow of the data erasure process executed by the processing circuit 110 of the image sensor 38 will be described with reference to FIG.

[0137] 14, first, in step ST50, the control circuit 110C determines whether or not attribute information has been output by the output I / F 110D2. If attribute information has not been output by the output I / F 110D2 in step ST50, the determination is negative, and the data erasure process proceeds to step ST60. If attribute information has been output by the output I / F 110D2 in step ST50, the determination is positive, and the data erasure process proceeds to step ST52.

[0138] In step ST52, the control circuit 110C judges whether or not the vertical synchronization signal has been received by the reception I / F 110D1. If the vertical synchronization signal has not been received by the reception I / F 110D1 in step ST52, the judgment is negative, and the judgment of step ST52 is performed again. If the vertical synchronization signal has been received by the reception I / F 110D1 in step ST52, the judgment is positive, and the data erasure process proceeds to step ST54.

[0139] In step ST54, the control circuit 110C judges whether or not attribute information with an erase flag set to on is held. If attribute information with an erase flag set to on is not held in step ST54, the judgment is negative, and the data erase process proceeds to step ST60. If attribute information with an erase flag set to on is held in step ST54, the judgment is positive, and the data erase process proceeds to step ST56.

[0140] In step ST56, the control circuit 110C first identifies the digital image data 70B from the address included in the attribute information with the erase flag turned on among all the attribute information currently held. Of these, the control circuit 110C erases the attribute information whose erasure flag is set to ON. Then, the control circuit 110C erases the digital image data 70B specified by the address from the memory 112, and thereafter the data erasure process proceeds to step ST60.

[0141] In step ST60, the control circuit 110C judges whether or not a condition for terminating the data erasure process (hereinafter referred to as a "data erasure process termination condition") has been satisfied. One example of the data erasure process termination condition is that an instruction to terminate the data erasure process has been accepted by the accepting device 84 (see FIG. 4). In step ST60, if the data erasure process termination condition has not been satisfied, the judgment is negative, and the data erasure process proceeds to step ST50. In step ST60, if the data erasure process termination condition has been satisfied, the judgment is positive, and the data erasure process ends.

[0142] As described above, in the imaging device 10, an instruction regarding the attribute information is received by the reception I / F 110D1, and the attribute information according to the received instruction is output by the output I / F 110D2 to the subsequent circuit 13. This allows the subsequent circuit 13 to request the imaging element 38 to provide digital image data 70B by using the attribute information. Therefore, the subsequent circuit 13 can selectively obtain the necessary digital image data 70B from the imaging element 38. Furthermore, the power consumption required for output by the imaging element 38 can be reduced compared to the case where both the digital image data 70B and at least a part of the attribute information are output simultaneously.

[0143] Furthermore, in the imaging device 10, the control circuit 110C generates attribute information according to the vertical synchronization signal from the controller 15, and the generated attribute information is output by the output I / F 110D2 to the signal processing circuit 34. Therefore, the subsequent circuit 13 can acquire the attribute information from the imaging element 38 without newly generating and using a signal different from the vertical synchronization signal.

[0144] Furthermore, in the imaging device 10, attribute information relating to the latest digital image data 70B among all the digital image data 70B stored in the memory 112 is output by the output I / F 110D2. Therefore, the subsequent circuit 13 can obtain the attribute information relating to the latest digital image data 70B.

[0145] Furthermore, in the imaging device 10, attribute information related to the latest digital image data 70B at the timing when the vertical synchronization signal is received by the reception I / F 110D1 is output by the output I / F 110D2. Therefore, the subsequent circuit 13 can obtain attribute information related to the latest digital image data 70B at the timing when the vertical synchronization signal is received by the reception I / F 110D1.

[0146] In addition, in the imaging device 10, the output I / F 110D2 can output attribute information for each of the frames of digital image data 70B, and the attribute information is output by the output I / F 110D2 in the order of imaging. Therefore, the subsequent circuit 13 can obtain the digital image data 70B from the imaging element 38 in the order of imaging.

[0147] Furthermore, the image capturing device 10 employs information including the address, image size, image capturing time, and image capturing conditions as the attribute information. Therefore, compared to a case where the attribute information does not include any of the address, image size, image capturing time, and image capturing conditions, the range of applications of the digital image data 70B acquired by the subsequent circuit 13 from the image capturing element 38 can be expanded.

[0148] In addition, in the imaging device 10, when the digital image data 70B is erased from the memory 112 with the next imaging, the attribute information output from the output I / F 110D2 includes a erase flag that is set to ON. It is possible to prevent the output of the data 70B from being instructed by the subsequent circuit 13.

[0149] Furthermore, in the imaging device 10, when the digital image data 70B is erased from the memory 112 following the next imaging, the attribute information whose erase flag is set is erased from the memory 112 by the control circuit 110C, and the digital image data 70B identified from the address included in the attribute information whose erase flag is set is erased from the memory 112 by the control circuit 110C. Therefore, both the attribute information and the digital image data 70B related to the attribute information can be erased at different timings.

[0150] Furthermore, in the imaging device 10, attribute information transmitted from the controller 15 is received by the reception I / F 110D1. In the control circuit 110C, digital image data 70B is acquired from the memory 112 in accordance with the attribute information received by the reception I / F 110D1, and the acquired digital image data 70B is output to the signal processing circuit 34 by the output I / F 110D2. Therefore, the imaging element 38 can output the digital image data 70B requested by the subsequent circuit 13 to the subsequent circuit 13.

[0151] In addition, in the imaging device 10, an imaging element in which the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single chip is adopted as the imaging element 38. This increases the portability of the imaging element 38 compared to an imaging element in which the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are not integrated into a single chip. In addition, the degree of freedom in design can be increased compared to an imaging element in which the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are not integrated into a single chip. Furthermore, this can contribute to the miniaturization of the imaging device 10 compared to an imaging element in which the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are not integrated into a single chip.

[0152] 6, a stacked type imaging element in which a memory 112 is stacked on a photoelectric conversion element 42 is adopted as the imaging element 38. This makes it possible to increase the transfer speed of image data from the photoelectric conversion element 42 to the memory 112 compared to a case in which the photoelectric conversion element 42 and the memory 112 are not stacked. The increase in the transfer speed also contributes to an increase in the processing speed of the entire processing circuit 110. In addition, the degree of freedom in design can be increased compared to a case in which the photoelectric conversion element 42 and the memory 112 are not stacked. Furthermore, it can also contribute to a reduction in the size of the imaging device 10 compared to a case in which the photoelectric conversion element 42 and the memory 112 are not stacked.

[0153] Furthermore, in the imaging device 10, a live view image or the like based on the digital image data 70B output by the output I / F 110D2 is displayed on the display 26 by the CPU 15A. This allows the user to visually recognize the live view image or the like based on the digital image data 70B output by the output I / F 110D2.

[0154] Furthermore, in the imaging device 10, the digital image data 70B output by the output I / F 110D2 is stored by the CPU 15A in the storage 15B, a smart device, a personal computer, a server, a USB memory, and / or a memory card, etc. This makes it possible to manage the digital image data 70B output by the output I / F 110D2.

[0155] In the above embodiment, an example in which digital image data 70B for multiple frames is stored in memory 112 has been described, but the technology of the present disclosure is not limited to this. For example, compressed image data may be stored in memory 112 in addition to digital image data 70B for multiple frames. The compressed image data is obtained by a compression process performed by image processing circuit 110E. Compression refers to a process of compressing digital image data 70B. That is, in addition to digital image data 70B for multiple frames, memory 112 stores digital image data 70B compressed by image processing circuit 110E, which is an example of a "compression circuit" according to the technology of the present disclosure. Alternatively, compressed image data obtained by compressing the pixel image data 70B may be stored.

[0156] Specifically, the control circuit 110C stores digital image data 70B for a predetermined number of frames (four frames in the example shown in FIG. 15) in the memory 112. The control circuit 110C also stores compressed image data obtained by compressing the digital image data by the image processing circuit 110E in the memory 112. Furthermore, the control circuit 110C associates attribute information related to the compressed image data and compression specification information capable of specifying that the image data has been compressed with the compressed image data.

[0157] In this case, as shown in FIG. 15 as an example, the memory 112 has addresses "0X00540000", "0X00580000", and "0X005C0000" in addition to the addresses described in the above embodiment, and each address has a storage area for one frame. In the example shown in FIG. 15, when the digital image data 70B of the fourth frame is stored in the fourth storage area 112D (storage area with an address of "0X00300000"), the digital image data 70B of the fourth frame is compressed by the image processing circuit 110E. The compressed image data ("compressed data" in the example shown in FIG. 15) obtained by compressing the digital image data 70B of the fourth frame is stored in the fifth storage area 112E (storage area with an address of "0X00400000") by the control circuit 110C.

[0158] 15, when the imaging of the fourth frame is completed, the digital image data 70B of the first frame is erased from the first storage area 112A (the storage area having the address "0X00000000") by the control circuit 110C, and the digital image data 70B of the fifth frame is stored in the first storage area 112A by the control circuit 110C. Then, when the digital image data 70B of the fifth frame is stored in the first storage area 112A, the digital image data 70B of the fifth frame is compressed by the image processing circuit 110E. The compressed image data obtained by compressing the digital image data 70B of the fifth frame is stored in the sixth storage area 112F (the storage area having the address "0X00500000") by the control circuit 110C.

[0159] 15, upon completion of imaging of the fifth frame, digital image data 70B for the second frame is erased from the second storage area 112B (storage area with an address of "0X00100000") by the control circuit 110C, and digital image data 70B for the sixth frame is stored in the second storage area 112B by the control circuit 110C. Then, when the digital image data 70B for the sixth frame is stored in the second storage area 112B, the image processing circuit 110E compresses the digital image data 70B for the sixth frame. The compressed image data obtained by compressing the digital image data 70B for the sixth frame is stored in the seventh storage area 112G (storage area with an address of "0X00540000") by the control circuit 110C.

[0160] In this manner, the digital image data 70B from the fourth frame onwards is compressed by the image processing circuit 110E, and the compressed image data obtained by the compression is stored in a free storage area of ​​the memory 112 by the control circuit 110C.

[0161] Furthermore, when digital image data 70B from the fourth frame onwards is compressed and stored in memory 112 as compressed image data, attribute information is generated by control circuit 110C as shown in Figs. 16A and 16B as an example.

[0162] As an example, as shown in FIG. 16A, the attribute information output in the first to third frames is the same as that in FIG. 11A. 16A, the attribute information of the digital image data 70B of the fourth frame includes an address of "0X00300000" capable of identifying the fourth storage area 112D, an image size, an image capture time, an image capture condition, and an erase flag_off. The erase flag of the attribute information of the first frame included in the attribute information outputted in the fourth frame is turned on. As a result, the digital image data 70B of the first frame is erased from the first storage area 112A with the imaging of the fifth frame. The attribute information outputted in the fourth frame also includes attribute information of the compressed image data of the digital image data 70B of the fifth frame. In the example shown in FIG. 16A, the attribute information of the compressed image data of the digital image data 70B of the fifth frame includes an address of "0X00400000" capable of identifying the fifth storage area 112E, an image size, an image capture time, an image capture condition, and compression specification information.

[0163] As an example, as shown in FIG. 16B, the attribute information output for the fifth frame includes attribute information related to the digital image data 70B for the fifth frame. In the example shown in FIG. 16B, the address of "0X00000000" capable of identifying the first storage area 112A, the image size, the imaging time, the imaging conditions, and the erase flag_off are shown as the attribute information related to the digital image data 70B for the fifth frame. In addition, the erase flag of the attribute information for the second frame included in the attribute information output for the fifth frame is turned on. As a result, the digital image data 70B for the second frame is erased from the second storage area 112B with the imaging of the sixth frame. In addition, the attribute information output for the fifth frame also includes attribute information related to the compressed image data of the digital image data 70B for the fifth frame and attribute information related to the compressed image data of the digital image data 70B for the sixth frame. In the example shown in FIG. 16B, attribute information related to the compressed image data of the sixth frame of digital image data 70B includes an address of “0X00500000” capable of identifying the sixth memory area 112F, image size, image capture time, image capture conditions, and compression identification information.

[0164] As an example, as shown in FIG. 16B, the attribute information outputted for the sixth frame includes attribute information related to the digital image data 70B of the sixth frame. In the example shown in FIG. 16B, the address of "0X01000000" capable of identifying the second storage area 112B, the image size, the imaging time, the imaging conditions, and the erase flag_off are shown as the attribute information related to the digital image data 70B of the sixth frame. In addition, the erase flag of the attribute information of the third frame included in the attribute information outputted for the sixth frame is turned on. As a result, the digital image data 70B of the third frame is erased from the third storage area 112C with the imaging of the seventh frame. In addition, the attribute information outputted for the fifth frame also includes attribute information related to the compressed image data of the digital image data 70B of the fifth frame, attribute information related to the compressed image data of the digital image data 70B of the sixth frame, and attribute information related to the compressed image data of the digital image data 70B of the seventh frame. In the example shown in FIG. 16B, the attribute information relating to the compressed image data of the seventh frame of digital image data 70B includes an address of "0X00540000", an image size, an image capturing time, image capturing conditions, and compression specification information.

[0165] In this way, the control circuit 110C stores digital image data 70B for a preset number of frames in the memory 112, the control circuit 110C also stores compressed image data in the memory 112, and the control circuit 110C associates attribute information related to the compressed image data with compression specification information (in the example shown in Figs. 16A and 16B, the compression specification information is included in the attribute information). Therefore, even when the digital image data 70B and the compressed image data are stored in the memory 112, the digital image data 70B and the compressed image data can be specified in a distinguishable manner.

[0166] In the example shown in FIGS. 16A and 16B, the attribute information regarding the digital image data 70B includes the following: Although an example including an address, an image size, an image capturing time, an image capturing condition, and an erase flag has been shown, the technology of the present disclosure is not limited thereto. For example, when compressed image data is stored in the memory 112, as shown in Figs. 17A and 17B as an example, the attribute information regarding the digital image data 70B may include a compression schedule flag indicating whether or not the compressed image data is scheduled to be stored in the memory 112, in addition to the address, the image size, the image capturing time, the image capturing condition, and the erase flag. The compression schedule flag that is turned on is an example of the "compression schedule information" according to the technology of the present disclosure, and indicates that the compressed image data is scheduled to be stored in the memory 112 with the next image capturing. The compression schedule flag that is turned off indicates that the compressed image data is not scheduled to be stored in the memory 112 with the next image capturing.

[0167] When the compression scheduled flag is turned on, the digital image data 70B related to the attribute information with the compression scheduled flag turned on is compressed by the image processing circuit 110E in association with the next imaging. For example, as shown in FIG. 17A, the attribute information related to the digital image data 70B of the first frame included in the attribute information outputted in the third frame includes "compression scheduled flag_on" indicating that the compression scheduled flag is on. In this case, the digital image data 70B of the first frame is compressed by the image processing circuit 110E in association with the next imaging (imaging of the fourth frame), and the compressed image data obtained by the compression is stored in the fifth storage area 112E. Accordingly, the attribute information outputted in the fourth frame includes attribute information related to the compressed image data stored in the fifth storage area 112E (storage area with an address of "0X00400000"). The attribute information related to the compressed image data stored in the fifth storage area 112E includes compression specification information, which enables the subsequent circuit 13 to specify that the compressed image data is stored in the fifth storage area 112E.

[0168] 17A and 17B, when the deletion flag is turned on by the control circuit 110C, the compression to be flag is also turned on by the control circuit 110C. As a result, even if the digital image data 70B is deleted from the memory 112, the compressed image data obtained by compressing the digital image data 70B before deletion is held by the memory 112.

[0169] In this way, when compressed image data is stored in memory 112, the attribute information output from output I / F 110D2 includes an on-set compression planned flag, thereby giving the output destination of the compressed image data (e.g., downstream circuit 13) time to prepare for accepting the compressed image data.

[0170] In addition, examples of the compression format when the digital image data 70B is compressed into compressed image data include MPEG, JPEG, TIFF, PNG, GIF, and BMP. When the compressed image data is stored in the memory 112, as shown in Figs. 18A and 18B as an example, the attribute information output from the output I / F 110D2 may include compression format specification information capable of specifying the compression format of the compressed image data. In the example shown in Figs. 18A and 18B, the compression format specification information is included in the attribute information together with the compression specification information. This allows the output destination of the compressed image data (for example, the subsequent circuit 13) to perform processing according to the compression format.

[0171] In the above embodiment, an example was given in which attribute information is output by the output I / F 110D2 after one frame of digital image data 70B is stored in the memory 112, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 19, attribute information may be output by the output I / F 110D2 at the timing when an instruction related to the attribute information (e.g., a vertical synchronization signal) is received by the reception I / F 110D1. This makes it possible to provide the attribute information to the subsequent circuit 13 instantly in response to an instruction regarding the attribute information.

[0172] In the above embodiment, an example was described in which the attribute information output for each frame includes attribute information for multiple frames, but the technology of the present disclosure is not limited to this, and for example, the attribute information output for each frame may be only attribute information related to the digital image data 70B of the corresponding frame, as shown in Fig. 20. In this case, the amount of attribute information output to the signal processing circuit 34 for each frame can be reduced compared to when the attribute information output for each frame includes attribute information for multiple frames.

[0173] In the example shown in Fig. 20, the attribute information output for each frame includes an erasure flag and a to-be-compressed flag, but the technology of the present disclosure is not limited to this, and for example, the attribute information output for each frame does not have to include an erasure flag and a to-be-compressed flag, as shown in Fig. 21. Also, the attribute information output for each frame may include a to-be-compressed flag without including an erasure flag.

[0174] In addition, in the above embodiment, an example form has been described in which attribute information for each of all frames is output by the output I / F 110D2, but the technology disclosed herein is not limited to this, and the downstream circuit 13 may instruct the image sensor 38 as to how many frames of attribute information should be output to the output I / F 110D2.

[0175] In this case, for example, instead of the attribute information output process shown in FIG. 13, the attribute information output process shown in FIG. 22 is executed by the processing circuit 110 of the image sensor .

[0176] The attribute information output process shown in Fig. 22 differs from the attribute information output process shown in Fig. 13 in that it has step ST10A instead of step ST10, step ST18A instead of step ST18, and step ST20A instead of step ST20. The controller 15 outputs an attribute information request instruction to the image sensor 38 via the output I / F 15D. The attribute information request instruction includes a vertical synchronization signal and output amount information. The output amount information refers to information indicating the amount of attribute information output by the output I / F 110D2 of the image sensor 38. The "output amount" referred to here is defined, for example, by the number of frames.

[0177] 22, in step ST10A, the control circuit 110C determines whether or not the attribute information request instruction output from the controller 15 has been accepted by the reception I / F 110D1. If the attribute information request instruction has not been accepted by the reception I / F 110D1 in step ST10A, the determination is negative, and the attribute information output process proceeds to step ST22. If the attribute information request instruction has been accepted by the reception I / F 110D1 in step ST10A, the determination is positive, and the attribute information output process proceeds to step ST12.

[0178] In step ST18A, the control circuit 110C selects attribute information to be output to the signal processing circuit 34 via the output I / F 110D2 from among the attribute information currently held, according to the output amount indicated by the output amount information included in the attribute information request instruction. For example, if the output amount indicated by the output amount information is one frame, the control circuit 110C selects attribute information related to the latest digital image data 70B for one frame, and if the output amount indicated by the output amount information is two frames, the control circuit 110C selects attribute information related to the latest digital image data 70B for two frames.

[0179] In step ST20A, the control circuit 110C causes the signal processing circuit 34 to output the attribute information selected in step ST18A to the output I / F 110D2. It is possible to prevent the I / F 110D2 from outputting more attribute information than necessary. In addition, since the output amount is specified by the number of frames, the subsequent circuit 13 can process the attribute information on a frame-by-frame basis.

[0180] In the above embodiment, an example in which the attribute information and digital image data 70B are output to the signal processing circuit 34 by the output I / F 110D2 has been described, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 23, the attribute information and digital image data 70B may be output to the signal processing circuit 34 by the output I / F 110D2 and the output I / F 110D3. In the example shown in FIG. 23, the processing circuit 110 has the output I / F 110D2 and the output I / F 110D3. The output I / F 110D3 is an example of a "first output unit" according to the technology of the present disclosure, and the output I / F 110D2 is an example of a "second output unit" according to the technology of the present disclosure.

[0181] The output I / F 110D3 is connected to the control circuit 110C. The signal processing circuit 34 also includes a reception I / F 34C. The output I / F 110D2 outputs attribute information to the signal processing circuit 34. The attribute information output by the output I / F 110D2 is received by the reception I / F 34A of the signal processing circuit 34. The output I / F 110D3 also outputs digital image data 70B to the signal processing circuit 34 under the control of the control circuit 110C. The digital image data 70B output by the output I / F 110D3 is received by the reception I / F 34C of the signal processing circuit 34.

[0182] According to the configuration shown in FIG. 23, the attribute information is output by the output I / F 110D2, and the digital image data 70B is output by the output I / F 110D3, so that the attribute information and the digital image data 70B can be output in parallel.

[0183] In the above embodiment, the lens-interchangeable imaging device 10 is exemplified, but the technology of the present disclosure is not limited thereto. For example, as shown in Fig. 24, a smart device 200 may be equipped with an imaging device body 214 having a configuration and functions equivalent to the imaging device body 12 described in the above embodiment.

[0184] 24, a smart device 200 includes a housing 212, and an imaging device main body 214 is housed in the housing 212. An example of the smart device 200 is a smartphone. Here, examples of the smart device 200 include a smartphone or a tablet terminal, which is an electronic device with an imaging function.

[0185] An imaging lens 216 is attached to the housing 212. In the example shown in Fig. 24, the imaging lens 216 is exposed from the rear surface 212A at the upper left of the rear surface 212A of the housing 212 when the smart device 200 is placed vertically. The center of the imaging lens 216 is located on the optical axis L1. An imaging element 38 is built into the imaging device body 214. The imaging device body 214 captures subject light from the imaging lens 216. The subject light captured in the imaging device body 214 is imaged on the imaging element 38, and imaging is performed as described in the above embodiment.

[0186] As an example, as shown in Fig. 25, command keys 222 and a touch panel display 224 are provided on a front surface 212B of the housing 212. When the smart device 200 is placed vertically, the command keys 222 are arranged on the lower part of the front surface 212B, and the touch panel display 224 is arranged above the command keys 222. The touch panel display 224 includes a display 226 and a touch panel 228, and the touch panel 228 is overlaid on the display 226. The touch panel display 224 includes a display 226 and a touch panel 228. The touch panel display 224 and the instruction keys 222 have the same functions as the UI device 17 described in the above embodiment. That is, the touch panel display 224 and the instruction keys 222 present information to the user and receive instructions from the user. Instructions from the user are realized, for example, by a touch operation on a soft key (for example, a soft key corresponding to the release button 20, etc.) displayed on the touch panel display 224.

[0187] The display 226 is an example of a "display unit (display)" according to the technology of the present disclosure. The smart device 200 is an example of an "imaging device" according to the technology of the present disclosure. In the example shown in FIG. 24, an example in which only the imaging device body 214 is built into the smart device 200 is shown, but the technology of the present disclosure is not limited to this, and multiple digital cameras may be built into the smart device 200. In this case, it is sufficient that at least one digital camera is equipped with the imaging device body 214.

[0188] In the above embodiment, an imaging element in which the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single chip is exemplified as the imaging element 38, but the technology of the present disclosure is not limited to this. For example, it is sufficient that at least the photoelectric conversion element 42 and the memory 112 out of the photoelectric conversion element 42, the processing circuit 110, and the memory 112 are integrated into a single chip.

[0189] In the above embodiment, the attribute information is exemplified as information including an address, an image size, an image capture time, and an image capture condition, but the technology of the present disclosure is not limited to this. For example, the attribute information may be information including an address among the address, the image size, the image capture time, and the image capture condition, and may be information including an address, an image size, an image capture time, and / or an image capture condition.

[0190] In the above embodiment, the communication I / Fs are connected according to the PCI-e connection standard, but the technology of the present disclosure is not limited to this. Instead of the PCI-e connection standard, other connection standards such as LVDS, SATA, SLVS-EC, or MIPI may be adopted.

[0191] In the above embodiment, the communication between the imaging element 38 and the signal processing circuit 34, the communication between the controller 15 and the imaging element 38, and the communication between the signal processing circuit 34 and the controller 15 are all wired communications. However, the technology of the present disclosure is not limited to this. The communication between the imaging element 38 and the signal processing circuit 34, the communication between the controller 15 and the imaging element 38, and / or the communication between the signal processing circuit 34 and the controller 15 may be wireless communications.

[0192] In the above embodiment, an example in which the UI device 17 is incorporated in the imaging device body 12 has been described, but at least some of the components included in the UI device 17 may be externally attached to the imaging device body 12. In addition, at least some of the components included in the UI device 17 may be used by being connected to the external I / F 104 as separate entities.

[0193] In addition, in the above embodiment, a frame rate of 120 fps is exemplified, but the technology of the present disclosure is not limited to this, and the frame rate for imaging (e.g., the frame rate applied in the imaging process shown in FIG. 9) and the frame rate for output (e.g., the frame rate applied in the output process shown in FIG. 9 and / or the frame rate for outputting digital image data 70B) may be different frame rates. The frame rate for imaging and / or the frame rate for outputting may be a fixed frame rate or a variable frame rate. In the case of a variable frame rate, for example, a predetermined condition (e.g., a condition that an instruction to change the frame rate has been accepted by the acceptance device 84 and / or a condition that a predetermined timing for changing the frame rate has arrived) may be set. If the frame rate is variable, the frame rate may be changed according to, for example, an instruction received by the receiving device 84, or according to the operation rate of the subsequent circuit 13 and / or the imaging element 38.

[0194] In addition, in the above embodiment, the signal processing circuit 34 is exemplified, but the technology of the present disclosure is not limited to this, and one or more signal processing circuits may be used in addition to the signal processing circuit 34. In this case, the imaging element 38 may be directly connected to each of the multiple signal processing circuits.

[0195] In the above embodiment, the processing circuit 110 is realized by a device including an ASIC and an FPGA, but the technology of the present disclosure is not limited to this. For example, at least the control circuit 110C among the multiple devices included in the processing circuit 110 may be realized by a software configuration using a computer.

[0196] In this case, for example, as shown in FIG. 26, the image sensor 38 has a built-in computer 852, and an attribute information output program 902 for causing the computer 852 to execute the attribute information output process according to the above embodiment and a data erasure program 904 for causing the computer 852 to execute the data erasure process according to the above embodiment are stored in a storage medium 900. An example of the storage medium 900 is any portable storage medium such as an SSD or a USB memory, which is a non-transient storage medium. In the following, for convenience of explanation, when it is not necessary to distinguish between the attribute information output process and the data erasure process, they will be referred to as "image sensor side processing", and when it is not necessary to distinguish between the attribute information output program 902 and the data erasure program 904, they will be referred to as "image sensor side programs".

[0197] The computer 852 includes a CPU 852A, a storage 852B, and a memory 852C. The storage 852B is a non-volatile storage device such as an EEPROM, and the memory 852C is a volatile storage device such as a RAM. The imaging element-side program stored in the storage medium 900 is installed in the computer 852. The CPU 852A executes imaging element-side processing in accordance with the imaging element-side program.

[0198] The imaging element-side program may be stored in the storage 852B instead of the storage medium 900. In this case, the CPU 852A reads out the imaging element-side program from the storage 852B and loads the read imaging element-side program in the memory 852C. The CPU 852A then executes imaging element-side processing in accordance with the imaging element-side program loaded in the memory 852C.

[0199] Further, although an example is given here in which the attribute information output program 902 and the data erasure program 904 are stored in storage 852B, this is not limited thereto, and one of the attribute information output program 902 and the data erasure program 904 may be stored in storage 852B, and the other may be stored in the storage medium 900.

[0200] In addition, the imaging element side program may be stored in a memory unit of another computer or server device connected to the computer 852 via a communication network (not shown), and the imaging element side program may be downloaded and installed in the computer 852 in response to a request from the imaging device 10 or smart device 200 described above.

[0201] It is not necessary to store all of the image sensor side programs in the memory of another computer or server device connected to computer 852; only a portion of the image sensor side programs (e.g., attribute information output program 902 or data erasure program 904) may be stored.

[0202] In the example shown in FIG. 26, an example is shown in which the computer 852 is built into the imaging element 38, but the technology of the present disclosure is not limited to this, and for example, the computer 852 may be provided outside the imaging element 38.

[0203] 26, the CPU 852A is a single CPU, but it may be a plurality of CPUs. Also, a GPU may be used in place of the CPU 852A.

[0204] 26 illustrates a computer 852, but the technology of the present disclosure is not limited thereto, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 852. Also, instead of the computer 852, a combination of a hardware configuration and a software configuration may be used.

[0205] The hardware resources for executing the image sensor side processing described in the above embodiment can be various processors as shown below. As the processor, for example, as described above, a CPU, which is a general-purpose processor that functions as a hardware resource for executing the image sensor side processing by executing software, i.e., a program, can be mentioned. In addition, as the processor, for example, a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process such as an FPGA, PLD, or ASIC, can be mentioned. Each processor has a built-in or connected memory, and each processor executes the image sensor side processing by using the memory.

[0206] The hardware resource that executes the image sensor side processing may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the image sensor side processing may be a single processor.

[0207] As an example of a system using one processor, first, as represented by computers such as a client and a server, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes image sensor side processing. Second, as represented by SoC, there is a system using a processor that realizes the functions of the entire system including multiple hardware resources that execute image sensor side processing in one IC chip. In this way, the image sensor side processing is realized using one or more of the above various processors as hardware resources.

[0208] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0209] 1 illustrates an image capturing device 10, and the example illustrated in Fig. 24 illustrates a smart device 200, but the technology of the present disclosure is not limited thereto. That is, the technology of the present disclosure is also applicable to various electronic devices (e.g., fixed-lens cameras, personal computers, wearable terminal devices, etc.) incorporating an image capturing device having a configuration and function equivalent to the image capturing device main body 12 described in the above embodiment, and the same actions and effects as those of the image capturing device 10 and the smart device 200 can be obtained with these electronic devices as well.

[0210] In addition, in the above embodiment, the displays 26 and 226 are exemplified, but the technology of the present disclosure is not limited thereto. For example, a separate display attached to the imaging device may be used as the "display unit" according to the technology of the present disclosure.

[0211] In addition, the above-described various processes are merely examples, and it goes without saying that unnecessary steps may be deleted, new steps may be added, and the order of processes may be changed without departing from the spirit of the invention.

[0212] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0213] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0214] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0215] The following supplementary notes are further disclosed regarding the above embodiment.

[0216] (Appendix 1) A processor; A memory that stores image data obtained by capturing an image and is built into the image capturing element, The processor is Controlling the storage of the image data in the memory; storing attribute information of the image data in the memory; outputting the image data stored in the memory; Accept instructions regarding the above attribute information, Output the attribute information according to the received instruction. Image sensor.

Claims

1. A memory that stores image data obtained by capturing an image; a control circuit for controlling the storage of the image data in the memory and storing attribute information of the image data in the memory; an output interface that outputs the image data stored in the memory; a reception interface for receiving an instruction regarding the attribute information, The output interface outputs the attribute information according to the instruction accepted by the acceptance interface before the recording of the image data corresponding to the attribute information in the memory is completed. Image sensor.

2. The imaging device according to claim 1 , wherein the output interface outputs the attribute information at a timing when the instruction is accepted by the acceptance interface.

3. 3. The image sensor according to claim 1, wherein the instruction is an external frame synchronization signal.

4. The imaging device according to claim 1 , wherein the output interface outputs the attribute information relating to the latest image data among the image data stored in the memory.

5. The imaging device according to claim 4 , wherein the output interface outputs the attribute information related to the latest image data at a timing when the instruction is accepted by the acceptance interface.

6. the output interface is capable of outputting attribute information for each of the plurality of image data; The imaging device according to claim 1 , wherein the attribute information is output by the output interface in the order in which the images are captured.

7. The imaging device according to claim 1 , wherein the attribute information includes at least one of an address, an image size, an image capturing time, and an image capturing condition.

8. 8. The imaging element according to claim 1, wherein, when the image data is erased from the memory upon the next image capture, the attribute information output from the output interface is information including erasure information indicating that the image data will be erased from the memory upon the next image capture.

9. 9. The imaging element according to claim 8, wherein when the image data is erased from the memory, the control circuit erases from the memory the attribute information relating to the image data to be erased, which is the image data in the memory that has been selected for erasure, and erases the image data to be erased from the memory.

10. a compression circuit for compressing the image data; The control circuit stores the image data for a predetermined number of frames in the memory, stores compressed image data obtained by compressing the image data by the compression circuit in the memory, and associates the attribute information regarding the compressed image data and compression identification information that can identify that the image data has been compressed with the compressed image data.

11. The imaging element according to claim 10, wherein when the compressed image data is stored in the memory, the attribute information output from the output interface is information including compression planned information indicating that the compressed image data is scheduled to be stored in the memory.

12. 12. The image pickup device according to claim 10, wherein the attribute information output from the output interface is information including information capable of identifying a compression format of the compressed image data.

13. the instruction includes an amount of the attribute information to be output by the output interface; The imaging device according to claim 1 , wherein the output interface outputs the attribute information in the output amount.

14. The image pickup device according to claim 13 , wherein the output amount is defined by the number of frames of the image data.

15. a receiver for receiving the attribute information transmitted from a subsequent circuit located subsequent to the imaging element; 15. The imaging element according to claim 1, wherein the control circuit acquires the image data from the memory in accordance with the attribute information received by the receiver, and outputs the acquired image data to the downstream circuit via the output interface.

16. the output interface includes a first output interface and a second output interface; the first output interface outputs the image data; The imaging device according to claim 1 , wherein the second output interface outputs the attribute information.

17. The imaging device according to claim 1 , wherein at least the photoelectric conversion element and the memory are integrated into a single chip.

18. 18. The imaging element according to claim 17, wherein the imaging element is a stacked type imaging element in which the memory is stacked on the photoelectric conversion element.

19. The imaging element according to any one of claims 1 to 18, a control device that performs at least one of control for displaying an image based on the image data output by the output interface on a display and control for storing the image data output by the output interface in a storage device; and An imaging device comprising:

20. 1. A method of operating an imaging device, comprising: Controlling storage of image data obtained by capturing an image in a memory that stores the image data; storing attribute information of the image data in the memory; outputting the image data stored in the memory; accepting an instruction regarding the attribute information; and and outputting the attribute information according to the received instruction before completing recording of the image data corresponding to the attribute information in the memory. How the image sensor works.

21. A computer applied to an image sensor, Storing the image data in a memory for storing the image data obtained by imaging Controlling storing attribute information of the image data in the memory; outputting the image data stored in the memory; accepting an instruction regarding the attribute information; and a program for executing a process including outputting the attribute information according to the received instruction before recording of the image data corresponding to the attribute information in the memory is completed;

Citation Information

Patent Citations

  • Image data management system

    JP2007081892A

  • Imaging apparatus

    JP2013012983A

  • Solid-state imaging device, driving method for solid-state imaging device, and electronic device

    WO2014007004A1