Imaging apparatus

The imaging device optimizes data transmission by using a pixel array with multiple column output lines and adaptive encoding to manage power consumption and circuit area, addressing the challenges of multi-stream output in conventional technologies.

JP2024162915APending Publication Date: 2024-11-21CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023078902
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Conventional imaging devices face challenges in managing circuit mounting area and power consumption due to the need for multiple encoding circuits and increased data transfer times as the amount of image data increases, particularly in multi-stream output configurations.

Method used

The imaging device employs a pixel array with multiple column output lines and an encoding unit that compresses and encodes image data, allowing for differential lane usage based on data type and mode, thereby optimizing transmission to an image processing unit.

Benefits of technology

This approach effectively controls transmission of captured image data to reduce power consumption and circuit area requirements, enhancing the device's miniaturization and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162915000001_ABST
    Figure 2024162915000001_ABST
Patent Text Reader

Abstract

To control transmission between an image pickup device and a processing unit on the basis of a standpoint of both the mounting area and power consumption.SOLUTION: An imaging apparatus has: a coding unit that performs coding of image data read in first reading and second reading according to preset compressibility; and a transmission-side I / F unit that transmits data obtained by the coding unit. The coding unit is shared in the first reading and the second reading, and the first reading and the second reading are different in the compressibility of the coding unit.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an imaging device. [Background technology]

[0002] Some imaging elements provided in imaging devices such as digital cameras can change the readout timing for each region independently for each line of the pixel section, and output image data with different frame rates and exposures in parallel (so-called multi-stream output). As the number of pixels read out increases due to multi-stream output, the amount of readout data increases, which causes a decrease in readout speed. Therefore, the decrease in readout speed can be suppressed by reducing the amount of data by compressing the image data inside the imaging element. On the other hand, the increase in the area of ​​the imaging element and power consumption due to the circuit scale of the compression circuit become problems.

[0003] The I / F section of the image sensor and image processing circuit is configured to transfer serial differential signals in order to meet the demands of high transfer speeds. This serial differential signal is implemented with a pair of positive and negative wires as one lane, and is defined so that multiple lanes constitute one channel. Since the transmission line bandwidth of the I / F section is constant, there is a problem in that an increase in the amount of image data results in a relatively longer transfer time for all image signals.

[0004] If a high-speed transfer configuration is adopted, such as by increasing the number of lanes to shorten the transfer time, issues arise such as the power consumption of the I / F section and the implementation area of ​​the transmission path between the image sensor and the image processing circuit, making it difficult to implement in a small-scale imaging device.

[0005] Patent document 1 describes that when R / B pixel signals and Gr / Gb pixel signals are output in a multi-stream, an encoding circuit is provided on each path, and the compression ratios and the number of output lanes in the encoding circuits for the R / B pixel signals and the Gr / Gb pixel signals are made different. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2021-136603 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the conventional technology disclosed in Patent Document 1, the R / B pixel signals and the Gr / Gb pixel signals are compressed and encoded by encoding circuits provided corresponding to each signal, which poses the problem that two encoding circuits require a large mounting area and consume a large amount of power.

[0008] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide an imaging device that can appropriately control the transmission of captured image data to an image processing unit from the standpoint of circuit implementation area and power consumption. [Means for solving the problem]

[0009] In order to achieve the above-mentioned object, an imaging device of the present invention comprises a pixel array in which a plurality of pixels are arranged in a matrix and each pixel column has a plurality of column output lines; an encoding unit that compresses and encodes image data based on a signal read out from the pixel array; an output unit that outputs the image data; and control means that controls the output unit to output first image data based on a signal read out from a first pixel row of the pixel array via a first column output line, and to output second image data based on a signal read out from a second pixel row different from the first pixel row of the pixel array via a second column output line different from the first column output line, in parallel with the first image data, wherein the number of lanes of the output unit used when outputting the first image data is greater than the number of lanes of the output unit used when outputting the second image data from the output unit, and a compression rate for the first image data in the encoding unit is higher than a compression rate for the second image data by the encoding unit. Effect of the Invention

[0010] According to the present invention, it is possible to provide an imaging device capable of appropriately controlling the transmission of captured image data to an image processing unit from the viewpoint of circuit mounting area and power consumption. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of the configuration of an imaging device according to an embodiment of the present invention. [Diagram 2] 1 is a diagram showing a configuration of an image sensor according to an embodiment of the present invention. [Diagram 3] 1 is a block diagram of an image processing circuit according to an embodiment of the present invention. [Figure 4] FIG. 2 is a block diagram of an encoding unit according to the embodiment of the present invention. [Diagram 5] FIG. 2 is a block diagram of a decoding unit according to an embodiment of the present invention. [Figure 6] 2 is a block diagram of an I / F transceiver unit according to the embodiment of the present invention. FIG. [Figure 7] 4A and 4B are diagrams showing readout rows of an image sensor in a live view and a flicker detection mode according to an embodiment of the present invention. [Figure 8] FIG. 11 is a timing diagram illustrating a case where a flicker detection mode is performed during a live view according to an embodiment of the present invention. [Figure 9] 4 is a flowchart showing an operation flow according to an embodiment of the present invention. [Figure 10] 10 is a flowchart showing an operation flow according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0013] Example 1 FIG. 1 is a diagram showing an example of the configuration of an imaging device 100 according to an embodiment of the present invention.

[0014] The imaging device 100 according to this embodiment is a digital camera. As shown in Fig. 1, a first lens group 101 is disposed at the tip of an imaging optical system, and the first lens group 101 is held so as to be movable forward and backward in the optical axis direction.

[0015] The aperture / shutter 102 adjusts the amount of light during shooting by adjusting its opening diameter, and also functions as a shutter for adjusting the exposure time when shooting still images. The aperture / shutter 102 and the second lens group 103 move forward and backward together in the optical axis direction, and perform a variable magnification function (zoom function) in conjunction with the forward and backward movement of the first lens group 101. The third lens group 104 adjusts the focus by moving forward and backward in the optical axis direction.

[0016] The optical low-pass filter 105 is an optical element for reducing false colors and moire in a captured image. The image sensor 106 includes a two-dimensional CMOS image sensor and peripheral circuits, and is disposed on the imaging plane of the imaging optical system.

[0017] The zoom actuator 111 rotates a cam barrel (not shown) to drive the first lens group 101 to the third lens group 104 forward and backward in the optical axis direction to perform a magnification change operation. The aperture shutter actuator 112 controls the aperture diameter of the aperture / shutter 102 to adjust the amount of light for shooting, and also controls the exposure time when shooting a still image. The focus actuator 113 drives the third lens group 104 forward and backward in the optical axis direction to perform focus adjustment.

[0018] The control unit 121 is responsible for various controls in the imaging device 100, and includes a CPU (arithmetic unit) and a communication interface circuit (not shown). Based on a predetermined program stored in a ROM in the memory 134, the control unit 121 drives various circuits of the imaging device 100 and executes a series of operations such as AF, shooting, image processing, and recording. It also performs flicker calculations to calculate the frequency of flicker, which will be described later.

[0019] The image processing circuit 122 performs processes such as subject detection, gamma conversion, color interpolation, and JPEG compression of the image captured by the image sensor 106. The memory 134 is a RAM, a ROM, or the like, and stores predetermined programs, holds image data during image processing, and parameter data required for image processing.

[0020] A focus driving circuit 123 drives and controls the focus actuator 113 based on the focus detection result, and drives the third lens group 104 forward and backward in the optical axis direction to perform focus adjustment. An aperture shutter driving circuit 124 drives and controls the aperture shutter actuator 112 to control the opening of the aperture / shutter 102. A zoom driving circuit 125 drives the zoom actuator 111 in response to a zoom operation by the photographer.

[0021] The display device 131 is configured with, for example, a rear liquid crystal display or an electronic viewfinder, and displays information regarding the shooting mode of the imaging device 100, a preview image before shooting, a confirmation image after shooting, an image showing the in-focus state at the time of focus detection, etc. The operation switch group 132 includes a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, etc.

[0022] The recording medium 133 is a removable flash memory or the like, and records image files acquired by photographing.

[0023] FIG. 2 is a diagram showing an example of a circuit configuration of the image sensor 106.

[0024] The image sensor 106 includes a pixel array 210, a vertical scanning circuit 203 that selects pixel rows in the pixel array 210, and a horizontal scanning circuit 202 that selects pixel columns in the pixel array 210. The column circuit 201 reads out signals from pixels selected by the vertical scanning circuit 203 and the horizontal scanning circuit 202 from among the pixels in the pixel array 210, and performs processes such as amplification of the pixel signals and analog-to-digital conversion (hereinafter referred to as AD conversion).

[0025] A plurality of unit pixels 205 are arranged in a matrix in the pixel array 210. Each unit pixel 205 has a photoelectric conversion element that converts incident light into an electric signal by photoelectric conversion, and a plurality of transistor elements. The symbol R attached to each unit pixel 205 indicates that a red color filter, the symbols Gr and Gb indicate that a green color filter, and the symbol B indicates that a blue color filter is arranged on the pixel.

[0026] A reference signal generating circuit 205 supplies a reference voltage to circuits constituting the multiple column circuits 201. Then, an AD conversion circuit in the column circuit 201 compares the pixel signal with the reference voltage. A register group 208 stores settings for each operation mode through communication from an external control device such as the control unit 121. A timing generator 209 operates the vertical scanning circuit 203 and the horizontal scanning circuit 202 based on the settings stored in the register group 208.

[0027] The vertical scanning circuit 203 selects a pixel row of the pixel array 210, and enables a read pulse outputted based on a horizontal synchronization signal outputted from the control unit 121 in the selected pixel row. The column circuit 201 includes an amplifier for amplifying signals for each column, an AD conversion circuit for converting the output of the unit pixels 205 into a digital signal, a memory for storing the AD conversion result, an addition / subtraction processing circuit for the AD conversion result or the signal before AD conversion, etc. Then, the pixel signal of the selected row is stored in the memory via the amplifier.

[0028] The pixel signals for one row stored in the memory are selected in sequence in the column direction by the horizontal scanning circuit 202, and the pixel value data converted into digital values ​​is sent to an encoding unit 211. The encoding unit 211 compression-encodes the pixel value data, and the encoded data encoded by the encoding unit 211 is temporarily held in a line memory 212. The pixel value data is read out from the line memory 212 and output to the outside from an output amplifier 214 via a transmission side I / F unit 213, which is an output unit. This operation is repeated for the number of rows, and all the pixel value data is output to the outside.

[0029] Taking the pixel column (first column) in which pixel R is arranged as an example, the pixel row of pixel R is connected to column output line 206, and the pixel row of pixel Gb is connected to column output line 207. In the column (second column) in which pixel Gr is arranged, the pixel row of pixel Gr is connected to column output line 206, and the pixel row of pixel B is connected to column output line 207. This connection pattern is repeated for the pixels in the third column and onwards.

[0030] With this configuration, for example, in readout of a still image, the signal readout of the pixel row of pixel R uses the column output line 206, and the signal readout of the pixel row of pixel Gb uses the column output line 207. On the other hand, in readout of a live view image signal, it becomes possible to simultaneously read out and add the signals of pixel R connected to the same column output line 206, and similarly simultaneously read out and add the signals of pixel Gb connected to the column output line 207.

[0031] Furthermore, signals of pixels used for a live view image can be read out via column output lines 206 and signals of thinned pixels not used for the live view image can be read out via column output lines 207 at the same time. Similarly, signals of pixels used for the live view image can be read out via column output lines 207 and signals of thinned pixels not used for the live view image can be read out via column output lines 206 at the same time.

[0032] Here, as an example, a configuration having two column output lines per pixel column is used, but a configuration having three or more column output lines 206, 207 per pixel column and simultaneously reading out pixel signals of multiple rows is also acceptable. For example, when 12 or more column output lines 206, 207 are provided for each column, image data for 12 rows can be read out in one readout operation. When using such a multi-line configuration of column output lines 206, 207, a back-illuminated or stacked configuration that allows circuit implementation on the opposite side to the light incident surface is preferable.

[0033] The interface for outputting image data to the downstream image processing circuit 122 is preferably a high-speed communication system using a differential pair such as an LVDS system or an SLVS system. In the case of a configuration having multiple lanes as an interface for outputting image data, a different signal may be output for each lane.

[0034] 3 is a diagram showing an example of a circuit configuration of the image processing circuit 122. The image processing circuit 122 includes a receiving side I / F unit 301 which is a receiving unit, a line memory 302, a decoding unit 303, an image correction processing unit 304, and a development processing unit 305.

[0035] A receiving side I / F unit 301 receives the encoded data transferred from the image sensor 106 and temporarily stores it in a line memory 302. The encoded data is read from the line memory and decoded by a decoding unit 303. An image correction processing unit 304 corrects various problems (dark current, shading, pixel defects, etc.) associated with the process characteristics of the image sensor for the decoded pixel values, and then performs image processing such as noise processing and false color countermeasures on the corrected pixel signals.

[0036] As image processing, correction of lens characteristics (influence of diffraction and aberration) may be performed. The image data developed by the development processing unit 305 is output to the display device 131, the recording medium 133, the memory 134, etc. via the control unit 121.

[0037] 4 is a block diagram of the encoding unit 211 in this embodiment. The encoding unit 211 in this embodiment performs encoding for each "encoding block" obtained by dividing input image data into units of a predetermined number of pixels.

[0038] The quantization unit 401 in Fig. 4 requantizes the digital image data quantized by the column circuit 201 according to a set quantization parameter (hereinafter referred to as QP). The code amount measurement unit 404 measures the amount of coded data of past blocks and judges whether the amount is larger or smaller than a target data amount, and the code amount control unit 405 determines (or updates) the next QP value. In this embodiment, the QP value is not changed during processing of a coding block, so the next QP value is determined before processing of the coding block. Of course, a specification in which the QP value is variable during processing of the coding block may be adopted.

[0039] If the QP value is fixed during data processing of the coding block, the prediction unit 402 may simply be a delay element. The adder (subtractor) 407 calculates the difference between the quantized pixel data of the target pixel and the quantized pixel data of the previous processing, and outputs the difference to the entropy coding unit 403.

[0040] The entropy coding unit 403 employs an entropy coding method for generating coded data by assigning codes based on the frequency of occurrence of input data values. There is no particular limit to the type of entropy coding method, but examples of the entropy coding method include Golomb coding and Huffman coding.

[0041] The multiplexing unit 406 multiplexes the coded data generated by the entropy coding unit 403 and the QP determined by the code amount control unit 405, and outputs the multiplexed data as stream data.

[0042] 5 is a block diagram of the decoding unit 303 in this embodiment. The separation unit 501 separates the QP value and the coded data from the stream data, supplies the coded data to the entropy decoding unit 502, and supplies the QP value to the inverse quantization unit 505. The entropy decoding unit 502 decodes the coded data by performing a process that is the reverse of the coding process.

[0043] The adder 503 is an adder that adds the decoding result obtained by the entropy decoding unit 502 and the predicted value from the prediction unit 504. The output value of this adder 503 is a value equivalent to the quantization result value of the pixel of interest in the encoding unit 211. The inverse quantization unit 505 inverse quantizes the output value of this adder 503 in accordance with the QP value supplied from the separation unit 501, and obtains decoded pixel data.

[0044] 6 is a block diagram of an I / F transmission / reception unit in an embodiment. Here, communication between a transmission side I / F unit 213 (output unit) and a reception side I / F unit 301 (reception unit) will be described. In this embodiment, an embedded clock method such as SLVS (Scalable Low Voltage Signaling) is exemplified as a high-speed differential I / F having a protocol. Of course, other methods such as LVDS (Low voltage differential signaling) may be appropriately adopted.

[0045] 6, the transmission side I / F unit 213 has a conversion processing unit 602, a payload generating unit 603, and a lane division unit 604. The conversion processing unit 602 converts the output stream data of the encoding unit 211 into data in units of bytes. The payload generating unit 603 generates payloads of the sensor data for each horizontal period. The lane division unit 604 divides the data into multiple transmission lanes as serial data.

[0046] A reference numeral 601 denotes a register that stores the control settings of each of these processing units. A lane division unit 604 distributes the payload data in byte units to multiple lanes. At this time, packet header information is added to the beginning of each lane for each line.

[0047] The receiving side I / F unit 301 includes a lane combining unit 606, a payload data processing unit 607, and a conversion unit 608. The lane combining unit 606 combines serial data from multiple lanes. The payload data processing unit 607 extracts valid data from the packet data after lane combining. The conversion unit 608 converts payload data in byte units into output stream data.

[0048] Also, a reference numeral 605 denotes a register unit that stores control settings of these processing units. By updating the register units of the transmitting side I / F unit 213 and the receiving side I / F unit 301, data transfer lane allocation during live view display and during shooting is appropriately changed, and lane activation / inactivation is controlled.

[0049] Next, the operation in the live view mode and the flicker detection mode will be described with reference to Fig. 7. Fig. 7 shows the arrangement of the unit pixels 205 in the image sensor 106, as well as which row of signals is to be read out in the operation in the live view mode and the operation in the flicker detection mode.

[0050] In this embodiment, a combination of a live view mode and a flicker detection mode will be described as an example of a multi-stream, but the combination and use are not limited to this. For example, combinations such as a live view mode and a dimming mode, a still image mode and a live view mode, or a video mode and a phase difference detection mode may be used.

[0051] First, the pixel rows from which signals are read out during operation in live view mode will be described. Live view operation is to display an image on the rear liquid crystal display or electronic viewfinder included in the display device 131, and is provided for the user to determine the angle of view and shooting conditions such as exposure conditions. In other words, since the image is not stored on the recording medium 133 or the like, and the number of pixels on the rear liquid crystal display or the like is less than the number of unit pixels 205 included in the image sensor 106, it is not necessary to read image data from all unit pixels 205. For this reason, it is common to read out the image data with a reduced number of pixels approximately equal to the number of pixels or rows on the rear liquid crystal display or the like.

[0052] As an example, when the image sensor 106 has 8000 effective pixels horizontally and 5400 effective pixels vertically, at least one-third the number of pixels in the vertical direction is sufficient to obtain a resolution equivalent to Full HD sufficient for display on the display device 131. In this embodiment, an example is shown in which readout rows are thinned out in a cycle of three rows.

[0053] 7, an example is shown in which pixels are reduced by thinning out every three rows in the row direction in each of a Red row including unit pixels 205 having an R color filter and a Blue row including unit pixels 205 having a B color filter. Also, an example is shown in which two rows are added and one row is not read out (thinned out) in the three row cycle.

[0054] The readout method of a three-row cycle or averaging two of those rows is one example, and other cycles may be used, such as a five-row cycle in which three of those rows are averaged and read out. The addition method may be a simple addition operation or an averaging operation. The method of adding two row signals assumes an averaging method in which different rows are simultaneously connected to the column output lines 206 and 207, but the averaging method between rows is not limited to this.

[0055] Next, the pixel rows from which signals are read out during operation in the flicker detection mode will be described. During operation in the flicker detection mode, it is preferable to set the frame rate and the readout rows so that high-frequency / low-frequency flicker detection can be achieved across the entire frame. For this reason, in this embodiment, the image sensor 106 is configured to read out one row every nine rows in the row direction for each of the red and blue rows.

[0056] Furthermore, pixel rows from which signals are read out when operating in flicker detection mode are pixel rows from which signals are not read out when operating in live view mode, as shown in Figure 3. In this way, by using pixel rows from which signals are not read out when operating in live view mode, it becomes possible to set the live view mode and flicker detection mode simultaneously. In other words, it becomes possible to control the accumulation and reset operations for each mode independently.

[0057] In this embodiment, signals output from pixel rows that are the subject of operation in the live view mode and pixel rows that are the subject of operation in the flicker detection mode are processed using a common readout circuit 201. Therefore, the readout timing to the readout circuit 201 is exclusively controlled.

[0058] Although the number of pixel rows that are the target of operation in the live view mode and the flicker detection mode has been described in FIG. 7, the number of pixels whose signals are read out in the horizontal direction (pixel columns) may be different. As an example, when operating in the live view mode, signals from all pixels may be read out without thinning or adding in the horizontal direction, and when operating in the flicker detection mode, signals from only one column may be read out in a cycle of three columns. With such a configuration, the readout rate when operating in the flicker detection mode may be improved.

[0059] Note that, if priority is given to the processing load on the image processing circuit 122, the number of pixel columns that are the targets of operation in the live view mode and the flicker detection mode may be set to be the same. Since the image processing circuit 122 performs sequential processing on a pixel row basis, if the number of pixels is the same in both modes, it is possible to standardize processing parameters and reduce the processing load.

[0060] An example of operating in parallel in the live view mode and the flicker detection mode will be described with reference to FIG.

[0061] Fig. 8 shows an example in which 12 frames of flicker detection images are read out while one live view image is read out in one VD period corresponding to the frame rate. The operation sequence shown in Fig. 8 makes it possible to operate in the flicker detection mode without stopping the operation of the image sensor 106 in the live view mode.

[0062] The vertical synchronization signal VD is a signal that is transmitted from the control unit 121 and input to the image sensor 106, and indicates the start timing of a frame. Here, an example is shown in which live view images are read out at a frame rate of 30 fps.

[0063] 8, at time T801, a vertical synchronization signal VD is input to the image sensor 106, a signal read operation in live view mode is started in synchronization with the vertical synchronization signal VD, and the Mth frame image is read out. Then, at time T803, a signal read operation in flicker detection mode is started for the Nth frame.

[0064] 2, signal readout in the live view mode is performed through the column output line 206, and signal readout in the flicker detection mode is performed through the column output line 207. Since signals are read out through different column output lines, it is possible to perform signal readout in the live view mode and signal readout in the flicker detection mode in parallel.

[0065] By starting the read operation in the flicker detection mode without waiting for the end of the read operation in the live view mode, the number of read operations in the flicker detection mode in one cycle can be increased. Furthermore, the period during which the image sensor 106 operates can be shortened, thereby realizing power saving. In order to improve the read speed in the flicker detection mode, the operation may be performed to read signals only from pixel rows in a narrower range than the read range in the live view mode.

[0066] The signal read operation in the flicker detection mode is repeated until time T806. In this embodiment, the read operation in the flicker detection mode is performed for 12 frames from the Nth frame to the (N+11)th frame. At this time, the read operation in the flicker detection mode does not need to be within 1 VD cycle (33.3 msec) of the live view mode, and the accumulation periods in both modes are determined independently based on a predetermined exposure calculation result.

[0067] For example, the accumulation period is determined so that the subject is properly exposed using the immediately preceding frame or the immediately preceding few frames. Note that an image acquired by operation in the flicker detection mode is used for flicker calculation and is not displayed on the display device 131 or the like. Therefore, it is possible to set an accumulation period 820 that is shorter than the accumulation period 810 set for operation in the live view mode.

[0068] At time T802, which is before the timing of the readout operation in the final pixel row in live view mode, a reset operation corresponding to the readout operation in flicker detection mode is performed. Since the column circuit 201 does not need to be used in the reset operation, it is possible to overlap the readout operation in live view mode and the reset operation in flicker detection mode in time. Similarly, at time T804, while the readout operation in flicker detection mode is being performed, a reset operation corresponding to the readout operation in live view mode at time T805 is performed.

[0069] 9 is a flowchart showing the imaging operation flow in the imaging device 100 of this embodiment.

[0070] For ease of understanding, the column circuit 201 in this embodiment converts the imaging signal from the pixel array unit 210 into digital data of 14 bits per pixel. That is, each of the R, B, Gr, and Gb pixels in the Bayer array immediately after imaging is represented by 14 bits.

[0071] Also, in live view mode when not recording (when the shutter button is not fully pressed = not pressed or half pressed), the encoding unit 211 that encodes pixels performs compression encoding with a target of 10 bits per pixel. At this time, the transmitting side I / F unit 213 activates three lanes (turns the corresponding power supply switch ON) to transfer, and the receiving side I / F unit 301 also activates the corresponding three lanes to receive.

[0072] In addition, in the flicker detection mode, the encoding unit 211 does not encode the image data, and the transmission side I / F unit 213 transmits uncompressed data to the image processing circuit 122. At this time, in this embodiment, the transmission side I / F unit 213 transfers the image data for flicker detection (uncompressed image data) using two lanes. That is, the remaining lane is deactivated (the corresponding power supply switch is turned OFF) to reduce power consumption. That is, the transmission side I / F unit 211 activates five lanes to perform transfer, and the reception side I / F unit 301 also activates the corresponding five lanes to perform reception. That is, the remaining lane is inactivated.

[0073] Based on the above, the imaging operation flow in the imaging device 100 of this embodiment will be described with reference to the flowchart of FIG.

[0074] First, in step S901 after the imaging operation is started, the control unit 121 sets various parameters for the live view mode for the image sensor 106 and the image processing circuit 122, and also starts supplying a vertical synchronization signal VD. The period of the vertical synchronization signal VD supplied to the image sensor 106 corresponds to the frame rate at which the image sensor 106 operates, and is set to 1 / 30 sec in this embodiment.

[0075] The control unit 121 sets the transmitting side I / F unit 213 and the receiving side I / F unit 301 so that three preset lanes are activated and the remaining three lanes are inactivated.

[0076] The various parameters include setting parameters for a drive mode for operating the image sensor 106, and exposure parameters such as accumulation period and gain setting.Furthermore, the parameters include at least information for setting the target compression ratio of the encoding unit 211 to "10 / 14", and information for activating three lanes preset in the transmitting side I / F unit 211 and deactivating the remaining three lanes.The lanes used for transmission in the transmitting side I / F unit 213 of the image sensor 106 and the lanes used for reception in the receiving side I / F unit 301 of the image processing circuit 122 are the same.

[0077] Next, in step S902, it is determined whether or not to read out the flicker detection mode, and if the flicker detection mode is not to be read out, the process proceeds to step S903.

[0078] In step S903, the control unit 121 controls the image sensor 106 to start operation in the live view mode, captures images at a preset frame rate, and reads out image data for live view.

[0079] In step S904, the encoded data for each frame of the live view image is compressed and encoded at a compression rate of "10 / 14" and stored in the line memory 212, and then transmitted to the image processing circuit 122 using three lanes of the transmission side I / F unit 213. Storing the encoded data in the line memory 212 makes it possible to perform time-division processing by sharing the encoding unit 211 between encoding the live view image and the flicker detection image. The image processing circuit 122 receives the encoded data transmitted in this manner and performs decoding processing.

[0080] In step S905, it is determined whether readout of all pixel rows required in the live view mode has been completed. Then, the output of image data in the live view mode is repeatedly executed in synchronization with the supply timing of the vertical synchronization signal VD, and is displayed on the display device 131, thereby performing live view display.

[0081] In step S902, it is determined whether or not to read out the flicker detection mode, and if the flicker detection mode is to be read out, the process proceeds to step S906.

[0082] In step S906, similarly to step S901, the control unit 121 sets various parameters for the flicker detection mode for the image sensor 106 and the image processing circuit 122. Then, the process proceeds to step S907.

[0083] The various parameters include at least exposure parameters such as accumulation period and gain setting, information for setting the target compression ratio of the encoding unit 211 to "14 / 14" (or no compression), and information for activating two lanes in the transmitting I / F unit 213 and the receiving I / F unit 301. Of the setting parameters, the aperture value is the same because the aperture / shutter 102 is common to the live view mode and the flicker detection mode.

[0084] In step S907, the control unit 121 controls the image sensor 106 to start operation in the flicker detection mode, captures images at a preset frame rate, and reads out image data for flicker detection.

[0085] In step S908, for the flicker detection image of each frame, data that has not been coded by the coding unit 211 is stored in the line memory 212, and then transmitted to the image processing circuit 122 using two lanes of the transmission side I / F unit 213. At this time, the two lanes used for data transfer of the flicker detection image are assigned to lanes different from the three lanes used for data transfer of the live view image. The image processing circuit 122 receives the data transmitted in this manner. Since the data has not been coded by the coding unit 211, no decoding is performed by the decoding unit 303 either.

[0086] In step S909, it is determined whether readout of all pixel rows required in the flicker detection mode has been completed, and the output of image data in the flicker detection mode is repeatedly executed in synchronization with the supply timing of VD, and flicker calculation is performed by control unit 121.

[0087] In step S910, the control unit 121 determines whether or not to perform an exposure operation. Here, it determines whether or not the shutter button included in the operation switch group 132 has been fully pressed (hereinafter, referred to as SW2 being ON). If SW2 has not been turned ON, the control unit 121 repeats the operations from step S901 to step S909 for each frame. If SW2 has been turned ON in step S910, the control unit 121 proceeds to step S911.

[0088] In step S911, the control unit 121 sets various parameters for the still image mode for the image sensor 106 and the image processing circuit 122, similar to steps S901 and S906. The various parameters include exposure parameters such as accumulation period and gain setting, and information for setting the target compression ratio of the encoding unit 211 to "14 / 14" (or no compression). Furthermore, the information includes at least information for activating five lanes in the transmitting side I / F unit 213 and the receiving side I / F unit 301. In other words, the remaining lane is inactivated.

[0089] Next, in step S912, it is determined whether or not the set exposure time has elapsed, and this is repeated until the exposure time has elapsed.

[0090] In step S913, the control unit 121 performs control to start operation in the still image mode in the imaging element 106, and reads image data for a still image.

[0091] In step S914, for a still image, the encoded data compressed and encoded at a compression rate of "14 / 14" is stored in the line memory 212 and then transmitted to the image processing circuit 122 using the five lanes of the transmission side I / F unit 213. The image processing circuit 122 receives the encoded data transmitted in this manner and performs a decoding process.

[0092] In step S915, it is determined whether or not reading of all rows required in the still image mode has been completed, and when reading of all rows has been completed, still image shooting is performed.

[0093] In this embodiment, the case where 12 frames are read in the flicker detection mode has been described, but the number of frames read in the flicker detection mode and other modes such as the dimming mode is not limited. For example, any number of frames may be read within a range that can be processed by the image processing circuit 122 per unit time. Furthermore, although the flicker detection mode is configured to be non-compressed and two lanes are activated, if the number of lanes can be reduced, the encoding unit 211 may compress and transfer the data.

[0094] In this embodiment, the compression ratio of the live view mode is set to "10 / 14" and three lanes are activated, but if the number of lanes can be further reduced, the compression ratio may be increased and the image may be transferred. Of course, the compression ratio is within a range that ensures live view image quality that can withstand display on the rear LCD or electronic viewfinder.

[0095] Similarly, in this embodiment, image data captured by the image sensor 106 during still image capture is transferred to the image processing circuit 122 uncompressed, but this is not limiting. For example, image data compressed at an arbitrary compression rate for purposes such as improving the curtain speed of the electronic shutter, implementing high-speed continuous shooting by reducing the transmission line bandwidth, and countermeasures against heat may be received by the image processing circuit 122 as the output of the image sensor 106. The lane activation state may also be changed accordingly. Of course, the compression of image data to be recorded is within the scope of ensuring image quality.

[0096] As described above, in the live view mode, compression is performed by encoding using the encoding unit 211, and the number of lanes related to transmission used is limited. In the flicker detection mode that operates simultaneously with the live view mode, compression is not performed by encoding using the encoding unit 211, and power consumption of the compression circuit is reduced. Similarly, in the still image mode, compression is not performed by encoding using the encoding unit 211, and power consumption of the compression circuit is reduced.

[0097] As a result, when live view mode and flicker detection mode are operating simultaneously, five lanes are active, and when still image mode is operating, five lanes are active and the remaining one lane is inactive, allowing for power-saving transmission. Battery consumption can be reduced by controlling the active state of the driver and receiver on the I / F according to the compression ratio of the encoding section.

[0098] Furthermore, if the maximum number of lanes that are activated in both live view mode and still image mode is five, the transmission flex and board wiring may be designed with five transmission lanes between the image sensor 106 and image processing circuit 122. Such a design can reduce the mounting area, contributing to the miniaturization of the image pickup device.

[0099] Example 2 In the first embodiment, a case has been described in which the number of lanes of the transmitting side I / F unit 213 and the receiving side I / F unit 301 are set in advance. In the second embodiment, a case will be described in which the number of lanes of the transmitting side I / F unit 213 and the receiving side I / F unit 301 is determined according to the amount of encoded data.

[0100] The imaging operation flow in the imaging device 100 of this embodiment will be described with reference to the flowchart of Fig. 10. The control unit 121 performs the processing of each step.

[0101] First, in step S1001 after the imaging operation is started, the control unit 121 sets various parameters for the live view mode for the image sensor 106 and the image processing circuit 122, and also starts supplying a vertical synchronization signal VD. The period of the vertical synchronization signal VD supplied to the image sensor 106 corresponds to the frame rate at which the image sensor 106 operates, and is set to 1 / 30 sec in this embodiment.

[0102] The various parameters include at least setting parameters for the drive mode for operating the image sensor 106, exposure parameters such as accumulation period and gain setting, and information for setting the target compression ratio of the encoding unit 211 to "10 / 14".

[0103] Next, in step S1002, it is determined whether or not to read out the flicker detection mode, and if the flicker detection mode is not to be read out, the process proceeds to step S1003.

[0104] In step S1003, the control unit 121 controls the image sensor 106 to start operation in the live view mode, captures images at a preset frame rate, and reads out image data for live view.

[0105] In step S1004, for the live view image of each frame, the code amount measurement unit 404 of the encoding unit 211 compares the data amount before encoding with the data amount after encoding compressed and encoded at a compression rate of "10 / 14". Assume that the number of lanes used by the transmitting side I / F unit 213 is measured to be four for the data amount before encoding, and three for the data amount after encoding. In this case, since the number of lanes used by the data amount after encoding is less than the number of lanes used by the data amount before encoding, three lanes are activated in the transmitting side I / F unit 213 and the receiving side I / F unit 301.

[0106] In step S1005, the encoded data compressed and encoded at a compression rate of "10 / 14" for the image data of the live view image is stored in the line memory 212, and then transmitted to the image processing circuit 122 using the three lanes of the transmission side I / F unit 213. The image processing circuit 122 receives the encoded data transmitted in this manner, and performs a decoding process.

[0107] In step S1006, it is determined whether readout of all rows required in live view mode has been completed, and the output of image data in live view mode is repeatedly executed in synchronization with the supply timing of the VD and displayed on the display device 131, thereby performing a live view display.

[0108] In step S1002, it is determined whether or not to read out the flicker detection mode, and if the flicker detection mode is to be read out, the process proceeds to step S1007.

[0109] In step S1007, similar to step S1001, the control unit 121 sets various parameters of the flicker detection mode for the image sensor 106 and the image processing circuit 122. Note that, similar to the setting parameters for the live view mode, the various parameters for the flicker detection mode include at least exposure parameters such as the accumulation period and gain setting, and information for not compressing the image in the encoding unit 211 (or setting the target compression ratio to "14 / 14"). Of the setting parameters, the aperture value is the same value because the aperture / shutter 102 is common to both the live view mode and the flicker detection mode.

[0110] In step S1008, the control unit 121 controls the image sensor 106 to start operation in the flicker detection mode, captures images at a preset frame rate, and reads out image data for flicker detection.

[0111] In step S1009, for the flicker detection image of each frame, the code amount measurement unit 404 of the encoding unit 211 compares the data amount before encoding with the data amount after encoding compressed and encoded at a compression rate of "14 / 14". Since the compression rate of "14 / 14" is uncompressed, the data amount before encoding and the data amount after encoding are equal. Assume that the number of lanes used by the transmitting side I / F unit 213 is measured to be two lanes. In this case, two lanes of the transmitting side I / F unit 213 and the receiving side I / F unit 301 are activated.

[0112] In step S1010, for the flicker detection image of each frame, data not coded by the coding unit 211 is held in the line memory 212, and then transmitted to the image processing circuit 122 using two lanes of the transmission side I / F unit 213. At this time, the two lanes used for data transfer of the flicker detection image are assigned to lanes different from the three lanes used for data transfer of the live view image. The image processing circuit 122 receives the data transmitted in this manner. Since the data has not been coded by the coding unit 211, no decoding is performed by the decoding unit 303 either.

[0113] In step S1011, it is determined whether readout of all rows required in the flicker detection mode has been completed, and the output of image data in the flicker detection mode is repeatedly executed in synchronization with the supply timing of the vertical synchronization signal VD, and flicker calculation is performed by the control unit 121.

[0114] In step S1012, the control unit 121 determines whether or not to perform an exposure operation. Here, it determines whether or not the shutter button included in the operation switch group 132 has been fully pressed (hereinafter, referred to as SW2 being ON). If SW2 has not been turned ON, the control unit 121 repeats the operations from step S1001 to step S1011 for each frame. If SW2 has been turned ON in step S1012, the control unit 121 proceeds to step S1013.

[0115] In step S1013, the control unit 121 sets various parameters for the still image mode for the image sensor 106 and the image processing circuit 122, similar to steps S1001 and S1007. The various parameters include exposure parameters such as accumulation period and gain setting, and information for setting the target compression ratio of the encoding unit 211 to "14 / 14" (or no compression). Furthermore, the information includes at least information for activating five lanes in the transmitting side I / F unit 213 and the receiving side I / F unit 301. In other words, the remaining one lane is inactivated.

[0116] Next, in step S1014, it is determined whether or not the set exposure time has elapsed, and the process is repeated until the exposure time has elapsed.

[0117] In step S1015, the control unit 121 performs control to start operation in the still image mode in the imaging element 106, and reads image data for a still image.

[0118] In step S1016, for still images, the code amount measurement unit 404 of the encoding unit 211 compares the amount of data before encoding with the amount of data after encoding compressed and encoded at a compression rate of "14 / 14". Since the compression rate of "14 / 14" is uncompressed, the amount of data before encoding and the amount of data after encoding are equal. Assume that the number of lanes used by the transmitting I / F unit 213 is measured to be 5 lanes. In this case, the five lanes of the transmitting I / F unit 213 and the receiving I / F unit 301 are activated.

[0119] In step S1017, for a still image, data not coded by the coding unit 211 is stored in the line memory 212, and then transmitted to the image processing circuit 122 using two lanes of the transmission side I / F unit 213. The image processing circuit 122 receives the data thus transmitted. Since the data has not been coded by the coding unit 211, the decoding unit 303 does not perform decoding either.

[0120] In step S1018, it is determined whether or not reading of all rows required in the still image mode has been completed, and when reading of all rows has been completed, still image shooting is performed.

[0121] As described above, in this embodiment, the number of lanes to be used in each of the live view mode, flicker detection mode, and still image mode is not set as a parameter in advance, and the parameter setting for the target compression ratio is performed in the encoding unit 211. In this way, the number of lanes appropriate for each operation mode can be activated, thereby reducing power consumption.

[0122] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0123] 106 Image sensor 206 column output line 211 Encoding section 213 Transmission I / F section

Claims

1. a pixel array in which a plurality of pixels are arranged in a matrix, each pixel column having a plurality of column output lines; an encoding unit that compresses and encodes image data based on a signal read from the pixel array; an output unit that outputs the image data; a control means for controlling so that first image data based on signals read from a first pixel row of the pixel array via a first column output line is output from the output unit, and second image data based on signals read from a second pixel row different from the first pixel row of the pixel array via a second column output line different from the first column output line is output from the output unit in parallel with the first image data, An imaging device characterized in that the number of lanes of the output unit used when outputting the first image data is greater than the number of lanes of the output unit used when outputting the second image data from the output unit, and the compression rate for the first image data in the encoding unit is higher than the compression rate for the second image data by the encoding unit.

2. 2. The imaging device according to claim 1, wherein the encoding unit compresses and encodes the first image data and the second image data in a time-division manner.

3. The imaging device according to claim 1 , wherein the encoding section does not compress and encode the second image data.

4. 2. The imaging device according to claim 1, wherein the control means controls to activate lanes of the output section in accordance with the respective data amounts of the first image data and the second image data compressed and encoded by the encoding section.

5. The imaging device described in claim 4, characterized in that the control means sets a compression rate by the encoding unit by comparing the number of lanes of the output unit based on the data amount of the first image data before it is compressed and encoded with the number of lanes of the output unit based on the data amount of the first image data after it is compressed and encoded, and sets a compression rate by the encoding unit by comparing the number of lanes of the output unit based on the data amount of the second image data before it is compressed and encoded with the number of lanes of the output unit based on the data amount of the second image data after it is compressed and encoded.

6. 2. The imaging apparatus according to claim 1, wherein the first image data is used for live view, and the second image data is used for flicker detection.

Citation Information

Patent Citations

  • Imaging apparatus, control method of the same and program

    JP2021136603A