Imaging device, method and program

Abstract

To restrain deterioration of an imaging element even when a strong amount of light is continuously detected in a specific region of the imaging element.SOLUTION: An imaging device includes an imaging element, a mechanism capable of changing the relative position between the imaging element and a subject image, deterioration degree estimation means for estimating the degree of deterioration over time for each region of the imaging element, brightness value acquisition means for acquiring a brightness value of a screen corresponding to the region where the degree exceeds a predetermined threshold value, and system control means for executing search processing for searching for an imaging range where the brightness value is equal to or less than a predetermined brightness value by controlling the mechanism while changing the relative position between the imaging element and the subject image when the brightness value exceeds the predetermined brightness value in a brightness value distribution over the entire screen corresponding to the imaging element.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, a method, and a program. [Background technology]

[0002] Some image sensors deteriorate when exposed to light. For example, one type of image sensor is the Complementary Metal-Oxide-Semiconductor (CMOS) sensor, which uses color filters used in general cameras. When a CMOS sensor is exposed to strong light such as sunlight, the microlenses and color filters in the areas exposed to the strong light deteriorate due to the effects of ultraviolet rays and heat, causing fading in only certain areas.

[0003] For example, a single photon avalanche diode (SPAD) sensor is one type of image sensor. A SPAD sensor counts photons that enter a pixel and converts the photon into an electric charge as soon as it enters the pixel. A SPAD sensor can detect the large electric charge generated by the avalanche amplification caused by the electrons as a signal. Because of the mechanism of the SPAD sensor, no noise is introduced during signal readout, so the subject can be clearly photographed without being affected by noise even in dark places. Therefore, SPAD sensors are expected to be widely used as sensors for surveillance applications, etc. When a SPAD sensor is exposed to strong light such as sunlight, the microlenses and color filters in the area exposed to the strong light deteriorate due to the effects of ultraviolet rays and heat, and the pixels that cause avalanche amplification also deteriorate.

[0004] One of the causes of image quality degradation in SPAD sensors is the local change in dark count rate (DCR) depending on the amount of strong light that is locally applied. DCR is the average count value of photons detected in the absence of incident light. Local changes in DCR cause image quality degradation. In addition, in SPAD sensors, degradation of microlenses and color filters occurs just like in CMOS sensors.

[0005] Due to their intended use, pan-head cameras and surveillance cameras are often operated for long periods of time with a fixed shooting angle, and there is a possibility that a strong light source such as sunlight may be included in a specific position within the fixed shooting angle. In such cases, deterioration progresses only in a specific area on the sensor, resulting in partial degradation of image quality.

[0006] For example, Patent Document 1 discloses a technology that reads the brightness value of each pixel of an image sensor, and when the detected value is equal to or greater than a threshold value, adjusts the magnification of the optical zoom and the magnification of the digital zoom section to reduce degradation of the image sensor. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2012-095082 A Summary of the Invention [Problem to be solved by the invention]

[0008] However, the technology disclosed in Patent Document 1 detects the amount of light by the integrated value of the entire image sensor, so when strong light is locally applied, the luminance value of the part where the strong light is locally applied may not be detected. Therefore, the technology disclosed in Patent Document 1 cannot avoid local deterioration of the image sensor. In addition, the technology disclosed in Patent Document 1 does not take into account the degree of deterioration due to the accumulation of strong light, and detects only the situation at that time.

[0009] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to suppress deterioration of the image sensor even when a strong amount of light is continuously detected in a specific area of ​​the image sensor. [Means for solving the problem]

[0010] In order to solve the above-mentioned problems, the imaging device of the present invention is characterized in comprising an imaging element, a mechanism capable of changing the relative position between the imaging element and a subject image, a deterioration degree estimation means for estimating the degree of deterioration over time for each region of the imaging element, a luminance value acquisition means for acquiring a luminance value of a screen corresponding to the region where the degree exceeds a predetermined threshold, and a system control means for executing a search process, when the luminance value exceeds a predetermined luminance value in a luminance value distribution over the entire screen corresponding to the imaging element, by controlling the mechanism to change the relative position between the imaging element and the subject image and to search for an imaging range where the luminance value is equal to or less than the predetermined luminance value. Effect of the Invention

[0011] According to the present invention, even if a strong amount of light is continuously detected in a specific area of ​​the image sensor, deterioration of the image sensor can be suppressed. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an example of the configuration of an imaging device. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of an imaging unit. [Diagram 3] FIG. 13 is a diagram illustrating a relative position change control parameter acquisition unit. [Figure 4] 13 is a flowchart showing a processing procedure. [Diagram 5] FIG. 11 is a diagram showing an example of the degree of deterioration for each region of an image. [Figure 6] FIG. 13 is a diagram illustrating an example of evaluation values ​​calculated for each region. [Figure 7] 11 is a diagram showing the angle of view and the luminance value for each region after optical adjustment. FIG. [Figure 8]13 is a flowchart showing a processing procedure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The imaging device according to the embodiment includes an imaging element, a mechanism, a deterioration degree estimation means, a brightness value acquisition means, and a system control means. The mechanism is capable of changing the relative position between the imaging element and the subject image. The deterioration degree estimation means estimates the degree of deterioration over time for each region of the imaging element. The brightness value acquisition means acquires the brightness value of the screen corresponding to the region whose degree exceeds a predetermined threshold. When the brightness value exceeds a predetermined brightness value in the brightness value distribution of the entire screen corresponding to the imaging element, the system control means executes the following process. That is, in such a case, the system control means executes a search process for searching for an imaging range in which the brightness value is equal to or less than the predetermined brightness value while changing the relative position between the imaging element and the subject image by controlling the mechanism.

[0014] The mechanism may also control at least one of an optical zoom, a lens shift, a shift of an image sensor, an imaging direction of an image capture device, and a roll of the image capture device.

[0015] The system according to the embodiment may further include an evaluation value storage unit that stores an evaluation value based on the brightness value when the imaging range is searched and the degree of deterioration of the area. In this case, when the system control unit cannot find an imaging range whose brightness value is equal to or less than a predetermined brightness value even after the search process is executed, the system control unit controls the mechanism to find an imaging range whose evaluation value is the smallest within the range where the search process is executed.

[0016] The evaluation value storage means may store, as an evaluation value, a product of the luminance value and the degree of deterioration calculated for each region.

[0017] The evaluation value storage means may store a total or average of the stored evaluation values ​​for an arbitrary period.

[0018] The imaging device according to the embodiment may further include an image processing unit. The image processing unit is capable of executing at least one of the following image processing: changing the cropping range of the image generated by the imaging element, transforming the image, and rotating the image. The image processing unit executes image processing so that the difference between the range shown in the image before the mechanism is controlled by the system control unit and the range shown in the image after the mechanism is controlled by the system control unit falls within a predetermined range.

[0019] The control amount calculation means may limit the control amount of the mechanism based on a correction amount that can be processed by image processing.

[0020] The image processing means may also disable image processing when the mechanism is controlled by the user.

[0021] Moreover, the imaging device according to the embodiment may further include a notification unit that notifies a user that the range shown in the image has changed as a result of the mechanism being controlled by the system control unit.

[0022] Moreover, the imaging device according to the embodiment may further include a notification unit that notifies a user that the image quality of an image has changed due to image processing.

[0023] The system control means may also warn that the imaging range is changing while the search process is being performed.

[0024] In addition, if the system control means is unable to find an imaging range in which the brightness value is equal to or less than a predetermined brightness value even after executing the search process, it controls the mechanism to find an imaging range in which the brightness value is the smallest within the range in which the search process was executed.

[0025] Moreover, the method and program according to the embodiment include a degradation degree estimation step, a brightness value acquisition step, and a search process execution step. In the degradation degree estimation step, the degree of degradation over time for each region of the image sensor is estimated. In the brightness value acquisition step, the brightness value of the screen corresponding to the region whose degree exceeds a predetermined threshold is acquired. In the search process execution step, if the brightness value exceeds a predetermined brightness value in the brightness value distribution of the entire screen corresponding to the image sensor, the following process is executed. That is, in such a case, in the search process execution step, a search process is executed to search for an image sensing range in which the brightness value is equal to or less than a predetermined brightness value while changing the relative position between the image sensor and the subject image by controlling a mechanism.

[0026] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0027] <First embodiment> 1 is a block diagram showing an example of the configuration of an imaging device 100 according to the first embodiment. The imaging device 100 includes an imaging lens 101, an aperture 102, an ND filter 103, an imaging element 110, an image processing unit 112, a memory control unit 113, a system control unit 120, a non-volatile memory 121, a system memory 122, and a system timer 123. The imaging device 100 further includes a memory 130, a power control unit 140, a power supply unit 170, an I / F 180, and a pan / tilt control unit 190.

[0028] The imaging lens 101 is a lens group including a zoom lens, a focus lens, and a shift lens, and forms a subject image. The aperture 102 is an aperture used to adjust the amount of light. The ND filter 103 is a neutral density filter.

[0029] The image sensor 110 has an image sensor, performs photoelectric conversion, and outputs digital image data. Here, an example is described in which a single photon avalanche diode (SPAD) sensor is used as the image sensor. The image sensor 110 also has functions such as controlling accumulation by an electronic shutter, changing gain, and changing readout speed, and a mechanism for shifting the image sensor in a direction perpendicular to the optical axis.

[0030] The image processing unit 112 performs image processing on image data from the imaging element 110 or the memory control unit 113. The image processing includes, for example, predetermined pixel interpolation processing, resizing processing such as enlargement / reduction processing, image rotation and geometric deformation, image cropping, detection processing of luminance information, color information, characteristic objects, etc., color conversion processing, gamma correction processing, digital gain addition processing, etc. Image processing methods include image processing by a dedicated arithmetic circuit, image processing by a 3D-LUT processing circuit, etc. Furthermore, the image processing unit 112 performs predetermined arithmetic processing using image data from the imaging element 110, and transmits the arithmetic result to the system control unit 120.

[0031] Based on the transmitted calculation results, the system control unit 120 performs exposure control, distance measurement control, white balance control, etc. As a result, the system control unit 120 performs through-the-lens (TTL) type autofocus (AF) processing, automatic exposure (AE) processing, auto white balance (AWB) processing, etc.

[0032] The output data from the imaging element 110 is written to the memory 130 via the image processing unit 112 and the memory control unit 113. Alternatively, the output data from the imaging element 110 is written to the memory 130 via the memory control unit 113. The memory 130 stores image data from the imaging element 110 or the image processing unit 112. The memory 130 may also be used to temporarily store an image that has been processed by the image processing unit 112 and to return the image to the image processing unit 112 again for application of another image processing. The memory 130 has a storage capacity sufficient to store moving images and audio for a predetermined period of time.

[0033] The non-volatile memory 121 is an electrically erasable and recordable memory, such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). Constants, programs, etc. for the operation of the system control unit 120 are stored in the non-volatile memory 121. The programs referred to here are programs for executing various flowcharts described later.

[0034] The system control unit 120 controls the imaging device 100. The system control unit 120 executes each process of the first embodiment described below by executing a program recorded in the above-mentioned non-volatile memory 121. The system memory 122 uses a RAM (Random Access Memory), and constants and variables for the operation of the system control unit 120, programs read from the non-volatile memory 121, and the like are expanded.

[0035] The system timer 123 is a timing unit that measures the time used for various controls and the time of a built-in clock.

[0036] The power supply control unit 140 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to be energized, and detects whether a battery is attached, the type of battery, and the remaining battery power. Based on the result of this detection and instructions from the system control unit 120, the power supply control unit 140 controls the DC-DC converter to supply the required voltage for the required period to each unit including the external recording medium 150.

[0037] The power supply unit 170 is made up of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li-ion battery, an AC adapter, etc. The I / F 180 is an interface for connecting to an external recording medium 150 such as a memory card or a hard disk, and an external display device 160. The external recording medium 150 is a recording medium such as a memory card for recording captured images and exchanging data with the outside, and a semiconductor memory or the like is used.

[0038] The pan / tilt / roll control unit 190 adjusts and rotates the optical axis direction by moving the entire imaging device 100 in at least one of the pan direction, tilt direction, and roll direction in response to an instruction from the system control unit 120. Alternatively, the pan / tilt / roll control unit 190 adjusts and rotates the optical axis direction by moving the entire imaging unit in at least one of the pan direction, tilt direction, and roll direction in response to an instruction from the system control unit 120. The entire imaging unit includes the imaging lens 101, the aperture 102, the ND filter 103, and the image sensor 110.

[0039] FIG. 2 is a diagram showing an example of the configuration of the image sensor 110 in FIG. 1. Hereinafter, the configuration of the image sensor 110 and a method of reading out an image signal from the image sensor 110 will be described with reference to FIG. 2. As shown in FIG. 2, the image sensor 110 has a plurality of pixels 201, a vertical scanning section 208, a column memory 209, and an output section 210. Note that, for the sake of explanation, FIG. 2 shows two pixels 201 having the same structure arranged in the row direction and two pixels in the column direction, but in reality, more pixels 201 are arranged. Furthermore, the two pixels 201 arranged in the column direction are connected to a vertical signal line 207.

[0040] The image sensor 110 is a light receiving element using a SPAD. Each pixel 201 has an avalanche photodiode (hereinafter referred to as APD) 202, a quench resistor 203, an inverter 204, a counter 205, and a read switch 206. The pixel 201 is covered with color filters of multiple colors to obtain a color image. Light of different wavelength bands that has passed through the color filters is incident on each pixel 201. There are various types of color filters, and they may be of a primary color system or a complementary color system, and the type is not limited as long as a color image can be obtained. As an example, a case where R, G, and B color filters are arranged in a Bayer array will be described here. R, G, and B mean red, green, and blue, respectively.

[0041] The APD 202 is an APD operated in Geiger mode to detect single photons. The Geiger mode is an operation mode of an APD operated by applying a reverse bias voltage larger than the breakdown voltage. When a reverse bias voltage Vbias larger than the breakdown voltage Va is applied to the APD 202 to enter the Geiger mode, carriers generated by the incidence of a single photon cause avalanche multiplication, generating a large current. Here, the reverse bias voltage Vbias is, for example, -20V.

[0042] The quench resistor 203 is a resistive element for stopping avalanche multiplication of the APD 202. When a photon is incident on the APD 202 and a current is generated by avalanche multiplication, a voltage drop occurs in the quench resistor 203. When this voltage drop causes the voltage of the cathode of the APD 202 to swing from the reverse bias voltage Vbias to a breakdown voltage Va or lower, the avalanche multiplication stops. After that, when the cathode of the APD 202 is charged via the quench resistor 203, the voltage returns to the reverse bias voltage Vbias, which is the voltage before the fluctuation.

[0043] In this way, one voltage signal pulse is generated in response to the incidence of a single photon at the cathode of the APD 202. At this time, the width of the voltage signal pulse changes in response to a time constant determined by the resistance value of the quench resistor 203. The APD 202 and the quench resistor 203 constitute a sensor means.

[0044] The inverter 204 as a buffer means functions as a shaping section, shapes the waveform of the voltage signal pulse generated by the APD 202 as described above, and outputs the waveform-shaped signal pulse to its own output terminal.

[0045] The counter 205 receives as an input the signal pulse output by the inverter 204 and counts the rising edges of the signal pulse. The inverter 204 is used as a buffer means because the counter 205 is configured to count rising edges. When a buffer means that does not invert polarity is used, the counter 205 only needs to count the falling edges of the signal pulse output by the buffer.

[0046] As an example, the counter 205 is configured to be able to count the number of pulses for 16 bits. In the following description, the "count value" refers to the value counted by the counter 205. In addition, the counter 205 receives a signal pulse res output by the vertical scanning unit 208, and the reset operation of the count value and the start timing of the count operation are controlled.

[0047] The read switch 206 is turned on in response to a signal pulse sel output sequentially for each row by a vertical scanning unit 208. The count value held in the counter 205 is written to a column memory 209 via a vertical signal line 207.

[0048] As described above, the vertical scanning unit 208 sequentially selects rows of pixels arranged in a matrix and outputs a signal pulse sel to each row. The vertical scanning unit 208 also outputs a signal pulse res that controls the reset operation of the count value of the counter 205 and the start timing of the count operation.

[0049] The count values ​​of each pixel 201 in the row selected by the vertical scanning unit 208 in response to the signal pulse sel are written to the column memory 209 via the vertical signal line 207, and the column memory 209 holds the count values ​​for each column. The vertical scanning unit 208 also sequentially selects the count values ​​of each pixel 201 held in the column memory 209 for each column, thereby sequentially outputting the count values ​​of each pixel 201 to the output unit 210.

[0050] The output unit 210 performs digital processing such as gain processing and signal rearrangement on the count values ​​of each pixel 201 output from the column memory 209 , and outputs the processed image signal to the outside of the image sensor 110 .

[0051] As described above, the SPAD sensor counts photons that enter the pixel 201 and converts the photons into an electric charge as soon as they enter the pixel 201. The SPAD sensor can detect, as a signal, a large electric charge that is generated when the electrons that make up the electric charge undergo avalanche amplification. Because of the way the SPAD sensor works, no noise is introduced during signal readout, so that the subject can be clearly photographed without being affected by noise even in dark places. For this reason, the SPAD sensor is expected to be used widely as a sensor for surveillance purposes, etc.

[0052] The first embodiment has an object to suppress local degradation in image quality even when a strong light source is included in a specific position within a fixed shooting angle of view.

[0053] In the first embodiment, an operation will be described in which the image processing unit 112 estimates the degree of deterioration of the image sensor 110, and the system control unit 120 performs control to change the optical zoom of the imaging lens 101 or to shift the image sensor 110 based on the result.

[0054] 3 is a diagram showing an example of the configuration of a relative position change control parameter acquisition unit 300 between a subject and the image sensor 110. The relative position change control parameter acquisition unit 300 is provided in the image processing unit 112 of FIG. 1. The relative position change control parameter acquisition unit 300 also includes a brightness value acquisition unit 302, a RAM 303, a deterioration degree estimation unit 304, an evaluation value storage unit 305, a control necessity determination unit 306, a control feasibility determination unit 307, a RAM 308, and a control amount calculation unit 309. The relative position change control parameter acquisition unit 300 generates an optical zoom control parameter for the image capture device 100, a shift parameter for the image sensor 110, and the like. The system control unit 120 acquires these parameters. An input image 301 is an image based on an output signal from the image sensor 110.

[0055] The RAM 303 records the accumulated luminance values ​​for each region of the image sensor 110, and updates the accumulated luminance values ​​by accumulating the luminance values ​​for each region newly acquired by the luminance value acquisition unit 302. The deterioration degree estimation unit 304 estimates the deterioration degree of the image sensor 110 based on the accumulated luminance values ​​recorded in the RAM 303.

[0056] Next, evaluation value storage unit 305 calculates and stores an evaluation value based on the results of brightness value acquisition unit 302 and deterioration degree estimation unit 304. Thereafter, control necessity determination unit 306 determines whether or not optical adjustment is possible from the evaluation value in evaluation value storage unit 305, and if possible, control feasibility determination unit 307 acquires the current state of image capture device 100 from RAM 308 and determines whether or not optical adjustment is possible. If optical adjustment is possible, control amount calculation unit 309 calculates a control amount for performing a small amount of optical adjustment and provides the control amount to system control unit 120.

[0057] FIG. 4 is a flowchart showing a processing method of the image processing unit 112 in FIG.

[0058] In step S401, the change degree estimation unit 302 estimates the deterioration degree of the image sensor 110 for each region of the image and records it in the RAM 303. For example, the deterioration degree estimation unit 304 estimates the deterioration degree based on the number of input photons for each region of the image sensor 110. The number of input photons corresponds to a count value output by the image sensor 110. The image processing unit 112 acquires the deterioration degree of the image sensor 110 from the RAM 303.

[0059] 5(A) and 5(B) are diagrams showing an example of the deterioration degree of the image sensor 110 for each region of an image. When the image sensor 100 is fixed at the angle of view of the image shown in FIG. 5(A) and driven constantly, the deterioration degree is estimated by scoring each 8×6 region as shown in FIG. 5(B). Here, the image sensor 110 is divided into 8×6 regions for the sake of simplicity of explanation, but it may be divided into finer regions, or an index of the deterioration degree may be given for each pixel. The score is expressed from 0.0 to 1.0, and the larger the number, the more the deterioration of the image sensor 110 progresses, and the higher the possibility of image quality deterioration occurring. The upper part of the image in FIG. 5(A) has a high score because the number of input photons to the image sensor 110 is always large due to the influence of the sky and sunlight. On the other hand, the lower part of the image in FIG. 5(A) has a low score because the luminance level is low even during the day and the number of input photons to the image sensor 110 is small.

[0060] The deterioration degree estimation unit 304 may estimate the deterioration degree based on the cumulative value of the output signal for each region of the image sensor 110 for each frame.

[0061] In step S402, the image processing unit 112 determines whether or not there is local degradation based on the degradation degree index calculated in step S401. For example, the image processing unit 112 determines that there is local degradation when one or more of the degradation degree indexes for each region are equal to or greater than a threshold, and determines that there is no local degradation when there is no region equal to or greater than the threshold. If the image processing unit 112 determines that there is local degradation, it advances the process to step S403. On the other hand, if the image processing unit 112 determines that there is no local degradation, it ends the process.

[0062] In step S403, the luminance value acquisition unit 404 acquires the luminance value of the input image 301 for each region of the image.

[0063] In step S404, the evaluation value storage unit 305 calculates and stores an evaluation value based on the degradation degree index estimated by the degradation degree estimation unit 304 and the luminance value acquired from the luminance value acquisition unit 404. Fig. 6 is a diagram showing an example of the calculated evaluation value. In Fig. 6, the evaluation value is calculated as the product of the degradation degree for each region and the luminance value for each region. Here, the evaluation value is calculated by simple multiplication, but it may be calculated by weighting, for example.

[0064] In step S405, the optical adjustment necessity determination unit 306 determines whether the luminance value of the deteriorated region is high based on the evaluation value calculated in step S404. For example, when the deterioration degree of each region is in the state shown in FIG. 6, the evaluation value of the region with a deterioration degree of 0.7 or more is evaluated, and if it is 0.5 or more, the luminance value of the deteriorated region is determined to be high, and if it is less than 0.5, the luminance value of the deteriorated region is determined to be low. For example, when the optical adjustment necessity determination unit 306 determines that there is one or more regions that can be determined to have a high luminance value among the deteriorated regions, the process proceeds to step S406. On the other hand, when the optical adjustment necessity determination unit 306 determines that there is no region that can be determined to have a high luminance value among the deteriorated regions, the process ends.

[0065] In step S406, the optical adjustment possibility determination unit 307 determines whether or not optical adjustment is possible based on the current state of the imaging device 100 acquired from the RAM 308. For example, the optical adjustment possibility determination unit 307 determines that optical adjustment is possible when the parameter of the optical zoom is away from the wide end by a predetermined amount or more, or when the image sensor 110 has a predetermined amount or more of surplus pixels. On the other hand, the optical adjustment possibility determination unit 307 determines that optical adjustment is impossible when the parameter of the optical zoom is within a predetermined amount from the wide end and the image sensor 110 has less than a predetermined amount of surplus pixels. When the optical adjustment possibility determination unit 307 determines that optical adjustment is possible, the process proceeds to step S407. On the other hand, when the optical adjustment possibility determination unit 307 determines that optical adjustment is impossible, the process ends.

[0066] In step S407, the control amount calculation unit 309 calculates the amount of minute optical adjustment based on the current state of the imaging device 100, and performs minute optical adjustment. Then, the control amount calculation unit 309 returns the process to step S403. The system control unit 120 executes at least one of zoom control by the imaging lens 101, shift control of the imaging element 110, pan control by the pan tilt roll control unit 190, tilt control, and roll control based on the amount of minute optical adjustment.

[0067] Figures 7(A) and 7(B) are diagrams showing an example of the effect of performing a small amount of optical adjustment using optical zoom from the state of Figure 5(A). When the angle of view of the image shown in Figure 5(A) is changed to the wide side by optical zoom, the image shown in Figure 7(A) is obtained as input image 301. At the angle of view shown in Figure 7(A), the luminance value acquired by luminance value acquisition unit 302 changes from that of Figure 5(B) as shown in Figure 7(B).

[0068] The above process is repeated until the optical adjustment necessity determining unit 306 determines that there is no area that is determined to have a high luminance value among the degraded areas, and then determines the final amount of optical adjustment and changes the state of the imaging device 100.

[0069] As described above, according to the first embodiment, even when a strong light source is included in a specific position within a fixed shooting angle of view, local degradation in image quality can be suppressed.

[0070] In the first embodiment, the operation of suppressing local degradation in image quality by changing the angle of view to the wide side using optical zoom has been described, but image processing may be performed by the image processing unit 112 so that the original angle of view is restored after optical adjustment. In other words, after optical zoom is used, digital zoom processing is performed so as to cancel the zoom amount, thereby suppressing local degradation in image quality without changing the angle of view.

[0071] In addition, in the first embodiment, the optical adjustment necessity determination unit 306 repeated the process until there were no areas among the deteriorated areas that were determined to have a high brightness value, but it is also possible to repeat the process a predetermined number of times and determine, as the final parameter, the amount of optical adjustment that optimizes the evaluation value stored in the evaluation value storage unit 305 from among the results.

[0072] In the first embodiment, an operation is described in which a minute amount of optical adjustment is calculated based on the current state of the imaging device 100. However, if an appropriate amount of adjustment can be estimated from the evaluation value in the evaluation value storage unit 305, optical adjustment may be performed using that amount of adjustment before processing is started.

[0073] <Second embodiment> Next, a second embodiment of the present invention will be described. Note that the description of the same components as those of the first embodiment will be omitted. In the first embodiment, processing is performed when the local deterioration of the image sensor 110 has progressed by a predetermined amount, so that the influence of the deterioration of image quality is not noticeable after the deterioration, and the deterioration cannot be prevented in advance. Therefore, in the second embodiment, an operation in the case where processing is performed to prevent the local deterioration of the image sensor 110 in advance based on the result of the brightness value acquisition unit 302 will be described.

[0074] FIG. 8 is a flowchart showing a processing method of the image processing unit 112 according to the second embodiment.

[0075] In step S801, the image processing unit 112 performs the same process as in step S403 in FIG.

[0076] In step S802, the image processing unit 112 determines whether or not there is a region with a high luminance value based on the luminance values ​​acquired in step S801. For example, the image processing unit 112 determines that there is a region with a high luminance value when one or more of the luminance values ​​for each region is equal to or greater than a threshold value, and determines that there is no region with a high luminance value when there is no region with a luminance value equal to or greater than the threshold value. If the image processing unit 112 determines that there is a region with a high luminance value, it advances the process to step S803. On the other hand, if the image processing unit 112 determines that there is no region with a high luminance value, it ends the process.

[0077] In step S803, the image processing unit 112 performs the same process as in step S401 in FIG.

[0078] In step S804, the image processing unit 112 performs the same process as in step S404 in FIG.

[0079] In step S805, the optical adjustment necessity determination unit 306 determines whether the deterioration degree of the region with a high luminance value is high based on the evaluation value calculated in step S804. For example, when there is one or more regions determined to have a high deterioration degree among the regions with a high luminance value, the optical adjustment necessity determination unit 306 determines that the deterioration degree of the region with a high luminance value is high. Also, for example, when there is no region determined to have a high deterioration degree among the regions with a high luminance value, the optical adjustment necessity determination unit 306 determines that the deterioration degree of the region with a high luminance value is not high. When the optical adjustment necessity determination unit 306 determines that the deterioration degree of the region with a high luminance value is high, the optical adjustment necessity determination unit 306 advances the process to step S806. On the other hand, when the optical adjustment necessity determination unit 306 determines that the deterioration degree of the region with a high luminance value is not high, the optical adjustment necessity determination unit 306 ends the process.

[0080] In step S806, the image processing unit 112 performs the same process as in step S406 in FIG.

[0081] In step S807, the image processing unit 112 performs the same process as in step S407 in FIG.

[0082] The above process is repeated until the optical adjustment necessity determining unit 306 determines that there is no area that is determined to have a high luminance value among the degraded areas, and then determines the final amount of optical adjustment and changes the state of the imaging device 100.

[0083] As described above, according to the second embodiment, even when a strong light source is included in a specific position within a fixed shooting angle of view, local degradation of image quality can be prevented.

[0084] The present invention includes the following inventions that appropriately combine the above-mentioned features.

[0085] (Configuration 1) an imaging device comprising: an imaging element; a mechanism capable of changing a relative position between the imaging element and an image of a subject; a deterioration degree estimation means for estimating a degree of deterioration over time for each region of the imaging element; a brightness value acquisition means for acquiring a brightness value of a screen corresponding to the region where the degree exceeds a predetermined threshold; and a system control means for, when the brightness value exceeds a predetermined brightness value in a brightness value distribution over the entire screen corresponding to the imaging element, executing a search process by controlling the mechanism to change the relative position between the imaging element and the image of the subject and searching for an imaging range where the brightness value is equal to or less than the predetermined brightness value.

[0086] (Configuration 2) 2. The imaging device according to configuration 1, wherein the mechanism controls at least one of an optical zoom, a lens shift, a shift of the imaging element, an imaging direction of the imaging device, and a roll of the imaging device.

[0087] (Configuration 3) The imaging device according to configuration 1 or 2, further comprising an evaluation value storage means for storing an evaluation value based on the luminance value when the imaging range is searched and a degree of deterioration of the area, wherein when the search process is executed but an imaging range in which the luminance value is equal to or less than a predetermined luminance value cannot be found, the system control means controls the mechanism to find an imaging range in which the search process is executed and the evaluation value is smallest.

[0088] (Configuration 4) 4. The imaging device according to configuration 3, wherein the evaluation value storage means stores a product of the luminance value and the degree of deterioration calculated for each of the regions as the evaluation value.

[0089] (Configuration 5) 5. The imaging device according to configuration 3 or 4, wherein the evaluation value storage means stores a total or average of the stored evaluation values ​​for an arbitrary period.

[0090] (Configuration 6) The imaging device according to any one of configurations 1 to 5, further comprising an image processing means that is capable of executing at least one of changing a cropping range of an image generated by the imaging element, transforming the image, and rotating the image as image processing, and that executes the image processing so that a difference between a range shown in the image before the mechanism is controlled by the system control means and a range shown in the image after the mechanism is controlled by the system control means falls within a predetermined range.

[0091] (Configuration 7) 7. The imaging apparatus according to configuration 6, wherein the control amount calculation means limits the control amount of the mechanism based on a correction amount that can be processed by the image processing.

[0092] (Configuration 8) 8. The imaging device according to claim 6, wherein the image processing means cancels the image processing when the mechanism is controlled by a user.

[0093] (Configuration 9) The imaging device according to any one of configurations 6 to 8, further comprising a notification means for notifying a user that a range shown in the image has changed as a result of the mechanism being controlled by the system control means.

[0094] (Configuration 10) 10. The imaging device according to any one of configurations 6 to 9, further comprising a notification unit that notifies a user that the image quality of the image has changed due to the image processing.

[0095] (Configuration 11) 11. The imaging device according to any one of configurations 1 to 10, wherein the system control means issues a warning that the imaging range is changing while the search process is being performed.

[0096] (Configuration 12) The imaging device according to any one of configurations 1 to 11, characterized in that, when the search process is executed but an imaging range in which the luminance value is equal to or less than a predetermined luminance value cannot be found, the system control means controls the mechanism to find an imaging range in which the luminance value is smallest within the range in which the search process is executed.

[0097] (Method 1) a method for estimating a degree of deterioration over time for each region of an image sensor, acquiring a luminance value of a screen corresponding to the region where the degree exceeds a predetermined threshold, and if the luminance value exceeds a predetermined luminance value in a luminance value distribution over the entire screen corresponding to the image sensor, performing a search process by controlling the mechanism to search for an image sensing range in which the luminance value is equal to or less than the predetermined luminance value while changing a relative position between the image sensor and the subject image.

[0098] (Program 1) causing a computer to execute a process for estimating a degree of deterioration over time for each region of the image sensor; a program for executing a process for acquiring a luminance value of a screen corresponding to the area where the degree exceeds a predetermined threshold, and if the luminance value exceeds a predetermined luminance value in a luminance value distribution over the entire screen corresponding to the imaging element, executing a search process by controlling the mechanism to search for an imaging range in which the luminance value is equal to or less than the predetermined luminance value while changing a relative position between the imaging element and the subject image.

[0099] <Other embodiments> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a recording medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit for implementing one or more of the functions, such as an ASIC (Application Specific Integrated Circuit).

[0100] The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments. In other words, the present invention includes embodiments in which various modifications have been made based on the gist of the present invention, and these embodiments are not excluded from the scope of the present invention. [Explanation of symbols]

[0101] 112: Image processing unit 120: System control unit 160:External display device 302: Brightness value acquisition unit 304: Deterioration degree estimation unit

Claims

1. An imaging element; a mechanism capable of changing a relative position between the image sensor and a subject image; a deterioration degree estimation means for estimating a degree of deterioration over time for each region of the image sensor; a brightness value acquiring means for acquiring a brightness value of a screen corresponding to the region where the degree exceeds a predetermined threshold; a system control means for executing a search process for searching for an imaging range in which the luminance value is equal to or less than the predetermined luminance value while changing a relative position between the imaging element and the subject image by controlling the mechanism when the luminance value exceeds a predetermined luminance value in a luminance value distribution over the entire screen corresponding to the imaging element; An imaging device comprising:

2. the mechanism controls at least one of an optical zoom, a lens shift, a shift of the image sensor, an imaging direction of the image capture device, and a roll of the image capture device; 2. The imaging device according to claim 1 .

3. an evaluation value storage means for storing an evaluation value based on the luminance value when the imaging range is being searched and a deterioration degree of the area; When the system control means cannot find an imaging range in which the luminance value is equal to or less than a predetermined luminance value even after the search process is executed, the system control means controls the mechanism to find an imaging range in which the evaluation value is minimum within the range in which the search process is executed.

3. The imaging device according to claim 1 or 2.

4. the evaluation value storage means stores a product of the luminance value and the deterioration degree calculated for each of the regions as the evaluation value; 4. The imaging device according to claim 3.

5. The evaluation value storage means stores a total or average of the stored evaluation values ​​for an arbitrary period of time.

4. The imaging device according to claim 3.

6. the image processing device is further provided with an image processing means that is capable of executing at least one of changing a cutout range of an image generated by the imaging element, transforming the image, and rotating the image as image processing, and that executes the image processing so that a difference between a range shown in the image before the mechanism is controlled by the system control means and a range shown in the image after the mechanism is controlled by the system control means falls within a predetermined range; 3. The imaging device according to claim 1 or 2.

7. the control amount calculation means limits the control amount of the mechanism based on a correction amount that can be processed by the image processing; 7. The imaging device according to claim 6.

8. the image processing means cancels the image processing when the mechanism is controlled by a user; 7. The imaging device according to claim 6.

9. a notification unit that notifies a user that a range shown in the image has changed due to the mechanism being controlled by the system control unit, 7. The imaging device according to claim 6.

10. a notification unit for notifying a user that the image quality of the image has been changed by the image processing, 7. The imaging device according to claim 6.

11. the system control means issues a warning that the imaging range is changing while the search process is being performed.

3. The imaging device according to claim 1 or 2.

12. When the system control means cannot find an imaging range in which the luminance value is equal to or less than a predetermined luminance value even after the search process is executed, the system control means controls the mechanism so as to find an imaging range in which the luminance value is minimum within the range in which the search process is executed.

3. The imaging device according to claim 1 or 2.

13. Estimate the degree of deterioration over time for each area of ​​the image sensor, obtaining a luminance value of the screen corresponding to the region in which the degree exceeds a predetermined threshold; when the luminance value exceeds a predetermined luminance value in a luminance value distribution over the entire screen corresponding to the imaging element, a search process is executed to search for an imaging range in which the luminance value is equal to or less than the predetermined luminance value while changing a relative position between the imaging element and the subject image by controlling the mechanism; A method comprising:

14. On the computer, Executing a process for estimating the degree of deterioration over time for each region of the image sensor; executing a process of acquiring a luminance value of a screen corresponding to the region in which the degree exceeds a predetermined threshold; when the luminance value exceeds a predetermined luminance value in a luminance value distribution over the entire screen corresponding to the image sensor, executing a search process for searching for an image capturing range in which the luminance value is equal to or less than the predetermined luminance value while changing a relative position between the image sensor and the subject image by controlling the mechanism; A program characterized by:

Images