Correction device, correction method, and program

Abstract

To reduce the loss of color information while reducing fluctuation.SOLUTION: The correction apparatus acquires fluctuation information related to fluctuations in the input image, and selects a wavelength band related to generation of the output image in accordance with the fluctuation information. Then, the correction device performs predetermined image processing on the input image based on the selected wavelength band to generate an output image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a correction technique for acquiring an image with reduced fluctuation. [Background technology]

[0002] In surveillance camera use cases such as port surveillance and infrastructure monitoring, for example, when taking telephoto shots of ships or aircraft, the visibility of the subject is likely to be reduced due to fluctuations in the subject image caused by uneven changes in the refractive index of the atmosphere (such as heat haze). The refractive index of the atmosphere depends on the wavelength of light and air conditions such as temperature, air pressure, humidity, and carbon dioxide concentration. For example, when the air conditions are constant, the refractive index of the atmosphere changes with the wavelength of light. Specifically, when the air conditions are the same, the shorter the wavelength of light, the greater the refractive index of the atmosphere, making it more susceptible to fluctuations.

[0003] A known method for reducing the effects of such fluctuations is to smooth multiple images captured consecutively in chronological order in the time direction. However, this method has the problem that, when a moving object is included in the subject, blur occurs in the image portion of the moving object. In response to this problem, Patent Document 1 discloses a technology that suppresses blur of moving objects while mitigating the deterioration of image visibility due to fluctuations by selectively inserting either an infrared light blocking filter or a visible light blocking filter between the subject and the image sensor depending on the presence or absence of fluctuations. Known visible light blocking filters include visible light cut filters that block wavelengths shorter than a certain wavelength band and transmit long wavelength bands, and bandpass filters that transmit only a specific wavelength band. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-90152 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the conventional technology disclosed in the above-mentioned patent document, when visible light is blocked, the output image is displayed in black and white. Also, depending on the infrared reflection characteristics of the subject, color difference information may be lost, which may result in the loss of text information. Thus, while the conventional technology disclosed in the patent document can reduce fluctuations, it may also result in a significant loss of color information.

[0006] Therefore, an object of the present invention is to make it possible to reduce the loss of color information while reducing fluctuations. [Means for solving the problem]

[0007] The correction device of the present invention is characterized by having a fluctuation information acquisition means for acquiring fluctuation information related to fluctuations in an input image, a band selection means for selecting a wavelength band related to generation of an output image in accordance with the fluctuation information, and an image processing means for performing a predetermined image processing on the input image based on the selected wavelength band to generate the output image. [Effects of the Invention]

[0008] According to the present invention, it is possible to reduce the loss of color information while reducing fluctuations. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of the configuration of an imaging device including a correction function according to a first embodiment. [Figure 2] 10 is a flowchart showing the flow of correction processing according to the first embodiment. [Figure 3] 1A and 1B are diagrams illustrating an example of an input image and an output image. [Figure 4] FIG. 2 is a diagram illustrating the spectral sensitivity characteristics of an image sensor. [Figure 5] FIG. 10 is a diagram illustrating an example of a GUI. [Figure 6] FIG. 10 is a diagram illustrating an example of the configuration of an imaging device including a correction function according to a second embodiment. [Figure 7] FIG. 2 is an explanatory diagram of the optical characteristics of an infrared cut filter. [Figure 8] 10 is a flowchart showing the flow of correction processing according to the second embodiment. [Figure 9] FIG. 2 is an explanatory diagram of the optical characteristics of a visible light cut filter. [Figure 10] FIG. 2 is an explanatory diagram of the optical characteristics of a bandpass filter. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in each embodiment are necessarily essential to the solution of the present invention. The configuration of each embodiment may be appropriately modified or changed depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). Furthermore, a configuration may be achieved by appropriately combining parts of each embodiment described below. In each of the following embodiments, the same or similar configurations and processing steps are designated by the same reference symbols, and redundant explanations will be omitted.

[0011] In this embodiment, as an example, a case where a surveillance camera is used for port surveillance, infrastructure surveillance, etc., and telephoto photography of a ship, an aircraft, etc. is described. In this embodiment, a correction device is described that performs appropriate correction depending on the scene for a phenomenon in which the visibility of a subject is reduced due to fluctuations in the subject image caused by uneven changes in the refractive index of the atmosphere (such as heat haze).

[0012] First Embodiment FIG. 1A is a block diagram showing an example of the configuration of an imaging device including the functions of a correction device according to this embodiment. The imaging device shown in FIG. 1( a ) includes an imaging optical system 101 , an imaging element 102 , a CPU 103 , a RAM 104 , and a ROM 105 , and these components are electrically connected via a bus 106 .

[0013] The imaging optical system 101 includes a lens group consisting of one or more lenses, and focuses incident light onto the imaging surface of the image sensor 102. The lenses of the imaging optical system 101 include lenses with coatings that change the transmittance of infrared light components, lenses that suppress the effects of aberration, lenses that change optical characteristics such as teleconverters, and lenses with different functions for different wavelengths such as metalenses. The imaging optical system 101 may be an optical system built into the imaging device, or an interchangeable lens that can be attached to and detached from the imaging device. This embodiment uses a surveillance camera for port surveillance and infrastructure monitoring as an example, and therefore the imaging optical system 101 includes a lens group that can capture telephoto images of ships, aircraft, and the like. Note that, as mentioned above, when capturing telephoto images of ships, aircraft, and the like, the visibility of the subject is likely to be reduced due to fluctuations in the subject image caused by uneven changes in the refractive index of the atmosphere (such as heat haze).

[0014] The imaging element 102 captures an optical image of a subject or the like formed on an imaging surface by the imaging optical system 101. Specifically, the imaging element 102 generates an image signal consisting of a plurality of pixel signals obtained by digitally converting electrical signals obtained by a plurality of photoelectric conversion elements arranged corresponding to each pixel. Here, color filters corresponding to red, green, and blue are provided in front of the imaging surface, and the imaging element 102 captures an optical image transmitted through these color filters. Therefore, the imaging element 102 outputs an image signal including red, green, and blue signals. Note that the color filters transmit not only red, green, and blue visible light components but also some infrared light components included in the invisible light range. Examples of the imaging element 102 include a complementary metal oxide semiconductor (CMOS) and a charge-coupled device (CCD). Alternatively, a single photon avalanche diode (SPAD) or the like may be used for the imaging element 102. In this embodiment, the use case of a surveillance camera for port surveillance, infrastructure surveillance, etc. is taken as an example, and therefore the image sensor 102 outputs images for each frame captured in chronological order, that is, image signals for a moving image.

[0015] The CPU 103 is a central processing unit for overall control of the imaging device. The RAM 104 is a non-volatile memory that provides a work area used by the CPU 103 when it executes processing. The RAM 104 also functions as a frame memory and a buffer memory. The ROM 105 stores programs for the CPU 103 to control the imaging device, image data, and the like. The programs stored in the ROM 105 include a program for the CPU 103 to execute the correction processing according to this embodiment. The ROM 105 also stores other data used in the correction processing according to this embodiment. For example, it also stores several threshold values ​​related to fluctuations, which will be described later.

[0016] 1(b) is a block diagram showing the functional configuration of a correction device included in the imaging device of this embodiment. The correction device of this embodiment acquires fluctuation information based on the captured image and selects a wavelength band related to image generation based on the fluctuation information, thereby making it possible to generate an output image in which fluctuations in the image are reduced while preserving color components (color information) as much as possible.

[0017] The correction device of this embodiment includes functional units, namely, an image acquisition unit 111, a fluctuation information acquisition unit 112, a band selection unit 113, an image processing unit 114, and an image output unit 115. In this embodiment, these functional units are realized by reading a correction program according to this embodiment stored in a ROM 105 into a RAM 104 and executing the program by a CPU 103. Note that these functional units may also be realized by a hardware configuration such as a circuit.

[0018] The image acquisition unit 111 acquires, as an input image, an image signal (so-called RAW image) output from the image sensor 102. Note that the image acquisition unit 111 may acquire, as an input image, an image that was captured and recorded in the past, an image that was captured by another image capture device, or an image that was acquired via a network or the like.

[0019] The image processing unit 114 performs image processing on the image signal output from the image sensor 102 to set appropriate values ​​for exposure, brightness, and color, and to correct various aberrations that occur in the imaging optical system 101. In this embodiment, the image processing unit 114 also performs processing to smooth multiple chronologically consecutive images in the time direction to correct fluctuations, and image processing corresponding to the wavelength band selected by the band selection unit 113, which will be described later.

[0020] The fluctuation information acquisition unit 112 acquires fluctuation information relating to fluctuation from the input image. In this embodiment, the fluctuation information acquisition unit 112 acquires fluctuation information based on the input image acquired by the image acquisition unit 111. For example, the fluctuation information acquisition unit 112 acquires fluctuation information representing the magnitude of fluctuation based on the magnitude of the pixel value difference of a pixel of interest between chronologically consecutive input images. In this case, the fluctuation information acquisition unit 112 sets the edge of a still subject in the input image as the pixel of interest, calculates the pixel value difference of the pixel of interest between chronologically consecutive input images as the amount of change, and acquires this amount of change as the amount of fluctuation (fluctuation information). In other words, the fluctuation information acquisition unit 112 acquires fluctuation information representing the amount of fluctuation such that the value is large when the amount of change of the pixel of interest between chronologically consecutive input images is large, and the value is small when there is almost no change.

[0021] In the above example, the pixel value difference of a pixel of interest between successive images is used as the amount of fluctuation, but the present invention is not limited to this. For example, the fluctuation information acquisition unit 112 may calculate the number of frames when the cumulative value of inter-frame differences of chronologically successive input images becomes equal to or greater than a predetermined value as the fluctuation period, and acquire this fluctuation period as the amount of fluctuation. Since the shorter the fluctuation period (i.e., the fewer the number of frames when the cumulative value of inter-frame differences becomes equal to or greater than a predetermined value), the greater the degree of fluctuation, the fluctuation information acquisition unit 112 acquires this fluctuation period as fluctuation information representing the amount of fluctuation.

[0022] Furthermore, for example, the fluctuation information acquisition unit 112 may acquire fluctuation information based on values ​​used in image processing by the image processing unit 114. For example, the fluctuation information acquisition unit 112 may acquire, as fluctuation information, a correction value (correction intensity) when the image processing unit 114 performs processing to smooth multiple chronologically consecutive images in the time direction and correct fluctuations. Furthermore, for example, the fluctuation information acquisition unit 112 may calculate the proportion of the area in the input image where fluctuation occurs as the fluctuation occurrence probability, and acquire the fluctuation occurrence probability as fluctuation information representing the amount of fluctuation. Furthermore, for example, the fluctuation information may include information such as a notification when fluctuation correction is disabled based on at least one of the passage of time and the environment, or a notification when fluctuation correction may be disabled based on the results of an estimation regarding fluctuation, such as illuminance estimation.

[0023] Alternatively, the fluctuation information acquisition unit 112 may acquire fluctuation information based on information input by the user via a GUI (Graphical User Interface) screen, which will be described later. Information input by the user via the GUI screen may include information in which the user directly specifies the magnitude of fluctuation, or information in which the user specifies whether to prioritize fluctuation correction or color information reproduction. Details of the GUI screen will be described later.

[0024] The band selection unit 113 selects a wavelength band related to image generation when an output image is generated in a subsequent image processing unit 114 from the input image acquired by the image acquisition unit 111, according to the fluctuation information acquired by the fluctuation information acquisition unit 112. In this embodiment, the wavelength band related to the generation of the output image is a wavelength band to which the image sensor 102 has sensitivity, and is a band ranging from the visible light band (around 400 nm to 750 nm) to the non-visible light band (around 750 nm to 1000 nm).

[0025] For example, when the amount of fluctuation indicated by the fluctuation information is equal to or greater than a predetermined threshold, the band selection unit 113 selects a long wavelength band excluding many of the short wavelength bands that are susceptible to fluctuation. As will be described in detail later, when the amount of fluctuation is large, the band selection unit 113 selects the wavelength band of the red signal, excluding the blue signal and the green signal on the short wavelength band side that are susceptible to fluctuation, from among the red, green, and blue signals that constitute the input image. In other words, when the amount of fluctuation is large, the band selection unit 113 selects the wavelength band of the red signal in the long wavelength band that is least susceptible to fluctuation from the wavelength band of the input image. Similarly, as will be described in detail later, when the amount of fluctuation is less than a predetermined threshold, the band selection unit 113 selects the remaining wavelength band of the input image excluding some of the short wavelength bands. In other words, when the amount of fluctuation is small, the band selection unit 113 selects the wavelength bands of the green and red signals from the wavelength band of the input image, excluding the blue signal in the short wavelength band that is most susceptible to fluctuation.

[0026] Alternatively, the band selection unit 113 may select a wavelength band by changing the signal ratio of each of the red, green, and blue signals that make up the input image. For example, the band selection unit 113 may select a wavelength band related to generation of an output image by changing the signal ratio, such as changing the use of a blue signal, which is a short wavelength band, from 100% to 50%.

[0027] The image output unit 115 outputs the image after image processing by the image processing unit 114 based on the wavelength band selected by the band selection unit 113 as an output image. Here, the image obtained after wavelength band selection by the band selection unit 113 is an image from which wavelength bands such as blue and green signals have been removed, as described above, and therefore has colors that differ from the colors of the original input image. For this reason, the image processing unit 114 performs processing on the image obtained after wavelength band selection by the band selection unit 113 to bring the colors that differ from those of the original input image closer to the color reproduction of the original input image. Furthermore, when wavelength band selection is performed by the band selection unit 113, signal components are reduced, which may result in the image being darker than the original input image. For this reason, the image processing unit 114 also performs processing to correct the exposure value and luminance corresponding to the amount of darkness caused by the reduced signal components. The image output unit 115 outputs the image obtained after the image processing by the image processing unit 114.

[0028] The image processing unit 114 can also generate an output image corresponding to the case where no wavelength band has been selected by the band selection unit 113. For example, when there is no fluctuation and therefore no need to select a wavelength band, or when the user compares both an input image and an image after selecting a wavelength band on the operation screen of the imaging device, the image processing unit 114 may output an image corresponding to the input image.

[0029] 1(b) shows an example of the functional configuration of the correction device in which the fluctuation information acquisition unit 112, the band selection unit 113, and the image processing unit 114 are each separate functional units, but these may be implemented together as a single functional unit. Also, the image processing unit 114 may include both the functions of the fluctuation information acquisition unit 112 and the band selection unit 113. Also, the image acquisition unit 111 and the image processing unit 114 may be implemented together as a single functional unit, or the image processing unit 114 and the image output unit 115 may be implemented together as a single function. That is, each functional unit shown in FIG. 1(b) is realized by executing a program on the CPU 103 (or an image processing engine not shown), and therefore each function can be integrated or divided as appropriate.

[0030] 2 is a flowchart showing the flow of correction processing performed by the correction device included in the imaging device of this embodiment. Note that each processing step shown in this flowchart is realized by, for example, the CPU 103 executing a correction processing program stored in the ROM 105.

[0031] First, in the process of step S201, the fluctuation information acquisition unit 112 acquires fluctuation information from an image. Here, an example will be described in which the fluctuation information acquisition unit 112 acquires fluctuation information based on an input image acquired by the image acquisition unit 111. Based on the input images acquired in chronological order by the image acquisition unit 111, the fluctuation information acquisition unit 112 acquires, as fluctuation information, a fluctuation amount indicating the magnitude of fluctuation in the input image.

[0032] FIG. 3 shows an example of an image of a scene with fluctuation and an example of an image resulting from performing fluctuation correction on the image of that scene. Image 301 in FIG. 3 is an example of an input image of a scene with fluctuation before fluctuation correction is performed. Image 301 shows an example in which fluctuation occurs in the edges of a building, a sign on top of the building, and the word "advertisement" on the sign due to influences such as the refractive index of the atmosphere, causing what are essentially straight lines to appear distorted. The distorted portions in the image due to this fluctuation change sequentially in each frame of the image in chronological order. The fluctuation information acquisition unit 112 acquires the pixel value difference of the pixel of interest between the input images in chronological order as fluctuation information indicating the amount of fluctuation. After acquiring the fluctuation information in step S201, the processing of the fluctuation information acquisition unit 112 proceeds to step S202.

[0033] Proceeding to step S202, the fluctuation information acquisition unit 112 determines whether the fluctuation amount acquired as fluctuation information in step S201 exceeds a predetermined amount threshold. If the fluctuation amount does not exceed the predetermined amount threshold (if it is equal to or less than the predetermined amount threshold), the fluctuation information acquisition unit 112 determines that fluctuation has not occurred and proceeds to step 7. On the other hand, if the fluctuation amount exceeds the predetermined amount threshold, the fluctuation information acquisition unit 112 determines that fluctuation has occurred and proceeds to step 3.

[0034] When the process proceeds to step S203, the fluctuation information acquisition unit 112 performs classification according to the strength of fluctuation based on the fluctuation information (amount of fluctuation) acquired in step S201. In this embodiment, an example is given in which the fluctuation is classified into three levels: "weak" fluctuation, "medium" fluctuation, and "strong" fluctuation. The fluctuation information acquisition unit 112 classifies the strength of fluctuation based on a comparison between two different classification thresholds that are set in advance for classification and the amount of fluctuation. In this example, the two classification thresholds for classification are set in advance: a first classification threshold that distinguishes between "weak" fluctuation and "medium" fluctuation, and a second classification threshold that distinguishes between "medium" fluctuation and "strong" fluctuation. The fluctuation information acquisition unit 112 compares the fluctuation amount acquired in step S201 with the first and second classification thresholds to classify the current fluctuation strength as "weak," "medium," or "strong." For example, if the fluctuation amount acquired by the fluctuation information acquisition unit 112 is less than the first classification threshold, the fluctuation information acquisition unit 112 classifies the fluctuation strength as "weak," and if it is equal to or greater than the first classification threshold and less than the second classification threshold, the fluctuation information acquisition unit 112 classifies the fluctuation strength as "medium." Furthermore, for example, if the fluctuation amount acquired by the fluctuation information acquisition unit 112 is equal to or greater than the second classification threshold, the fluctuation information acquisition unit 112 classifies the fluctuation strength as "strong." Then, the processing of the correction device proceeds to step S204 if the strength of the fluctuation based on the amount of fluctuation is "weak", to step S205 if the fluctuation is "medium", and to step S206 if the fluctuation is "strong".

[0035] When the process proceeds to step S204, the band selection unit 113 selects, from the wavelength bands of the input image, a wavelength band related to the generation of an output image corresponding to the case where the fluctuation strength is "weak." That is, when the fluctuation is "weak," the band selection unit 113 selects, from the wavelength bands of the input image, a wavelength band other than a portion of the short wavelength band that is most susceptible to the influence of fluctuation. In other words, when the fluctuation is "weak," the band selection unit 113 selects, from the wavelength bands of the input image, a wavelength band excluding a portion of the short wavelength band that is most susceptible to the influence of fluctuation. Specifically, from the red, green, and blue signals that constitute the input image, the band selection unit 113 selects the wavelength bands of the green and red signals, excluding the blue signal, which is in the short wavelength band that is most susceptible to the influence of fluctuation.

[0036] 4 is a diagram showing an example of the spectral sensitivity characteristics of the image sensor 102 of the image sensor according to this embodiment with respect to light wavelength, with the vertical axis representing the spectral sensitivity of the image sensor 102 and the horizontal axis representing the wavelength of light received by the image sensor 102 in the range of 400 nm to 1000 nm. In the diagram, a solid spectral curve 401 represents the spectral sensitivity of a red pixel having a peak at a wavelength of approximately 630 nm. Furthermore, a dashed spectral curve 402 represents the spectral sensitivity of a green pixel having a peak at a wavelength of approximately 530 nm, and a dotted spectral curve 403 represents the spectral sensitivity of a blue pixel having a peak at a wavelength of approximately 450 nm.

[0037] In step S204, if the fluctuation is "weak," the band selection unit 113 selects from the wavelength bands of the input image a wavelength band excluding the spectral curve 403 of the blue pixel, which has a peak near a wavelength of 450 nm. That is, if the fluctuation is "weak," the band selection unit 113 selects the wavelength band indicated by the spectral curve 401 of the red pixel, which has a peak near a wavelength of 630 nm, and the wavelength band indicated by the spectral curve 402 of the green pixel, which has a peak near a wavelength of 530 nm. Thus, when proceeding to step S204, the band selection unit 113 selects, as the wavelength band related to generation of the output image, the wavelength band obtained by excluding the blue signal based on the spectral curve 403 on the short wavelength side from the image signals acquired by the image sensor 102. That is, the band selection unit 113 selects, as the wavelength band related to generation of the output image, the red signal based on the spectral curve 401 and the green signal based on the spectral curve 402 from the image signals acquired by the image sensor 102. Once the wavelength band is selected in step S204, the process of the correction device proceeds to step S207, which will be described later.

[0038] As described above, the band selection unit 113 may select the wavelength band by changing the signal ratios of the red, green, and blue signals. That is, in step S204, the band selection unit 113 may reduce the signal ratio of the blue signal based on the spectral curve 403 on the short wavelength side when generating the output image.

[0039] When the process proceeds to step S205, the band selection unit 113 selects, from the wavelength bands of the input image, a wavelength band related to generation of an output image corresponding to a case where the fluctuation strength is "medium." In this embodiment, when the fluctuation is "medium," the band selection unit 113 selects, from the wavelength bands of the input image, wavelength bands other than the short wavelength band and part of the wavelength band of the green signal that are excluded when the fluctuation is "weak."

[0040] Explaining this using the spectral sensitivity characteristics of Fig. 4, when the fluctuation is "medium," the wavelength band selected by the band selection unit 113 from the wavelength bands of the input image is a wavelength band other than the spectral curve 403 of the blue pixel and part of the wavelength band of the spectral curve 402 of the green pixel. In other words, when the fluctuation is "medium," the band selection unit 113 selects the wavelength band excluding part of the short wavelength side of the spectral curve 402 of the green pixel, which has a peak around a wavelength of 530 nm, and the wavelength band indicated by the spectral curve 401 of the red pixel, which has a peak around a wavelength of 630 nm. After the wavelength band selection is performed in step S205, the processing of the correction device proceeds to step S207, which will be described later.

[0041] Note that even when the fluctuation is "medium," the band selection unit 113 may select wavelength bands by changing the ratios of blue, green, and red signals as described above. For example, when the fluctuation is "medium," the band selection unit 113 may reduce the ratio of blue signals based on the spectral curve 403 even more than the signal ratio when the fluctuation is "weak." Furthermore, for example, when the fluctuation is "medium," the band selection unit 113 may, in addition to excluding the wavelength band of blue signals in the short wavelength band, further reduce the signal ratio of green signals based on the spectral curve 402 of green pixels having a peak at a wavelength around 530 nm.

[0042] When the process proceeds to step S206, the band selection unit 113 selects, from the wavelength bands of the input image, a wavelength band related to generation of an output image corresponding to a case where the fluctuation strength is "strong." In this embodiment, when the fluctuation strength is "strong," the band selection unit 113 selects the wavelength band of the red signal from the wavelength bands of the input image, excluding the blue signal and the green signal.

[0043] 4, in the case of "strong" fluctuation, the wavelength band selected by band selection unit 113 is the wavelength band indicated by red pixel spectral curve 401 having a peak at a wavelength near 630 nm, excluding blue pixel spectral curve 403 and green pixel spectral curve 402. Once the wavelength band is selected in step S206, the process of the correction device proceeds to step S207, which will be described later.

[0044] In the case of "strong" fluctuation, the band selection unit 113 may also select wavelength bands by changing the ratios of blue, green, and red signals in the same manner as described above. For example, in the case of "strong" fluctuation, the band selection unit 113 may set the ratio of green signals based on the spectral curve 402 to a ratio even smaller than the signal ratio in the case of "medium" fluctuation. Furthermore, in the case of "strong" fluctuation, the band selection unit 113 may set the ratio of blue signals based on the spectral curve 403 and the ratio of green signals based on the spectral curve 402 to a ratio even smaller than the signal ratio in the case of "medium" fluctuation.

[0045] In this way, the band selection unit 113 selects, as a wavelength band for generating an output image, a wavelength band that can preserve as much color information as possible of the input image while gradually reducing the amount of fluctuation in one of steps S204 to S206. That is, the band selection unit 113 selects a wavelength band that minimizes loss of color information when the amount of fluctuation is small, and selects a wavelength band that further reduces the amount of fluctuation as the amount of fluctuation increases. As the amount of fluctuation increases, color information is gradually lost, but the band selection unit 113 selects a wavelength band that preserves color information even when the amount of fluctuation increases. Note that when it is determined in step S202 that the amount of fluctuation is equal to or less than a predetermined threshold and the process proceeds to step S207, the wavelength band is the wavelength band of the input image. That is, in this case, the band selection unit 113 selects a wavelength band of the input image. In the above example, the fluctuation strength is divided into three categories: "weak," "medium," and "strong." However, the number of categories of fluctuation strength is not limited to three and may be, for example, two categories or four or more categories. In any of these cases, the band selection unit 113 selects wavelength bands so as to retain some color information.

[0046] If the process proceeds to step S207 after step S204, step S205, or step S206, the image processing unit 114 generates an output image based on the wavelength bands selected in steps S204 to S206. For example, if the fluctuation intensity is "weak," the band selection unit 113 selects a wavelength band other than the wavelength band of the blue signal or a wavelength band in which the proportion of the blue signal has been reduced. In this case, however, although the fluctuation in the original input image is reduced, the color of the image will be shifted and the brightness and exposure will also be reduced by the amount of the wavelength band of the blue signal removed or the proportion of the blue signal reduced. Therefore, in step S207, the image processing unit 114 corrects the shifted color and brightness and exposure based on the wavelength band selected according to the amount of fluctuation, and the image output unit 115 outputs the corrected image as an output image.

[0047] For example, if the wavelength band of the blue signal is removed or the ratio of the blue signal is low, the image processing unit 114 may recalculate the white balance gain, for example, to bring the color reproduction closer to that of the original input image. Furthermore, because the luminance is reduced due to the reduction in signal, the image processing unit 114 adds an offset amount corresponding to the amount of signal reduction. Alternatively, the image processing unit 114 may keep the white balance fixed, for example, and increase or decrease the offset amount corresponding to the amount by which the signal ratio has changed, to bring the color reproduction closer to that of the original image. Note that even if the color reproduction is brought closer to that of the original input image, it may not completely match the original colors, and therefore, it may be necessary to restrict the use of other color-related functions.

[0048] As described above, in this embodiment, by selecting a wavelength band related to the generation of an output image according to fluctuation information, fluctuation can be appropriately reduced while preserving color information. Image 303 in FIG. 3 illustrates an example of an output image generated by the correction device of this embodiment for the image with fluctuation exemplified in image 301 based on signals in a wavelength band selected when the fluctuation is "strong." Image 304 in FIG. 3 illustrates an example of an output image generated by the correction device of this embodiment for the image with fluctuation exemplified in image 301 based on signals in a wavelength band selected when the fluctuation is "medium." Image 302 in FIG. 3 illustrates an example of an image obtained by applying a conventional technology that reduces fluctuation by blocking, for example, a visible light wavelength band. For example, conventional technology that reduces fluctuation by blocking a visible light wavelength band can correct strong fluctuation, but results in a black-and-white image as illustrated in image 302, resulting in the loss of color information. In other words, with the conventional technology, for example, the color difference information is lost due to the infrared reflection characteristics of the "advertisement" character and the signboard, making the signboard characters unreadable. In contrast, the correction device of this embodiment can obtain an image in which the fluctuation is reduced to a certain extent while retaining more color information than image 302, such as image 304 in the case of "medium" fluctuation. Also, in image 303 in the case of "strong" fluctuation, not only is the fluctuation reduced favorably, but an image in which more color information is retained than image 302 is obtained. Furthermore, the correction device of this embodiment can also appropriately reduce the fluctuation while retaining more color information, for example, in the case of "weak" fluctuation.

[0049] 5 is a diagram showing an example of a GUI (Graphical User Interface) screen for making various settings for an imaging device (surveillance camera) including the correction device according to this embodiment. This GUI screen is a screen displayed on a monitor mounted on the imaging device or a monitor connected to the imaging device via a communication network by CPU 103 executing a program according to this embodiment.

[0050] In the example GUI screen of FIG. 5, an image 501 is an example image of a scene in which fluctuation occurs before the correction process according to this embodiment is performed, similar to the image 301 of FIG. Furthermore, in the GUI screen of FIG. 5 , a mode selection UI 502 is a UI used when the user selects the priority of fluctuation correction. For example, when the user selects the "fluctuation priority" option in the mode selection UI 502, the CPU 103 performs processing to effectively reduce fluctuation by selecting a wavelength band according to the present embodiment as described above. Note that correcting fluctuations by selecting a wavelength band or the like may result in color shift. Therefore, when "fluctuation priority" is selected, the CPU 103 generates and displays a message 503 to notify the user of the possibility of color shift. On the other hand, if correcting fluctuations by selecting a wavelength band or the like may result in color shift, the user can select an image with less color shift by selecting the "color priority" option. Note that when the user selects the "color priority" option, the CPU 103 may correct fluctuations using a general method of smoothing in the time direction.

[0051] 5, a correction level setting UI 504 is a UI used when the user sets the correction level of fluctuations caused by the correction process according to this embodiment. The correction level setting UI 504 provides fluctuation correction levels from level 0 to level 3. Level 0 indicates "no correction," level 1 indicates a fluctuation correction level corresponding to "weak" fluctuation, level 2 indicates a fluctuation correction level corresponding to "medium" fluctuation, and level 3 indicates a fluctuation correction level corresponding to "strong" fluctuation.

[0052] 5, the color blend ratio UI 505 displays the signal ratio according to the correction level set in the correction level setting UI 504. That is, the color blend ratio UI 505 displays the signal ratio in each wavelength band of the red signal, green signal, and blue signal used to generate the output image. Note that the user can also directly input numerical values ​​into the color blend ratio UI 505. The color blend ratio UI 505 also has a reset button 507, and when the reset button 507 is pressed, the signal ratio returns to the preset ratio. On the GUI screen of FIG. 5, an image 506 is displayed, such as the image 303 or image 304 shown in FIG. 3, which is generated based on the correction level of the correction level setting UI 504 and the color blend ratio of the color blend ratio UI 505.

[0053] <Second embodiment> Next, an imaging device including a correction device according to a second embodiment will be described. In the first embodiment, an example was described in which a wavelength band related to the generation of an output image was selected based on fluctuation information acquired from an input image, specifically, whether or not to use a red signal, a green signal, or a blue signal, or an example in which the signal ratio was changed. That is, in the first embodiment, an example was given in which an output image was generated using a wavelength band selected from an input image. In contrast, in the second embodiment, an example was given in which an imaging device including a correction device has an optical filter whose optical characteristics can be changed, or a lens that can be set to a different function for each wavelength of light. In the second embodiment, an example was described in which a wavelength band related to the generation of an output image is selected by switching the function of the optical filter or lens according to the fluctuation information.

[0054] As in the first embodiment described above, it is possible to reduce fluctuations while preserving color information by changing the use or ratio of the electrical signals that make up the image signal. However, for example, when stronger fluctuations occur, leaving even a small amount of electrical signal on the short wavelength side may leave the effects of the fluctuations. In other words, when strong fluctuations occur, if even a small amount of blue signal on the short wavelength side, which is susceptible to the effects of fluctuations, remains, a certain degree of large fluctuations will remain in the output image. Therefore, in the second embodiment, the imaging device is equipped with an optical filter whose optical characteristics can be changed and a lens that can be set to a different function for each wavelength of light, and by switching between them according to the fluctuation information, the short wavelength side that is susceptible to the influence of fluctuation is cut off at the image capturing stage. In the second embodiment, by switching the functions of the optical filter and lens in stages according to the fluctuation information, it is possible to effectively reduce fluctuation while preserving color information.

[0055] FIG. 6(a) is a block diagram showing an example of the configuration of an imaging device including the functions of a correction device according to a second embodiment. The imaging device shown in FIG. 6(a) includes an optical filter 201 in addition to the imaging optical system 101, image sensor 102, CPU 103, RAM 104, and ROM 105 described above, and these components are electrically connected via a bus 106. FIG. 6(b) is a block diagram showing the functional configuration of a correction device included in the imaging device according to the second embodiment. As shown in FIG. 6(b), the correction device according to the second embodiment includes a filter control unit 211 in addition to the functional units of the image acquisition unit 111, fluctuation information acquisition unit 112, band selection unit 113, image processing unit 114, and image output unit 115 described above. Note that in the configuration of FIG. 6, the same components as those in the first embodiment are denoted by the same reference numerals, and their description will be omitted. Below, components different from those in the first embodiment will be described.

[0056] In the imaging device of the second embodiment, the optical filter 201 includes at least one of an infrared cut filter, a visible light cut filter, and a bandpass filter. The infrared cut filter is an optical filter that attenuates a long wavelength band on the infrared side and transmits a wavelength band on the visible side. The visible light cut filter is an optical filter that attenuates a wavelength band shorter than a certain wavelength band within the visible light wavelength band and transmits a long wavelength band. The bandpass filter is an optical filter that transmits only a specific wavelength band. Note that the specific wavelength band transmitted by the bandpass filter may be a single wavelength band or multiple wavelength bands. These filters included in the optical filter 201 are removably arranged on the optical axis of the imaging optical system 101 of the imaging device and are individually inserted and removed by a driving unit (not shown) under the control of the filter control unit 211. For example, when an infrared cut filter is inserted on the optical axis of the imaging optical system 101, the infrared light component of light incident on the imaging element 102 is attenuated, thereby reducing the influence of the infrared light component on the image signal captured by the imaging element 102.

[0057] In this embodiment, the optical filter 201 may include multiple filters, such as an infrared cut filter, a visible light cut filter, and a bandpass filter, instead of just one. For example, the visible light cut filter may include two or more visible light cut filters that attenuate different wavelength bands. Similarly, the bandpass filter may include two or more bandpass filters that transmit different wavelength bands. Furthermore, the bandpass filter may include a so-called dual bandpass filter that transmits two different wavelength bands. In this way, the optical filter 201 includes, for example, an infrared cut filter, two or more visible light cut filters, and two or more bandpass filters, and is configured to allow these multiple filters to be used in any combination. In this embodiment, the optical filter 201 includes, for example, five filters, including an infrared cut filter, two visible light cut filters that attenuate different wavelength bands, and two bandpass filters that transmit different wavelength bands. Of course, the number of filters is not limited to five, as long as the wavelength band can be switched in a gradual manner by arbitrarily combining these multiple filters.

[0058] The filter control unit 211 controls the insertion and removal of the infrared cut filter, two visible light cut filters, and two bandpass filters included in the optical filter 201, thereby arbitrarily combining the filters and arranging them on the optical axis of the imaging optical system 101. The filter control unit 211 may control the insertion and removal of the filters according to settings specified by a user, or may automatically control the insertion and removal of the filters according to fluctuation information. That is, in the second embodiment, the selection of a wavelength band by the band selection unit 113 is realized by the filter control unit 211 controlling the insertion and removal of the filters included in the optical filter 201.

[0059] 7(a) and 7(b) are diagrams showing an example of the transmittance of an infrared cut filter included in the optical filter 201, where the horizontal axis represents wavelength and the vertical axis represents transmittance. Note that a spectral curve 701 shown by a solid line in FIGS. 7(a) and 7(b) is the spectral sensitivity curve of the image sensor 102. In the case of a typical surveillance camera or similar imaging mode, an infrared cut filter 702 having optical characteristics as indicated by the dashed line in FIG. 7A is inserted on the optical axis. When the infrared cut filter 702 is inserted, the region sensitive to the image sensor 102 is the region indicated by the hatched area 703 in FIG. 7A. On the other hand, when the infrared cut filter 702 is removed, the region sensitive to the image sensor 102 is the region indicated by the hatched area 704 in FIG. 7B. In this embodiment, the infrared cut filter may be used simultaneously with other filters (visible light cut filter and bandpass filter) included in the optical filter 201, or may be removed in preference to these other filters. For example, to reduce strong fluctuations, the user may select a bandpass filter that can best cut the short wavelength band, or the band selection unit 113 may automatically select a filter that can reduce fluctuations while preserving color information according to fluctuation information. In either case, the infrared cut filter may be removed or may be used simultaneously with other filters.

[0060] Fig. 8 is a flowchart of correction processing according to the second embodiment. In the flowchart of Fig. 8, the same processing steps (processing steps) as those in the flowchart of Fig. 2 described above are given the same reference numerals as those in Fig. 2, and their description will be omitted. Below, processing steps that differ from those in the flowchart of Fig. 2 will be described. 2, the process proceeds to step S203, where the fluctuation information acquisition unit 112 performs classification according to the fluctuation strength as described above. In the second embodiment, if the fluctuation strength is "weak," the process proceeds to step S808, if the fluctuation strength is "medium," the process proceeds to step S809, and if the fluctuation strength is "strong," the process proceeds to step S810.

[0061] If the process proceeds to step S808, the band selection unit 113 selects a wavelength band related to generation of an output image for the case of "weak" fluctuation. In the second embodiment, the selection of a wavelength band by the band selection unit 113 is achieved by the filter control unit 211 controlling the insertion and removal of each filter included in the optical filter 201. In this embodiment, in the case of "weak" fluctuation, the band selection unit 113 selects, from the wavelength band of light incident on the imaging device, all but a portion of the short wavelength band as a wavelength band related to generation of an output image. For this reason, the filter control unit 211 controls the insertion, on the optical axis of the imaging optical system 101, of a filter from the optical filter 201 that can transmit all but a portion of the short wavelength band of the wavelength band of light incident on the imaging device.

[0062] In this embodiment, when the fluctuation is "weak," a visible light cut filter is used as a filter that can transmit all but a short wavelength band of the wavelength band of light incident on the imaging device. 9(a) and 9(b) are diagrams showing an example of the transmittance of a visible light cut filter, with the vertical axis representing the transmittance of the filter and the horizontal axis representing the wavelength range of 400 nm to 1000 nm of light received by the image sensor 102. The spectral curve 701 is the spectral sensitivity curve of the image sensor 102, as described above. In this embodiment, two visible light cut filters, a first and a second, which attenuate different wavelength bands, are used as the visible light cut filters. The dashed line in FIG. 9(a) represents the optical characteristics of the first visible light cut filter 905, and the dashed line in FIG. 9(b) represents the optical characteristics of the second visible light cut filter 907. That is, the first visible light cut filter 905 has optical characteristics of attenuating light on the shorter wavelength side than approximately 530 nm and transmitting light on the longer wavelength side. According to the first visible light cut filter 905, the region that is sensitive to the image sensor 102 is the region indicated by the shaded area 906 in Fig. 9(a). The second visible light cut filter 907 is a filter that has optical properties of attenuating light on the shorter wavelength side than about 660 nm and transmitting light on the longer wavelength side. According to the second visible light cut filter 907, the region that is sensitive to the image sensor 102 is the region indicated by the shaded area 908 in Fig. 9(b).

[0063] Here, of these two visible light cut filters, the second visible light cut filter 907, which attenuates the shorter wavelength side more, has a greater effect of reducing fluctuation. On the other hand, the second visible light cut filter 907 transmits less light than the first visible light cut filter 905, resulting in a greater decrease in exposure and luminance and a greater loss of color information. For this reason, when the fluctuation is "weak," the band selection unit 113 selects a wavelength band using the first visible light cut filter 905 to retain as much color information as possible and reduce the decrease in exposure and luminance. In other words, when the fluctuation is "weak," the filter control unit 211 controls the insertion or removal of the first visible light cut filter 905 onto the optical axis of the imaging optical system 101. After the filter control unit 211 controls the insertion or removal of the optical filter in step S808, the processing of the imaging device proceeds to step S207.

[0064] If the process proceeds to step S809, the band selection unit 113 selects a wavelength band related to generation of an output image for the case of "medium" fluctuation. In the second embodiment, the filter control unit 211 controls insertion and removal of each filter included in the optical filter 201 according to the selected wavelength band. In this embodiment, for the case of "medium" fluctuation, the band selection unit 113 selects a wavelength band from the wavelength band of light incident on the imaging device that excludes more wavelength bands than the short wavelength band that is excluded for the case of "weak" fluctuation. Therefore, the filter control unit 211 controls the insertion, on the optical axis of the imaging optical system 101, of a filter from the optical filter 201 that can transmit wavelength bands other than the short wavelength band that is excluded for the case of "weak" fluctuation.

[0065] In this embodiment, if the first visible light cut filter 905 shown in Fig. 9(a) is selected when the fluctuation is "weak," then the second visible light cut filter 907 shown in Fig. 9(b) is selected when the fluctuation is "medium." That is, by selecting the second visible light cut filter 907, an output image is generated based on a wavelength band obtained by removing a wavelength band that is larger than that of the first visible light cut filter 905. After the filter control unit 211 controls the insertion / removal of the optical filter in step S809, the processing of the imaging device proceeds to step S207.

[0066] When the process proceeds to step S810, the band selection unit 113 selects a wavelength band related to generation of an output image for the case of "strong" fluctuation. In the second embodiment, the filter control unit 211 controls insertion and removal of each filter included in the optical filter 201 according to the selected wavelength band. In this embodiment, for the case of "strong" fluctuation, the band selection unit 113 selects a wavelength band that excludes even more wavelength bands from the wavelength bands of light incident on the imaging device than the short wavelength bands that are excluded for the cases of "weak" and "medium" fluctuation. Therefore, the filter control unit 211 controls the insertion, on the optical axis of the imaging optical system 101, of a filter from the optical filter 201 that can transmit light other than those in the cases of "weak" and "medium" fluctuation.

[0067] In this embodiment, as a filter for "strong" fluctuation, for example, a visible light cut filter (not shown) that can cut out an even shorter wavelength band than the second visible light cut filter shown in Fig. 9(b) may be used. However, when a visible light cut filter that can cut out an even shorter wavelength band than the second visible light cut filter is used, there is a possibility that a greater loss of color information will occur than when the second visible light cut filter is used, and that the exposure and brightness will also be further reduced.

[0068] For this reason, in this embodiment, when the fluctuation is "strong," a band-pass filter is used as a filter that can effectively reduce the fluctuation while suppressing the loss of color information and the reduction in exposure and brightness. 10(a) and 10(b) are diagrams showing an example of the transmittance of a bandpass filter, with the vertical axis representing the transmittance of the filter and the horizontal axis representing the wavelength of light received by the image sensor 102. A spectral curve 701 is the spectral sensitivity curve of the image sensor 102, as described above. In this embodiment, two bandpass filters, each transmitting a different wavelength band, are used. The dashed line in FIG. 10(a) represents the optical characteristics of the first bandpass filter 1009, and the two dashed lines in FIG. 10(b) represent the optical characteristics of the second visible light cut filter, which is a dual bandpass filter. In this embodiment, the first bandpass filter 1009 has optical characteristics that allow only light in a wavelength band range around 800 nm to 900 nm to pass through. The first bandpass filter 1009 has a sensitivity range for the image sensor 102 indicated by the shaded area 1010 in FIG. 10(a). The second bandpass filter is a dual bandpass filter including a filter 1011 having optical characteristics that pass a wavelength band range around wavelengths of 800 nm to 900 nm, and a filter 1013 having optical characteristics that pass a wavelength band range around wavelengths of 470 nm to 570 nm. According to the second bandpass filter, the areas that are sensitive to the image sensor 102 are the areas indicated by the hatched areas 1012 and 1014 in FIG. 10(b).

[0069] Here, the first band-pass filter 1009 can more effectively reduce fluctuations. However, using the first band-pass filter 1009 results in greater loss of color information and reductions in exposure and brightness than using the second band-pass filter. Therefore, in this embodiment, the second band-pass filter, which is a dual band-pass filter including a filter 1013 that also transmits a portion of the wavelength band of visible light, is used to preserve as much color information as possible and suppress reductions in exposure and brightness. Note that the first band-pass filter 1009 may be selected, for example, when the user indicates that reduction of fluctuations is prioritized over loss of color information and reductions in exposure and brightness. After the filter control unit 211 controls the insertion / removal of the optical filter in step S810, the process of the imaging device proceeds to step S207.

[0070] In the second embodiment, when the process proceeds to step S207 after step S808, step S809, or step S810, the image processing unit 114 performs processing to generate an output image based on the wavelength bands selected in steps S808 to S810. That is, in the image processing unit 114 of the second embodiment, an output image is generated from an image captured by the image sensor 102 through a filter selected in any one of steps S808, S809, and S810.

[0071] For example, if the visible light cut filter 905 is used in step S808 in response to "weak" fluctuation, the color of the image will shift and the brightness and exposure will also decrease by the amount that the blue component on the short wavelength side is attenuated by the visible light cut filter 905. For this reason, in step S207 of the second embodiment, the image processing unit 114 corrects the color, brightness, and exposure that have shifted due to the optical characteristics of the visible light cut filter 905, and the image output unit 115 outputs the corrected image as an output image.

[0072] For example, the image processing unit 114 recalculates the white balance gain to bring the color reproduction closer to that of the original input image. Furthermore, if the luminance has decreased due to a decrease in the signal, the image processing unit 114 adds an offset amount corresponding to the amount of signal decrease. Alternatively, for example, the white balance may be fixed, and the color reproduction closer to that of the original image may be brought closer by increasing or decreasing the offset amount corresponding to the amount by which the signal ratio has changed. Note that even if the color reproduction closer to that of the original input image is brought closer, it may not completely match the original colors, and therefore, there may be cases in which other color system functions cannot be used.

[0073] In the second embodiment, the optical filter 201 attenuates short wavelength components such as the blue component of incident light, but the image signal output from the image sensor 102 contains a blue signal, and an output image is generated from that image signal. The signal reduction due to the attenuation of short wavelength components by the optical filter 201 can be corrected by adjusting, for example, the white balance gain. That is, according to the second embodiment, the optical filter 201 attenuates components that are susceptible to fluctuations, and then the signal is amplified by gain adjustment, so that the gain adjustment does not increase fluctuations. Therefore, according to the second embodiment, it is possible to further reduce fluctuations while preserving color information, compared to the first embodiment.

[0074] As described above, according to the second embodiment, the optical filter is controlled in stages according to the fluctuation information to cut out the short wavelength side, which is susceptible to the influence of fluctuation, thereby reducing fluctuation while preserving color information.

[0075] In the above-described embodiment, an example in which the correction device is applied to an imaging device has been described. However, the correction device may also be implemented by an information processing device, such as a personal computer or smartphone, connected to the imaging device. In this case, the imaging device outputs to the information processing device (correction device) an RWA image signal captured by the imaging unit, shooting parameters indicating the exposure time, frame rate, exposure setting value, and the like, as well as information indicating the presence or absence of fluctuation influence on the input image. The CPU of the information processing device then executes the correction processing program according to this embodiment using the image signal, shooting parameters, and information indicating the presence or absence of fluctuation influence transmitted from the imaging device, thereby implementing the correction processing described above. Note that even when the correction device is implemented by an information processing device connected to the imaging device, the example in which an optical filter such as that of the second embodiment is mounted on the imaging device can be applied. In this case, information regarding the optical filter of the imaging device such as that of the second embodiment is also transmitted to the information processing device. Note that information regarding the optical filter of the imaging device such as that of the second embodiment may be input by the information processing device or a user of the imaging device.

[0076] In the above-described embodiment, the correction device is applied to a surveillance camera, but the present invention is not limited to this example. The image capture device equipped with the correction device may be any camera that captures images that are susceptible to fluctuations due to the atmosphere, etc. For example, the image capture device may be a general camera with an integrated lens that is capable of telephoto shooting or a camera with interchangeable lenses.

[0077] The present invention can also be realized by providing a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. The above-described embodiments are merely examples of specific embodiments for implementing the present invention, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0078] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) a fluctuation information acquiring means for acquiring fluctuation information relating to fluctuation of an input image; a band selection means for selecting a wavelength band related to generation of an output image in accordance with the fluctuation information; an image processing unit that performs predetermined image processing on the input image based on the selected wavelength band to generate the output image; A correction device comprising: (Configuration 2) 2. The correction device according to configuration 1, wherein the band selection means selects a wavelength band related to generation of the output image from among wavelength bands contained in the input image based on the fluctuation information. (Configuration 3) The correction device according to configuration 1, wherein the band selection means selects a wavelength band related to generation of the output image by changing a ratio of signals for each band within the wavelength band of the input image based on the fluctuation information. (Configuration 4) The correction device according to configuration 1, characterized in that the band selection means selects one or more optical filters from among a plurality of optical filters that transmit or block different wavelength bands from the wavelength band of the incident light based on the fluctuation information, thereby selecting a wavelength band related to generation of the output image. (Configuration 5) 5. The correction device according to configuration 4, wherein the optical filter includes at least one of an infrared cut filter, a visible light cut filter, and a bandpass filter. (Configuration 6) 6. The correction device according to claim 5, wherein the bandpass filter includes a dual bandpass filter that transmits two different wavelength bands. (Configuration 7) The correction device according to configuration 5 or 6, wherein the band selection means selects a wavelength band involved in generating the output image by selecting one visible light cut filter from two or more visible light cut filters that block different wavelength bands, or by selecting one band pass filter from two or more band pass filters that transmit different wavelength bands, based on the fluctuation information. (Configuration 8) 8. The correction device according to any one of configurations 1 to 7, wherein the information acquisition means acquires the fluctuation information representing an amount of fluctuation based on the input image. (Configuration 9) 9. The correction device according to configuration 8, wherein the band selection means selects a wavelength band that is longer in wavelength as the amount of fluctuation increases. (Configuration 10) The correction device according to configuration 8 or 9, wherein the information acquisition means acquires the fluctuation information representing the amount of fluctuation based on a difference in a pixel of interest between the input images that are consecutive in chronological order. (Configuration 11) The correction device according to configuration 8 or 9, wherein the information acquisition means acquires, as the fluctuation information representing the amount of fluctuation, a period of fluctuation corresponding to the number of frames when a cumulative value of differences between frames of the input image that are consecutive in chronological order becomes equal to or greater than a predetermined value. (Configuration 12) The correction device according to configuration 8 or 9, wherein the information acquisition means acquires a proportion of the input image occupied by an area where fluctuation occurs as a fluctuation occurrence probability, and acquires the fluctuation occurrence probability as the fluctuation information representing the fluctuation amount. (Configuration 13) 8. The correction device according to any one of configurations 1 to 7, wherein the information acquisition means acquires the fluctuation information designated by a user through a predetermined user interface. (Configuration 14) The correction device described in configuration 13, characterized in that the fluctuation information specified by the user through a predetermined user interface includes at least one of information on the correction level when correcting the fluctuation and information specifying priority for correcting the fluctuation. (Configuration 15) The correction device according to configuration 1, characterized in that the fluctuation information includes at least one of information for disabling fluctuation correction based on at least one of the passage of time and the environment, and information for disabling fluctuation correction based on an estimated result of illuminance. (Configuration 16) 16. The correction device according to any one of configurations 1 to 15, wherein the predetermined image processing includes processing for correcting the fluctuation using a plurality of the input images in chronological order. (Configuration 17) 17. The correction device according to configuration 16, wherein the information acquisition means acquires the fluctuation information based on a correction strength used in the image processing for correcting the fluctuation. (Configuration 18) 18. The correction device according to any one of configurations 1 to 17, wherein the predetermined image processing includes processing for correcting color shift of an image based on the selected wavelength band. (Configuration 19) 19. The correction device according to any one of configurations 1 to 18, wherein the predetermined image processing includes processing for correcting brightness or exposure of an image based on the selected wavelength band. (Configuration 20) 20. The correction device according to any one of configurations 1 to 19, wherein the correction device is mounted on an imaging device that captures the input image. (Method 1) a fluctuation information acquisition step of acquiring fluctuation information relating to fluctuations in an input image; a band selection step of selecting a wavelength band related to generation of an output image in accordance with the fluctuation information; an image processing step of performing predetermined image processing on the input image based on the selected wavelength band to generate the output image; A correction method comprising: (Program 1) A program that causes a computer to function as the correction device according to any one of configurations 1 to 20. [Explanation of symbols]

[0079] 101: imaging optical system, 102: imaging element, 103: CPU, 104: RAM, 105: ROM, 111: image acquisition unit, 112: fluctuation information acquisition unit, 113: band selection unit, 114: image processing unit, 115: image output unit

Claims

1. a fluctuation information acquiring means for acquiring fluctuation information relating to fluctuation of an input image; a band selection means for selecting a wavelength band related to generation of an output image in accordance with the fluctuation information; an image processing unit that performs predetermined image processing on the input image based on the selected wavelength band to generate the output image; A correction device comprising:

2. 2. The correction device according to claim 1, wherein the band selection means selects a wavelength band related to generation of the output image from among wavelength bands contained in the input image based on the fluctuation information.

3. The correction device according to claim 1, characterized in that the band selection means selects a wavelength band related to the generation of the output image by changing the ratio of signals for each band within the wavelength band of the input image based on the fluctuation information.

4. The correction device described in claim 1, characterized in that the band selection means selects one or more optical filters from a plurality of optical filters that transmit or block different wavelength bands from the wavelength band of the incident light based on the fluctuation information, thereby selecting the wavelength band related to generation of the output image.

5. 5. The correction device according to claim 4, wherein the optical filter includes at least one of an infrared cut filter, a visible light cut filter, and a band-pass filter.

6. 6. The correction device according to claim 5, wherein the bandpass filter includes a dual bandpass filter that transmits two different wavelength bands.

7. The correction device described in claim 5, characterized in that the band selection means selects a wavelength band related to generation of the output image by selecting one visible light cut filter from two or more visible light cut filters that block different wavelength bands, or by selecting one band pass filter from two or more band pass filters that transmit different wavelength bands, based on the fluctuation information.

8. 2. The correction device according to claim 1, wherein the information acquisition means acquires the fluctuation information representing an amount of fluctuation based on the input image.

9. 9. The correction device according to claim 8, wherein the band selection means selects a wavelength band that is longer in wavelength as the amount of fluctuation increases.

10. 9. The correction device according to claim 8, wherein the information acquisition means acquires the fluctuation information representing the amount of fluctuation based on a difference in a pixel of interest between the input images that are successive in time series.

11. The correction device described in claim 8, characterized in that the information acquisition means acquires, as the fluctuation information representing the amount of fluctuation, a fluctuation period corresponding to the number of frames when the cumulative value of the difference between frames of the input image that are consecutive in chronological order becomes greater than or equal to a predetermined value.

12. The correction device according to claim 8, characterized in that the information acquisition means acquires the proportion of the area in the input image where fluctuation occurs as a fluctuation occurrence probability, and acquires the fluctuation occurrence probability as the fluctuation information representing the fluctuation amount.

13. 2. The correction device according to claim 1, wherein the information acquisition means acquires the fluctuation information designated by a user through a predetermined user interface.

14. The correction device described in claim 13, characterized in that the fluctuation information specified by the user through a specified user interface includes at least one of information on the correction level when correcting the fluctuation and information specifying a priority for correcting the fluctuation.

15. The correction device according to claim 1, characterized in that the fluctuation information includes at least one of information for disabling fluctuation correction based on at least one of the passage of time and the environment, and information for disabling fluctuation correction based on an estimated result of illuminance.

16. 2. The correction device according to claim 1, wherein the predetermined image processing includes a process of correcting the fluctuation using a plurality of the input images in chronological order.

17. 16. The correction device according to claim 15, wherein the information acquisition means acquires the fluctuation information based on a correction strength used in the image processing for correcting the fluctuation.

18. 2. The correction device according to claim 1, wherein the predetermined image processing includes processing for correcting color shift of an image based on the selected wavelength band.

19. 2. The correction device according to claim 1, wherein the predetermined image processing includes processing for correcting brightness or exposure of an image based on the selected wavelength band.

20. The correction device according to claim 1 , wherein the correction device is mounted on an imaging device that captures the input image.

21. a fluctuation information acquisition step of acquiring fluctuation information relating to fluctuations in an input image; a band selection step of selecting a wavelength band related to generation of an output image in accordance with the fluctuation information; an image processing step of performing predetermined image processing on the input image based on the selected wavelength band to generate the output image; A correction method comprising:

22. Computer, a fluctuation information acquiring means for acquiring fluctuation information relating to fluctuation of an input image; a band selection means for selecting a wavelength band related to generation of an output image in accordance with the fluctuation information; an image processing unit that performs predetermined image processing on the input image based on the selected wavelength band to generate the output image; A program that functions as a correction device having the above.

Images