Sudden noise detection device, imaging device, sudden noise detection method, display image data generation method, sudden noise detection program, and recording medium

Abstract

To accurately detect sudden noise generated in a photosensor in which a plurality of stages of line sensors are arrayed.SOLUTION: In an imaging device, on a TDI sensor, L stages of line sensors where N pieces of pixels are arrayed are arrayed. A data temporary storage part 2 includes a line memory part 21, and an A / D converter 22. The line memory part stores an electric signal based on electric charges accumulated in the TDI sensor 1. The A / D converter converts pixel data stored in the line memory part into digital pixel data, and outputs the converted digital pixel data. A sudden noise detection part 4 includes an outlier detection part 41, a majority processing part 421 constituting a sudden noise generation pixel coordinate detection part 42, and an output part 43. The outlier detection part detects the pixel supposed to have generated the sudden noise, as an outlier pixel. The sudden noise detection part executes majority processing based on an outlier detection result, and acquires a pixel coordinate where the sudden noise has been generated. The output part outputs a majority processing result to a display part 5.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a sudden noise detection device that detects sudden noise occurring in an optical sensor in which sensors for one line, each having a plurality of pixels arranged in a straight line, are arranged in multiple stages. [Background technology]

[0002] It is known that image sensors based on semiconductor materials can generate sudden noise such as random telegraph signals (RTS) caused by defect levels in the elements and noise caused by crystal defects during the manufacturing of the circuits that read out signals from the elements.

[0003] An imaging device that corrects RTS noise is disclosed in Patent Document 1. The imaging device disclosed in Patent Document 1 is intended for single-lens reflex cameras, and compares each pixel signal with an RTS threshold to determine whether or not each pixel signal is RTS noise. Pixel signals determined to be RTS noise are excluded, and an RTS exclusion pixel value for each pixel is generated based on the remaining pixel signals. The RTS threshold is generated from a histogram of a plurality of pixel signals repeatedly output a predetermined number of times for one pixel of a pixel unit in an imaging chip of an imaging element.

[0004] Also known is an imaging device equipped with a TDI (Time Delay Integration) sensor that ensures a good signal-to-noise ratio (S / N ratio) in the image, prevents deterioration of image sharpness, and achieves high-speed, high-sensitivity, and high-resolution readout. A TDI (Time Delay Integration) sensor is a sensor that uses a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and even in a TDI sensor, sudden noise such as RTS occurs. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-9824 Summary of the Invention [Problem to be solved by the invention]

[0006] Sudden noise is not a steady signal that repeatedly appears and disappears, and its occurrence is difficult to predict. Therefore, to improve image quality, it is necessary to detect it in real time and use the detected sudden noise to correct pixel signals. In an imaging device equipped with a TDI sensor, there is relative movement between the TDI sensor and the subject, and therefore it is not possible to simply apply a technology that uses an RTS threshold generated from a histogram of multiple pixel signals repeatedly output for one pixel a predetermined number of times, as in the imaging device shown in Patent Document 1.

[0007] Furthermore, in cameras using line scan sensors mounted on moving objects such as drones, aircraft, or satellites, the output of pixels (elements) in the line scan sensor may change suddenly at each readout timing depending on the state of the ground, which is why the technology disclosed in Patent Document 1 cannot be applied.

[0008] The present disclosure has been made in consideration of the above points, and aims to provide a sudden noise detection device that detects sudden noise that occurs in an optical sensor in which one-line sensors, each having a plurality of pixels arranged in a straight line, are arranged in multiple rows. [Means for solving the problem]

[0009] The sudden noise detection device for detecting sudden noise occurring in an optical sensor according to the present disclosure includes an outlier detection unit that performs an outlier detection process, in which sensors each corresponding to one line of a plurality of pixels are arranged in a straight line are arranged in a plurality of stages, and each of the sensors arranged in a plurality of stages captures the same location of a subject from the optical sensor, and detects whether or not the data values ​​of each of the plurality of digital pixel data are outliers from the plurality of digital pixel data based on pixel information of the plurality of pixels in the plurality of stages of sensors that captured the same location of the subject, thereby determining whether or not the digital pixel data is an outlier, and obtains an outlier detection result indicating whether or not the digital pixel data is an outlier; and an outlier detection unit that performs a majority vote process based on the outlier detection result obtained by the outlier detection unit, for the plurality of digital pixel data based on the pixel information for each of the plurality of pixels in each stage of sensors in the plurality of stages of sensors in the optical sensor within a certain time period traveling in the relative traveling direction of the optical sensor and the subject, and when the majority vote process result indicates the presence of an outlier, acquires the pixel coordinates of the digital pixel data for the majority vote process result that indicates the presence of an outlier as the sudden noise occurrence pixel coordinates where the sudden noise has occurred. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to realize high-speed, high-sensitivity, and high-resolution readout, and to accurately detect the occurrence of sudden noise in an optical sensor in which one-line sensors are arranged in multiple stages. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic configuration diagram showing an imaging device including an abrupt noise detection device according to a first embodiment. [Figure 2] 1 is a schematic diagram illustrating a mechanism for TDI imaging by a TDI sensor in an imaging device including the abrupt noise detection device according to the first embodiment. FIG. [Figure 3] 10 is a diagram schematically illustrating an example of pixel outputs of pixels in the i-th column in the first to third line sensors in the TDI sensor in the imaging device according to the first embodiment. FIG. [Figure 4]10 is a diagram illustrating an example of the characteristics of sudden noise occurring in a pixel in the i-th column in the second line sensor of the TDI sensor in the imaging device according to the first embodiment. FIG. [Figure 5] FIG. 10 is a diagram showing an example of a result of determining an outlier in a pixel in the i-th column of the second line sensor in the TDI sensor by the outlier detection unit in the sudden noise detection device in the imaging device according to the first embodiment. [Figure 6] 1 is a configuration diagram showing a hardware configuration of a sudden noise detection device in an imaging device according to a first embodiment. [Figure 7] 4 is a flowchart including the operation of the sudden noise detection device in the imaging device according to the first embodiment. [Figure 8] FIG. 10 is a configuration diagram showing a sudden noise detection device in an imaging device according to a second embodiment. [Figure 9] FIG. 10 is a diagram schematically illustrating an example of smoothing processing of pixel outputs of pixels in the i-th column in the first to third line sensors in the TDI sensor in the imaging device according to the second embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a result of determining an outlier in a pixel in the i-th column of the second line sensor in the TDI sensor by the outlier detection unit in the sudden noise detection device in the imaging device according to the second embodiment. [Figure 11] 10 is a flowchart including the operation of the sudden noise detection device in the imaging device according to the second embodiment. [Figure 12] FIG. 11 is a configuration diagram showing an abrupt noise detection device in an imaging device according to a third embodiment. [Figure 13] 11 is a flowchart including the operation of the sudden noise detection device in the imaging device according to the third embodiment. [Figure 14] FIG. 10 is a configuration diagram showing a sudden noise detection device in an imaging device according to a fourth embodiment. [Figure 15] 10 is a flowchart including the operation of the sudden noise detection device in the imaging device according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Embodiment 1 An imaging device including a sudden noise detection device according to the first embodiment will be described with reference to FIGS. FIG. 1 shows a schematic diagram of the configuration. The imaging device equipped with the sudden noise detection device of embodiment 1 is an imaging device in which an optical sensor moves relative to the subject, such as a fixed camera that captures images of a constantly moving product, or a camera whose subject is fixed and whose platform is moving and which is mounted on a moving body such as a drone, aircraft, or artificial satellite.

[0013] As shown in FIG. 1, the imaging device according to the first embodiment includes an optical sensor 1, a temporary data storage unit 2, a cumulative processing unit 3, a sudden noise detection device 4, and a display unit 5. The sudden noise detection device 4 is a sudden noise detection unit configured as part of a computer. The optical sensor 1 is formed by arranging a plurality of line sensors in a row, each line sensor having a plurality of pixels arranged in a straight line. In the first embodiment, the optical sensor 1 includes N pixels P 11 ~P 1N The TDI sensor is a linearly arranged line sensor L arranged in M ​​stages.

[0014] As an example, a case will be described below where an image of 1024 horizontal pixels by 2048 vertical pixels is obtained by TDI sensors arranged in M ​​stages. In this case, M is 16, for example. Also, N pixels P 11 ~P 1N is 1024 pixels, that is, N is 1024. The number of horizontal and vertical pixels can be 2048 x 1024.

[0015] That is, the first line sensor L1 has N pixels, that is, 1024 pixels P 11 ~P 1N are arranged in a straight line, and the second line sensor L2 has N pixels, that is, 1024 pixels P 21 ~P2N are arranged in a straight line, and the (M-1)th row, that is, the 15th row line sensor L(M-1) has N pixels, that is, 1024 pixels P (M-1)1 ~P (M-1)N are arranged in a straight line, and the Mth row, that is, the 16th row of the line sensor LM, has N pixels, that is, 1024 pixels P M1 ~P MN are arranged in a straight line.

[0016] In the following description, when it is not necessary to individually describe the first line sensor L1 to the Mth line sensor LM, they will be collectively referred to as line sensor L in order to avoid complexity. Furthermore, the subscript before P, which indicates a pixel, indicates the row, and the subscript after it indicates the column. When there is no need to explain pixels individually, they will be collectively referred to as pixel P in the following explanation to avoid complexity.

[0017] Each line sensor L has N pixels P arranged in a straight line perpendicular to the direction of relative movement with respect to the subject. The line sensors L are aligned in rows in the direction of relative movement with respect to the subject. Therefore, when viewed from the pixel P, the TDI sensor 1 has pixels P arranged in a matrix of N columns and M rows, or in the illustrated example, a matrix of 1024×16. 11 ~P MN have

[0018] The TDI sensor 1 is configured with each line sensor L being an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor, and is configured with a semiconductor integrated circuit in which M stages of line sensors L are integrated.

[0019] Each pixel P has at least a photoelectric conversion unit, which generates electric charges according to the amount of light of an incident light beam, and outputs the accumulated electric charges as an analog pixel signal. Each pixel P generates sudden noise, typically RTS. The amplitude of the sudden noise, i.e., the amount of change in offset current due to the sudden noise, does not change from the time it occurs until it disappears. In other words, the output of pixel P where the sudden noise occurs remains fluctuating at a constant value while the sudden noise is occurring.

[0020] The relative speed between the TDI sensor 1 and the subject is constant, and each of the line sensors L1 to LM captures an image of the subject in synchronization with the relative speed. As shown in Figure 1, if the imaging device is a fixed camera, the TDI sensor 1 is fixed and the subject moves in the direction indicated by arrow A, and if the imaging device is a camera mounted on a moving body, the TDI sensor 1 moves in the direction indicated by arrow B relative to the subject.

[0021] The mechanism of TDI imaging by the TDI sensor 1 will be briefly explained with reference to FIG. 2, the TDI sensor 1 has three line sensors, from line sensor L1 to line sensor L3, and the pixel P 11 , P 21 , P 31 This indicates the case where light from the subject is incident on the TDI sensor 1 from the first readout timing to the third readout timing. For ease of understanding, the subject is shown as one pixel, and it is assumed that the TDI sensor 1 is fixed and the subject moves at a constant speed in the direction of the arrow in FIG. 2.

[0022] That is, as shown in FIG. 2, at the first readout timing, the pixel P 11 At the second readout timing, the pixel P 21 At the third readout timing, the pixel P 31 When light from the subject enters the sensor, it accumulates electrical charges.

[0023] Since the relative speed between the TDI sensor 1 and the subject is synchronized with the first readout timing to the third readout timing, the pixel P 11 , P 21 , P 31 Image information from the same subject is read out.

[0024] Pixel P 11 , P 21 , P 31 have exactly the same performance, and the relative velocity is constant, so each pixel P 11 , P 21 , P 31 If the time during which light from the subject is incident is constant and there is no influence of noise, the amount of accumulated charge will be the same. It should be noted that even when the TDI sensor 1 moves relative to the subject, the mechanism of TDI imaging by the TDI sensor 1 is the same.

[0025] After that, at the first accumulation processing timing, the pixel P 11 , P 21 , P 31 The digital pixel data corresponding to the charges accumulated in the first row is accumulated to obtain accumulated pixel data for the first row of the subject. That is, cumulative pixel data is obtained by cumulatively processing digital pixel data based on the image information read by the first line sensor L1 at the first read timing, the image information read by the second line sensor L2 at the second read timing, and the image information read by the third line sensor L3 at the third read timing.

[0026] The digital pixel data of the subject obtained in this way indicates the cumulative value of the charges accumulated in the first line sensor L1 to the third line sensor L3 for the subject, and as a result, the time during which light from the subject is incident on the TDI sensor 1 becomes a pseudo-long time, resulting in a pixel signal with high sensitivity and high resolution.

[0027] The optical sensor 1 is not limited to a TDI sensor, but may be an area sensor if the condition is met in which one line of sensors L, each having a plurality of pixels P arranged in a straight line, is arranged in multiple rows, and light from the same location on the subject is incident for the same period of time between pixels P arranged in the same row.

[0028] The temporary data storage unit 2 includes a line memory unit 21 and an analog / digital converter (hereinafter referred to as an A / D converter) 22. The line memory unit 21 stores analog pixel data obtained by reading out electrical signals based on charges accumulated in pixels P on N columns in the line sensor L on M stages of the TDI sensor 1 at T read timings.

[0029] For example, if the number of pixels per image is 1024 horizontally and 2048 vertically, T is 2048 + (16 - 1), where 2048 is the number of vertical pixels and 16 is the number of stages of the line sensor L in the TDI sensor 1. The readout timing is the same as the sampling timing, and T is the number of samplings required to obtain one image.

[0030] For example, in the M (=16) line sensors L, the pixel P 1i , pixel P 2i , pixel P 3i An example of pixel output is shown in FIG. FIG. 3 shows the pixel P of the i-th column of the first line sensor L1 and the third line sensor L3. 1i and pixel P 3i No sudden noise occurs in the pixel P of the i-th row of the second line sensor L2. 2i 10A and 10B are diagrams showing the results of simulating an example in which sudden noise occurs in a signal.

[0031] In FIG. 3, the horizontal axis represents the time of relative movement between the TDI sensor 1 and the object, and the vertical axis represents the pixel P 1i , pixel P 2i , pixel P 3i The pixel output is shown. In addition, the pixel P in the i-th column of the M-th (=16)-th line sensor LM from the fourth-stage line sensor L3 3i and pixel P Mi is the pixel P of the i-th column of the first line sensor L1 and the third line sensor L3. 1i and pixel P 3i This is an example in which no sudden noise occurs, similar to the example in (1).

[0032] Furthermore, the first-stage line sensor L1 to the third-stage line sensor L3 have a timing difference in sampling the same line of the object, that is, the same location, depending on the distance in the direction perpendicular to the relative movement direction with respect to the object and the timing (reading cycle) of reading out the charges. In FIG. 3, the time direction coordinate j is adjusted to correct the timing difference, and the pixel P 1i , pixel P 2i , pixel P 3i The pixel outputs from the two sensors are displayed as if they were obtained at the same time.

[0033] That is, in Figure 3, the horizontal axis indicates time direction information j, i.e., the pixel output from the first line sensor L1 to the third line sensor L3 at sampling timing j, which represents the pixel output at sampling timing j for the first line sensor L1, the pixel output at sampling timing j+1 for the second line sensor L2, and the pixel output at sampling timing j+2 for the third line sensor L3.

[0034] Pixel P 1i , pixel P 2i , pixel P 3i The reason why pixel outputs from the two sensors show different values ​​is due to the influence of random noise. Pixel P 1i and pixel P 3i The pixel output is affected by random noise, 2i shows a simulated state where the signal is affected by random noise and sudden noise after 1 second. The read timing of the line memory unit 21 is, for example, 20 μs.

[0035] The line memory unit 21 includes T line memories M that temporarily store analog pixel data read out at T read timings from the line sensor L at each stage. 11 ~M 1T From line memory M M1 ~M MT It has. The subscript before M, which indicates the line memory, indicates a number corresponding to the stage of the line sensor L, and the subscript after it indicates the sampling timing at each stage. When there is no need to explain the line memories individually, to avoid complexity, in the following explanation they will be collectively referred to as line memory M. Each line memory M has memory sections corresponding to pixels P of the line sensor L, and the number of memory sections is the same as the number of pixels P. That is, each line memory M has memory sections for N columns.

[0036] Therefore, the line memory unit 21 has (N×M×T) memory units corresponding to (N×M×T) pixels equivalent to one image. In short, the image data temporarily stored in the line memory unit 21 is made up of analog pixel data with a data size of N×M×T. Data for (N×M×T) pixels is continuously acquired image data.

[0037] The temporary storage in each of the T line memories M corresponding to the line sensors L in each stage in the line memory unit 21 is repeated sequentially every T times of charge readout. For example, the line memory M for the first stage line sensor L1 11 ~M 1T is M for each charge readout 11 From M 1T The temporary storage in the line memory M for the line sensor L in the other stages is also repeated in this order. 11 ~M 1T The temporary storage is repeated in the same manner. In short, the line memory unit 21 temporarily stores image data captured by the M stages of line sensors L at T sampling timings, and it is sufficient that the correspondence between the M stages of line sensors L and the T sampling timings is clear.

[0038] The A / D converter 22 converts the line memory M 11 ~M 1T From line memory M M1 ~M MT A / D conversion unit AD 11 ~AD 1T A / D conversion section AD M1 ~AD MT It has. Each A / D conversion section AD 11 ~AD 1T ~AD M1 ~AD MT is the line memory M corresponding to the read timing. 11 ~M 1T ~M M1 ~M MT The analog pixel data temporarily stored in the memory is converted into digital pixel data and output. The A / D conversion units are also given subscripts like the line memories M, and when there is no need to explain them individually, they will be collectively referred to as A / D conversion units AD in the following explanation to avoid complexity.

[0039] The accumulation processing unit 3 accumulates data for multiple pixels P in the same column that capture the same line of the subject on the TDI sensor 1, which is stored in the data temporary storage unit 2, and obtains accumulated pixel data for one line of pixels P for one line of the subject. The accumulation processing unit 3 reads out the continuously acquired image data (N×M×T pixels) stored in the temporary data storage unit 2 at each read timing, accumulates digital pixel data based on pixel information of column i corresponding to each of N columns in the M-stage line sensor L that photographs the same line of the subject, and obtains accumulated pixel data of column i.

[0040] For example, the cumulative data of column i on the same line of the subject is cumulative pixel data obtained by accumulating digital pixel data based on pixel information obtained by the first-stage line sensor L1 at sampling timing j, digital pixel data based on pixel information obtained by the second-stage line sensor L2 at sampling timing (j+1), ..., and digital pixel data based on pixel information obtained by the Mth-stage line sensor LM at sampling timing (j+M-1). That is, in the example where the number of pixels per image is 1024 horizontally by 2048 vertically, the accumulated pixel data is 16 stages of digital pixel data accumulated for each of the 1024 horizontally by 2048 vertically pixels.

[0041] As a result, image information captured by the multiple line sensors L in the TDI sensor 1 is accumulated for one line of the subject, and cumulative pixel data accumulated for each line of the subject is output to the display unit 5. In short, the TDI sensor 1 outputs the accumulated pixel data for one line obtained by accumulating digital pixel data based on image information from multiple stages of line sensors L for one line of the subject, i.e., one vertical row of the image to be displayed, for each column of the line sensors L to the display unit 5 as one vertical row of the image to be displayed. Similarly, the accumulated pixel data for each row is output to the display unit 5 for all rows in the vertical direction of the image to be displayed. An imaging method in which charges are read out from each pixel P of the line sensor L in the TDI sensor 1 and accumulated in the form of digital values ​​to obtain image data is called digital TDI imaging.

[0042] As shown in FIG. 1, the sudden noise detection unit 4 includes an outlier detection unit 41, a majority processing unit 421 that constitutes the sudden noise occurrence pixel coordinate detection unit 42, and an output unit 43. The outlier detection unit 41 executes an outlier detection process in which, from a plurality of digital pixel data corresponding to pixel information of a plurality of pixels P in a multi-stage line sensor L that captures an image of the same location of a subject from the TDI sensor 1, it detects whether or not the data value of each of the plurality of digital pixel data is an outlier, determines whether or not the digital pixel data is an outlier, and obtains an outlier detection result indicating whether or not the digital pixel data is an outlier.

[0043] That is, the outlier detection unit 41 reads out the continuously acquired image data (N×M×T pixels) stored in the temporary data storage unit 2 at each read timing, and for digital pixel data based on pixel information of column i corresponding to each of N columns in the M-stage line sensor L obtained by photographing the same line of the subject, detects whether the data value of each digital pixel data is an outlier, performs outlier detection for the digital pixel data, and obtains an outlier detection result indicating whether the digital pixel data is an outlier.

[0044] The coordinate of pixel P of the line sensor L of each stage of the TDI sensor 1 is defined as horizontal coordinate i (i=1 to N), and the read timing in T read timings is defined as time coordinate j (j=1 to T). The horizontal coordinate i is the row of the line of the subject, that is, the pixel P of the line sensor L of each stage. 11 ~P 1N The time coordinate j indicates the sampling timing. The outlier detection unit 41 reads out the continuously acquired image data (N×M×T pixels) stored in the temporary data storage unit 2, extracts digital pixel data for each horizontal coordinate i and time coordinate j, and detects whether the data value of the extracted digital pixel data is an outlier, thereby determining whether the digital pixel data is an outlier.

[0045] The digital pixel data extracted by the outlier detector 41 at horizontal coordinate i and time coordinate j is the pixel P in the i-th column of the line sensor L capturing the same location, and has a data size of M pixels P. The data size of the extracted digital pixel data is, for example, M pieces of digital pixel data for column i on the same line of the subject, including digital pixel data based on pixel information obtained by the first-stage line sensor L1 at sampling timing j, digital pixel data based on pixel information obtained by the second-stage line sensor L2 at sampling timing (j+1), ... and digital pixel data based on pixel information obtained by the M-th-stage line sensor LM at sampling timing (j+M-1). The digital pixel data values ​​based on the pixel information of the i-th column of each line sensor L are based on image information obtained by photographing the same location on the subject, and therefore will be the same if there is no influence of noise.

[0046] The outlier detection unit 41 detects outliers using an outlier testing method such as the Smirnov-Grubbs test as follows. That is, for pixels P in the i-th column of the line sensor L that captured the same extracted location, the average or median (hereinafter, the average or median will be referred to as the reference value) and standard deviation of the data values ​​of the digital pixel data are calculated from the digital pixel data having a data size of M pixels P, and if the data value of the digital pixel data of the target pixel Pa differs from the reference value by more than C times the standard deviation (C is any real number greater than 1), it is determined to be an outlier, and the digital pixel data of the target pixel Pa is detected as digital pixel data that is an outlier.

[0047] The reason why the data values ​​of the digital pixel data differ depending on the pixel P is due to the influence of random noise, and the standard deviation is calculated based on the random noise. Therefore, the detection of outliers by the outlier detection unit 41 results in excluding pixels P that have not experienced sudden noise but have been affected by random noise from the outliers, and detecting pixels P that are suspected to have experienced sudden noise as outlier pixels P.

[0048] In this way, the outlier detection unit 41 detects whether all pixel data stored in the temporary data storage unit 2 for pixel information from the line sensor L that captured the same location, in this example N x M pieces of digital pixel data, are outliers, and determines whether the N x M pieces of digital pixel data are outliers.

[0049] That is, the outlier detection for N×M digital pixel data is performed by determining the pixel P 11 ~P MN This means that we are checking whether or not everything is an outlier. The outlier detection unit 41 detects whether the digital pixel data described above is an outlier for each line photographed of the subject, and determines whether all pixel data stored in the temporary data storage unit 2 are outliers, obtaining an outlier detection result indicating whether or not an outlier is present.

[0050] In this way, the outlier detection unit 41 obtains an outlier detection result, and the pixels P in the i-th column of the line sensor L in the TDI sensor 1 that image the same location have the same data value of the digital pixel data based on the image information from the pixels P in the i-th column, unless noise is present, because the line sensors L of each stage image the same location but at the same imaging time, although not at the same time. Conversely, this indicates that the difference in the data value of the digital pixel data based on the image information between the pixels P in the i-th column is caused by noise.

[0051] The Smirnoff-Grubbs test performed by the outlier detector 41 can detect pixels P whose values ​​fluctuate more than the random noise represented by thermal noise. In other words, when the magnitude of the random noise corresponds to the standard deviation and the amplitude of the sudden noise is C times the amplitude of the thermal noise, the outlier detection unit 41 recognizes and detects the target pixel Pa as outlier information containing sudden noise, based on the image information from the target pixel Pa.

[0052] Incidentally, when the number of stages of the line sensor L in the TDI sensor 1 is large, that is, when the number of pixels P in the i-th column of the line sensor L in the TDI sensor 1 capturing the same location is sufficient as a sample number for calculating statistics, the standard deviation matches the magnitude of the random noise, and sudden noise can be reliably detected.

[0053] However, if the number of stages of the line sensor L in the TDI sensor 1 is less than the number of samples required to calculate the statistics, the magnitude of the current due to the random noise fluctuates over time, and therefore the standard deviation measured from the random noise for the digital pixel data of the data size extracted by the outlier detection unit 41 fluctuates, and the standard deviation does not necessarily match the magnitude of the random noise. As a result, the outlier detection unit 41 may erroneously detect the occurrence of sudden noise even when no sudden noise is occurring at a certain moment, and may determine the data value of the digital pixel data as an outlier.

[0054] For example, the pixel P in the i-th column of the second line sensor L2 shown in FIG. 2i The characteristics of the sudden noise that occurred can be simulated as shown in Figure 4. In FIG. 4, the horizontal axis represents the time of relative movement between the TDI sensor 1 and the subject, and the vertical axis represents the amplitude of the sudden noise. Figure 4 shows that a sudden noise occurred after 1 second.

[0055] 5 shows the pixel P in the i-th column of the second line sensor L2 shown in FIG. 2i 10 shows an example of a determination result of outlier determination performed on the data. In FIG. 5, the horizontal axis represents the time of relative movement between the TDI sensor 1 and the object, and the vertical axis represents the pixel P 2i The results of outlier determination are shown below.

[0056] Since the TDI sensor 1 has 16 line sensors L, the pixel P 2i and Pixel P 1iand pixel P 3i From pixel P 16i Since there is a certain degree of difference between the pixel output of the pixel 10 and the pixel output of the pixel 11, the outlier detector 41 may determine that an outlier is present even though no sudden noise has occurred, as shown in FIG. Furthermore, there are also moments when sudden noise is not judged as an outlier, even though it occurs.

[0057] The sudden noise detection unit 4 in the imaging device according to the first embodiment further includes a majority processing unit 421 constituting the sudden noise occurrence pixel coordinate detection unit 42 so that sudden noise can be detected even when the number of stages of the line sensors L in the TDI sensor 1 is small. The majority voting processing unit 421 performs majority voting processing based on the outlier detection result obtained by the outlier detection unit 41, which indicates whether an outlier is present, for multiple digital pixel data based on multiple pixel information for each of multiple pixels P in the line sensor L of each stage in the TDI sensor 1, obtained within a certain period of time traveling in the relative traveling direction between the TDI sensor 1 and the subject, and if the majority voting processing result indicates the presence of an outlier, it acquires the pixel coordinates of the digital pixel data for the majority voting processing result indicating the presence of an outlier as the pixel coordinates where the sudden noise occurred.

[0058] The sudden noise represented by the RTS is observed as a fluctuation in the offset level of the output from the pixel P in the line sensor L. Unlike random noise, in which the magnitude of the current constantly fluctuates over time, sudden noise does not change the amount of change in the offset current due to the sudden noise, i.e., the magnitude of the amplitude of the sudden noise, from the time the sudden noise occurs to the time it disappears.

[0059] That is, while the sudden noise is occurring, the output of the pixel P where the sudden noise is occurring remains fluctuating at a constant value, as shown in Fig. 4 as an example, which is observed as a difference between the outputs from the pixel P of the line sensor L within a certain period of time. Therefore, by executing a majority vote process to determine whether or not the outlier detection results obtained by the outlier detection unit 41 within a certain period of time indicate an outlier, it is possible to confirm whether or not sudden noise has occurred.

[0060] Now, let the row of line sensor L in the TDI sensor 1 be the line sensor coordinate k, which is the coordinate in the relative traveling direction between the TDI sensor 1 and the subject, and let the outlier detection result outlier(i, j, k) be the outlier detection result obtained by the outlier detection unit 41 at the time direction coordinate j for the pixel in the horizontal coordinate i column in the kth row of line sensor L. The pixel at the jth time coordinate relative to the pixel at the ith column of the horizontal coordinate in the kth row of the line sensor L is set as the pixel of interest.

[0061] The majority voting processing unit 421 performs majority voting on the outlier detection result outlier(i, j, k) at the line sensor coordinate k and horizontal coordinate i using a window size w for the time direction coordinate j to obtain the majority voting processing result F(i, j, k) for the pixel of interest, and identifies and obtains the coordinates (i, j, k) indicating the pixel of interest for the digital pixel data in which sudden noise is occurring based on the majority voting processing result F(i, j, k).

[0062] That is, the majority processing unit 421 performs majority processing in three-dimensional data of horizontal coordinate i, time coordinate j, and line sensor coordinate k, with a window size of w for time coordinate j and a window size of 1 for horizontal coordinate i and line sensor coordinate k, i.e., with a scope of application only in the time direction. The coordinates (i, j, k) for the digital pixel data indicate the image information captured by pixel P in which row i of which row k of the line sensor L among the multiple rows of line sensors L in the TDI sensor 1, at what time, i.e., at what sampling timing j.

[0063] In short, the majority voting processing unit 421 obtains the majority voting result F(i, j, k) by performing a majority voting process on the digital pixel data of the i-th column of the k-th column line sensor L to determine whether or not the outlier detection result outlier(i, j, k) indicates an outlier, with the application range of the window size w in the time direction, based on the outlier detection result outlier(i, j, k) indicating whether or not an outlier is detected, obtained from image information from pixels P of the i-th column of the multiple-stage line sensor L in the TDI sensor 1 that photographed the same location of the subject.

[0064] Therefore, the outliers of less than 1 second in FIG. 5 shown as an example are determined to be no occurrence of sudden noise based on the majority vote result F(i, j, k), which coincides with the occurrence of sudden noise shown in FIG. The majority decision result F(i, j, k) can be expressed by the following equation (1).

[0065] TIFF0007770205000001.tif104166

[0066] In equation (1), i is the horizontal coordinate, j is the time coordinate, k is the line sensor coordinate, w is the window size in the time direction, overbar indicates negation, + indicates logical sum, and · indicates logical product. Further, outlier(i, j, k) indicates 1 if the outlier detector 41 detects an outlier, and indicates 0 if it does not.

[0067] If the result of the calculation F(i, j, k) is 1, it indicates that sudden noise has occurred at the coordinates (i, j, k) of the target pixel Pa, and if the result of the calculation is 0, it indicates that sudden noise has not occurred at the coordinates (i, j, k) of the target pixel Pa. The coordinates (i, j, k) of the pixel of interest Pa where sudden noise occurs are called the sudden noise occurrence pixel coordinates (i, j, k).

[0068] The output unit 43 outputs the majority processing result F(i, j, k) by the majority processing unit 421, which includes the pixel coordinates (i, j, k) where the sudden noise occurred, to the accumulation processing unit 3, and outputs the majority processing result F(i, j, k) to the display unit 5. The accumulation processing unit 3 identifies the pixel where the sudden noise occurred based on the pixel coordinates (i, j, k) where the sudden noise occurred from the sudden noise detection unit 4, and excludes the digital pixel data corresponding to the identified pixel from the accumulation processing.

[0069] That is, the accumulation processing unit 3 calculates the accumulated pixel data y(i, j) in the i-th column of the TDI sensor 1 using the input digital pixel data x(i, j, k) and the majority processing result F(i, j, k) from the sudden noise detection unit 4, using the following equation (2):

[0070] TIFF0007770205000002.tif17166

[0071] In equation (2), i is the horizontal coordinate, j is the time coordinate, k is the line sensor coordinate, F(i, j, k) is the result of the majority vote process, and x(i, j, k) is the value of digital pixel information based on pixel information from pixel P at time coordinate j for the pixel in the horizontal coordinate i column in the kth row of the line sensor L, and is the digital pixel data input to the accumulation processing unit 3.

[0072] The display unit 5 generates image data for display by a generally known method based on the accumulated pixel data y(i, j) from the accumulation processing unit 3, and displays an image. The majority processing result F(i, j, k) by the accumulation processing unit 3 indicates 0, that is, the digital pixel data x(i, j, k), which is the value of the digital pixel information indicated by the pixel coordinates (i, j, k) where the sudden noise occurred, is not added to the accumulated pixel data y(i, j), so an image with an improved S / N ratio through digital TDI processing while eliminating the effects of the sudden noise is obtained from the display unit 5.

[0073] In addition, since the output of the pixel P where the sudden noise, represented by RTS, occurs is maintained fluctuating at a constant value while the noise occurs, the value of the digital pixel data x(i, j, k) based on the pixel information may be excluded from the accumulation processing of the accumulation processing unit 3 to obtain accumulated pixel data y(i, j) from the time when the sudden noise occurrence pixel coordinates (i, j, k) are obtained for the pixel (i, k) in the TDI sensor 1 indicated by the sudden noise occurrence pixel coordinates (i, j, k) until the end of imaging by the TDI sensor 1.

[0074] Furthermore, from the point in time when the pixel coordinates (i, j, k) where the sudden noise occurred are obtained for pixel (i, k) in the TDI sensor 1, the digital pixel data x(i, j, k) based on the pixel information is excluded from the target of accumulation processing by the accumulation processing unit 3 to obtain accumulated pixel data y(i, j).If no outlier is detected by the outlier detection unit 41 for pixel (i, k) for a certain period of time after the digital pixel data x(i, j, k) is excluded from the target of addition, it is possible to consider that the sudden noise has disappeared and add the digital pixel data x(i, j, k) based on the pixel information of pixel (i, k) to the target of accumulation processing by the accumulation processing unit 3 to obtain accumulated pixel data y(i, j).

[0075] Furthermore, the output unit 43 may generate digital corrected pixel data for the pixel P indicated by the pixel coordinates (i, j, k) where the sudden noise occurs, and provide the digital corrected pixel data together with the majority processing result F(i, j, k) to the accumulation processing unit 3, which may then convert the digital pixel data x(i, j, k) indicated by the pixel coordinates (i, j, k) where the sudden noise occurs into the value of the corrected pixel data, which the accumulation processing unit 3 may then convert into corrected data and add, and an image may be displayed using the accumulated pixel data y(i, j). The corrected pixel data is generated by a commonly known method by the output unit 43. For example, the average value of the pixel data of the pixels P adjacent on both sides of the column indicated by the coordinates (i, j, k) of the pixel where the sudden noise occurred is set as the corrected pixel data.

[0076] In this case, the accumulation processing unit 3 deletes the digital pixel data x(i, j, k) indicated by the pixel coordinates (i, j, k) where the sudden noise occurs from the target of accumulation processing, and changes the value of the corrected pixel data obtained by changing the digital pixel data x(i, j, k) indicated by the pixel coordinates (i, j, k) where the sudden noise occurs, and this is called correcting the pixel data indicated by the coordinates where the sudden noise occurs. The coordinates of the pixel where the sudden noise occurs are expressed as (i, j, k), where j is used as the time direction coordinate when detecting outliers, but the coordinates for the image data displayed on the display unit 5 may also be expressed as (i, k).

[0077] As shown in FIG. 6, the sudden noise detection unit 4 has a hardware configuration realized by one or more processors having a semiconductor integrated circuit such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array), or one or more processors including an arithmetic device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) that executes program code of software or firmware read from a non-volatile memory, or one or more processors including a combination of a semiconductor integrated circuit such as a DSP and an arithmetic device such as a CPU.

[0078] As shown in FIG. 6, the sudden noise detection unit 4 includes a processor 100, a memory 200, an input interface unit 300, an output interface unit 400, and a signal path 500. The signal path 500 is a bus that interconnects the processor 100, memory 200, input interface unit 300, and output interface unit 400. The input interface unit 300 transfers the digital pixel data x(i, j, k) stored in the data temporary storage unit 2 to the processor 100 via a signal path 500 .

[0079] The processor 100 controls and manages the memory 200 , the input interface unit 300 and the output interface unit 400 . The processor 100 executes a process of generating pixel coordinates (i, j, k) where sudden noise occurs and corrected pixel data using digital pixel data x(i, j, k) input via the input interface unit 300 in accordance with a program stored in a ROM (Read Only Memory) that constitutes the memory 200. The output interface unit 400 outputs the coordinates (i, j, k) of the pixel where the sudden noise occurs and the corrected pixel data generated by the processor 100 via the signal path 500 to the display unit.

[0080] The memory 200 includes a ROM and a SDRAM (Synchronous Dynamic Random Access Memory). The ROM stores software or firmware program code that is executed by the processor 100 . The memory 200 may include a line memory unit 21, and the ROM may also include an accumulation processing program executed by the accumulation processing unit 3, so that the processor 100 generates image data from accumulated pixel data in accordance with the accumulation processing program.

[0081] Next, the operation of the sudden noise detection unit 4 will be mainly described with reference to FIG. Before the processing by the sudden noise detection unit 4, there is a step in which the data temporary storage unit 2 temporarily stores image information of the pixel P of the line sensor L in the TDI sensor 1 as analog pixel data. Step ST1 is an image reading step in which analog pixel data based on image information at pixel P of line sensor L in TDI sensor 1 is read into temporary data storage unit 2.

[0082] Now, the image data temporarily stored in the data temporary storage unit 2 is image data stored at T sampling timings by a TDI sensor 1 in which line sensors L, each having an array of N pixels P, are arranged in M ​​stages, and the continuously acquired image data is N×M×T digital pixel data x(i, j, k).

[0083] The timing at which data is read into the line memory unit 21 in the temporary data storage unit 2 is synchronized with the relative speed between the TDI sensor 1 and the subject, and the timing at which A / D conversion is performed by the A / D converter 22 is also synchronized with the read timing. That is, image information of the same line of the subject is read into the data temporary storage unit 2 in synchronization with the read timing of the image information from the line sensor L at each stage, and therefore is read into the line memory unit 21 in synchronization with each captured line.

[0084] In step ST2, the outlier detection unit 41 in the sudden noise detection unit 4 reads out the continuously acquired image data (N×M×T pixels) stored in the temporary data storage unit 2 and extracts digital pixel data for each horizontal coordinate i and time coordinate j. That is, step ST2 is an imaging pixel data extraction step for extracting digital pixel data based on pixel information of a row i corresponding to each of N rows in the line sensor L of M stages obtained by imaging the same line of the subject. The data size of the extracted pixel data is M pieces of digital pixel data x(i, j, k) at horizontal coordinate i in the time direction coordinate j, and M pieces of digital pixel data are extracted sequentially according to the time direction coordinate j.

[0085] That is, the outlier detection unit 41 extracts digital pixel data x(i, j, k) based on image information from M stages of line sensors L that have captured images of the same location on the subject for horizontal coordinates i from 1 to N, and uses this as digital pixel data for one line of the subject, and then performs a process of sequentially extracting the digital pixel data x for each line of one image. That is, in the example where the number of pixels per image is 1024 horizontally by 2048 vertically, processing is performed to extract 16 digital pixel data for each of the 1024 horizontally by 2048 vertically pixels.

[0086] Step ST3 is an outlier detection step in which outlier detection is performed for each data size of the digital pixel data of M data sizes, 16 data sizes as an example, extracted by the outlier detection unit 41 in step ST2, and an outlier detection result indicating whether or not the data size is an outlier is obtained.

[0087] The outlier detection unit 41 detects outliers by calculating a reference value, which is the average or median value of the data values ​​of M pieces of digital pixel data x(i, j, k), that is, digital pixel data based on pixel information of pixels P in M ​​stages of line sensors L that captured the same location on the subject, and a standard deviation, and if the data value of the digital pixel data based on pixel information of a pixel of interest Pa among pixels P in M ​​stages of line sensors L that captured the same location on the subject differs from the reference value by a real number multiple of the standard deviation or more, it is determined to be an outlier and detected. The outlier detection unit 41 detects outliers by, for example, the Smirnoff-Grubbs test.

[0088] The outlier detection unit 41 sequentially performs outlier detection processing on the digital pixel data extracted by the outlier detection unit 41 for horizontal coordinates i from 1 to N, thereby detecting outliers in the digital pixel data based on image information from multiple stages of line sensors L for one line. In addition, the outlier detection unit 41 sequentially performs outlier detection processing on the digital pixel data extracted by the outlier detection unit 41 for time direction coordinates j ranging from 1 to T, thereby detecting outliers in the digital pixel data based on image information from multiple stages of line sensors L for T lines. In short, the outlier detection unit 41 detects outliers from the (N×M×T) digital pixel data and obtains an outlier detection result indicating whether or not an outlier is present.

[0089] Step ST4 is a time-direction majority processing step in which the majority processing unit 421 performs majority processing on multiple digital pixel data based on multiple pixel information within the application range of the window size w in the time direction for each horizontal coordinate i at the line sensor coordinate k, based on the outlier detection result obtained by the outlier detection unit 41 in step ST3, to obtain the majority processing result.

[0090] In the majority process step ST4, majority process is executed sequentially for horizontal coordinates i from 1 to N at the line sensor coordinate k. In the majority decision processing step ST4, the majority decision processing is similarly executed for the line sensor coordinate k from 1 to M in order.

[0091] In majority processing step ST4 by the majority processing unit 421, majority processing of outliers is performed based on the outlier detection results obtained by the outlier detection unit 41 for digital pixel data based on pixel information of pixels P within the application range of window size w in the time direction in the kth line sensor L at line sensor coordinate k for each horizontal coordinate i, that is, for multiple digital pixel data based on pixel information of pixels P in one column, i.e., the ith column, of the kth line sensor L in the vertical direction relative to the line of the subject, and the occurrence of sudden noise is determined for pixel P at horizontal coordinate i for the kth line sensor L.

[0092] The majority voting processing unit 421 sequentially performs majority voting processing of outliers on digital pixel data within the application range of the window size w in the time direction extracted by the outlier detection unit 41 for horizontal coordinates i from 1 to N in the k-stage line sensor L, thereby determining the occurrence of sudden noise in the digital pixel data based on pixel information for all pixels P in the k-stage line sensor L.

[0093] Furthermore, the majority voting processing unit 421 sequentially performs majority voting processing of outliers on the digital pixel data extracted by the outlier detection unit 41, in the same manner as described for the k-stage line sensor L, for line sensor coordinates k ranging from 1 to M, thereby determining the occurrence of sudden noise in the digital pixel data based on the pixel information of all pixels P in all M stages of line sensors L. In short, the majority processing unit 421 determines whether or not sudden noise occurs for (N×M) pixels P in the line sensors L at M stages in the TDI sensor 1.

[0094] Furthermore, the majority voting processing unit 421 sequentially performs majority voting processing of outliers on the digital pixel data extracted by the outlier detection unit 41 for time direction coordinates j from 1 to T, in the same manner as described for the k-stage line sensor L, thereby determining the occurrence of sudden noise for all lines of one image.

[0095] That is, the majority processing step ST4 is a step in which the majority processing unit 421 obtains the majority processing result F(i, j, k) expressed by the above equation (1) and determines the occurrence of sudden noise for T lines of one image for (N × M) pixels P in the M-stage line sensor L in the TDI sensor 1. In addition, the pixel coordinates (i, j, k) indicated by the majority decision processing result F(i, j, k) that determines that abrupt noise has occurred are acquired as the pixel coordinates where abrupt noise has occurred.

[0096] In step ST5, when the majority processing unit 421 determines that the majority processing result indicates the occurrence of abrupt noise, it acquires the pixel coordinates indicated by the majority processing result indicating the occurrence as the abrupt noise occurrence pixel coordinates. The step of acquiring the abrupt noise occurrence pixel coordinates is the pixel coordinate acquisition step. Step ST5 is also a step in which the output unit 43 outputs the majority processing result F(i, j, k) by the majority processing unit 421 to the display unit 5 and outputs the sudden noise occurrence pixel coordinates (i, j, k) to the accumulation processing unit 3.

[0097] The accumulation processing unit 3 identifies the pixel where the sudden noise occurred based on the pixel coordinates (i, j, k) where the sudden noise occurred from the sudden noise detection unit 4, and corrects the digital pixel data x(i, j, k) corresponding to the identified pixel, that is, excludes it from the target of accumulation processing or accumulates the target of accumulation processing as corrected data. The display unit 5 generates image data for display based on the image data of the accumulated pixel data from the accumulation processing unit 3, and displays an image based on the image data for display. The steps up to the step of generating image data for display are the method for generating display image data.

[0098] The image data from the cumulative pixel data from the cumulative processing unit 3 is generated by an image data generation step in which the cumulative processing unit 3 reads out continuously acquired image data (N×M×T pixels) of an object photographed on the TDI sensor 1 that was temporarily stored in the temporary storage step, accumulates digital pixel data x(i, j, k) based on pixel information of column i corresponding to each of N columns in the M-stage line sensor L that photographed the same line of the object, obtains cumulative pixel data for each of i from 1 to N to obtain cumulative data for each pixel of one line, and further obtains cumulative pixel data in the same manner for each of time direction coordinates j from 1 to T to generate cumulative pixel data for one image.

[0099] The image data generation step by the accumulation processing unit 3 involves performing an accumulation process in which the accumulation processing unit 3 deletes the digital pixel data x(i, j, k) for a pixel identified as having experienced sudden noise based on the pixel coordinates (i, j, k) where the sudden noise has occurred from the sudden noise detection unit 4 to obtain accumulated pixel data, or performs an accumulation process in which the digital pixel data x(i, j, k) for the identified pixel is converted into digital correction data and added to obtain accumulated pixel data.

[0100] The sudden noise detection method for detecting sudden noise occurring in the TDI sensor 1 in steps ST2 to ST5 is performed by the processor 100 executing processes in accordance with a program stored in the ROM that constitutes the memory 200.

[0101] That is, the program stored in the ROM includes a procedure for extracting digital pixel data based on pixel information of a plurality of pixels P in the sensors of the plurality of stages that have captured an image of an object from an optical sensor having sensors of the plurality of stages, each of which has one line of sensors in which a plurality of pixels P are arranged in a straight line, and each sensor photographing the same location of the object; a procedure for detecting whether or not each of the plurality of digital pixel data based on pixel information of a plurality of pixels P in the sensors of the plurality of stages that have captured the same location of the object extracted by the extraction procedure is an outlier, and for determining whether or not the digital pixel data is an outlier; the majority voting processing procedure for performing majority voting processing based on the outlier detection results obtained by the outlier detection procedure on a plurality of digital pixel data based on a plurality of pixel information for each of a plurality of pixels P in each stage of sensors in the optical sensor within a certain period of time traveling in the relative traveling direction of the optical sensor and the subject, extracted by the extraction procedure, to obtain a majority voting processing result; and a pixel coordinate acquisition procedure for acquiring, when the majority voting processing procedure determines that abrupt noise has occurred as the pixel coordinates indicated by the majority voting processing result that determines that abrupt noise has occurred, as the pixel coordinates where the abrupt noise has occurred.

[0102] As described above, the sudden noise detection device in the imaging apparatus according to the first embodiment includes an outlier detection unit 41 that performs an outlier detection process to detect whether or not each data value of a plurality of digital pixel data based on pixel information of a plurality of pixels P in the plurality of line sensors L in the TDI sensor 1 having the plurality of line sensors L, which captures an image of the same location of the subject, is an outlier, to determine whether or not the digital pixel data is an outlier, and to obtain an outlier detection result indicating whether or not the digital pixel data is an outlier; and The TDI sensor is provided with an abrupt noise occurrence pixel coordinate detection unit 42 that performs majority voting based on the outlier detection result outlier(i, j, k), and when the majority voting result F(i, j, k) indicates the presence of an outlier, outputs the pixel coordinates (i, j, k) of the digital pixel data corresponding to the majority voting result F(i, j, k) that indicates the presence of an outlier as the pixel coordinates where the abrupt noise has occurred. Therefore, the outlier detection unit 41 performs primary detection of the occurrence of abrupt noise, and even if no abrupt noise is detected by the primary detection, the abrupt noise occurrence pixel coordinate detection unit 42 performs secondary detection of the occurrence of abrupt noise, thereby reducing false detections in which the occurrence of abrupt noise is not detected, and enabling highly accurate detection of the occurrence of abrupt noise in a TDI sensor that can achieve high-speed, high-sensitivity, and high-resolution readout.

[0103] Embodiment 2 An imaging device including a sudden noise detection device according to the second embodiment will be described with reference to FIGS. An imaging device equipped with the sudden noise detection device according to embodiment 2 differs from the sudden noise detection device according to embodiment 1 in that a smoothing processing unit 40 is placed before the outlier detection unit 41, but the other components are the same or equivalent. 8 to 11, the same reference numerals as those shown in FIGS. 1 to 7 indicate the same or corresponding parts.

[0104] The imaging device according to the second embodiment includes an optical sensor 1, a temporary data storage unit 2, a cumulative processing unit 3, an abrupt noise detection unit 4 which is an abrupt noise detection device, and a display unit 5. The optical sensor 1, data temporary storage unit 2, cumulative processing unit 3, and display unit 5 are the same as the optical sensor 1 (TDI sensor 1), data temporary storage unit 2, cumulative processing unit 3, and display unit 5 in the imaging device of embodiment 1, so their explanation will be omitted.

[0105] As shown in FIG. 8, the sudden noise detection unit 4 includes a smoothing processing unit 40, an outlier detection unit 41, a sudden noise occurrence pixel coordinate detection unit 42, and an output unit 43. The smoothing processing unit 40 performs smoothing processing in the time direction on the digital pixel data x(i, j, k) based on the pixel information of each of the pixels P of the multiple stages of line sensors L of the TDI sensor 1, to obtain smoothed pixel data.

[0106] The smoothing processing unit 40 performs a smoothing process in the time direction on the digital pixel data of each pixel P based on digital pixel data x(i, j, k) obtained by A / D conversion of multiple analog pixel data for pixel information of multiple pixels P stored in multiple line memories M for multiple stages of line sensors L in the data temporary storage unit 2, and obtains smoothed pixel data.

[0107] For example, among the M (=16) line sensors L shown in FIG. 3, the pixel P 1i , pixel P 2i , pixel P 3i When smoothing processing is performed in the time direction on the digital pixel data based on the pixel output by the above, smoothed pixel data that has been converted to analog form for comparison with FIG. 3 is obtained as the digital data shown in FIG. By smoothing the pixel output by pixel P, random noise is suppressed.

[0108] Now, the coordinate of pixel P of the line sensor L of each stage of the TDI sensor 1 is horizontal coordinate i (i = 1 to N), the readout timing at T sampling timings is time coordinate j (j = 1 to T), and the stage of the line sensor L in the TDI sensor 1 is line sensor coordinate k, which is the coordinate in the relative traveling direction between the TDI sensor 1 and the subject.

[0109] The smoothing processing unit 40 smoothes the digital pixel data in a window size w in the time direction for pixel P in the horizontal coordinate i column in the k-th row of the line sensor L, and calculates the smoothed pixel data A(i, J, k) using the following equation (3). That is, the smoothing processing unit 40 performs smoothing processing on three-dimensional data of horizontal coordinate i, time coordinate j, and line sensor coordinate k, with a window size of w for time coordinate j and a window size of 1 for horizontal coordinate i and line sensor coordinate k, i.e., with a range of application only in the time direction.

[0110] TIFF0007770205000003.tif21166

[0111] In equation (3), weight(tj) is an arbitrary weighting function whose sum is 1 at each readout timing from (-w / 2) to (+w / 2), and x(i, j, k) is the value of digital pixel information based on pixel information from pixel P at time coordinate j for pixels in horizontal coordinate column i in k-stage line sensor L, i.e., digital pixel data. The weight(tj) may be set in advance using a test subject.

[0112] That is, the smoothing processing unit 40 performs smoothing processing with a window size w in the time direction on each digital pixel data x(i, j, k) of continuously acquired image data with a data size of (N×M×T) using the above equation (3), and obtains smoothed pixel data A(i, J, k). The value of the smoothed pixel data A(i, J, k) based on the pixel information of the i-th column of each line sensor L is the same if there is no influence of noise.

[0113] While the outlier detection unit 41 in embodiment 1 directly uses the digital pixel data x(i, j, k) from the data temporary storage unit 2 to obtain the outlier detection result outlier(i, j, k), the outlier detection unit 41 obtains the outlier detection result outlier(i, j, k) using smoothed pixel data A(i, J, k) obtained by smoothing the digital pixel data x(i, j, k) in the continuously acquired image data (N×M×T) from the data temporary storage unit 2 by the smoothing processing unit 40.

[0114] The process of determining whether an outlier is present by the outlier detection unit 41 and obtaining the outlier detection result outlier(i, j, k) indicating whether an outlier is present is the same as the process of obtaining the outlier detection result outlier(i, j, k) by the outlier detection unit 41 in embodiment 1, and is performed using an outlier testing method such as the Smirnoff-Grubbs test.

[0115] 10 shows the pixel P in the i-th column of the second line sensor L2 shown in FIG. 2i 10 shows an example of the determination result of outlier detection performed by the outlier detection unit 41 on the data. In FIG. 10, the horizontal axis represents the time of relative movement between the TDI sensor 1 and the object, and the vertical axis represents the pixel P 2i The results of outlier determination are shown below.

[0116] The outlier detection unit 41 obtains the outlier detection result outlier(i, j, k) using the smoothed pixel data A(i, J, k) that has been smoothed. Therefore, random noise is suppressed by smoothing, the amplitude of sudden noise is maintained, and it becomes easy to distinguish between sudden noise and random noise. As a result, outliers, that is, sudden noise, can be detected more accurately.

[0117] The majority decision processing unit 421 and output unit 43 that constitute the sudden noise occurrence pixel coordinate detection unit 42 are the same as the majority decision processing unit 421 and output unit 43 that constitute the sudden noise occurrence pixel coordinate detection unit 42 in the imaging device according to embodiment 1, so their description will be omitted. In the imaging device according to the second embodiment, similarly to the first embodiment, the accumulation processing unit 3 performs accumulation processing by correcting the digital pixel data x(i, j, k) for a pixel identified as having abrupt noise based on the pixel coordinates (i, j, k) where the abrupt noise has occurred from the abrupt noise detection unit 4.

[0118] The display unit 5 receives the image data from the accumulation processing unit 3 and the coordinates (i, j, k) where the sudden noise occurs from the sudden noise detection unit 4, and displays an image in which the pixel data indicated by the coordinates (i, j, k) where the sudden noise occurs in the image data has been corrected. The sudden noise detection unit 4 includes a processor 100, a memory 200, an input interface unit 300, an output interface unit 400, and a signal path 500, as shown in FIG.

[0119] Next, the operation of the sudden noise detection unit 4 will be mainly described with reference to FIG. Before the processing by the sudden noise detection unit 4, there is a step in which the data temporary storage unit 2 temporarily stores image information of the pixel P of the line sensor L in the TDI sensor 1 as analog pixel data. Step ST1 is an image reading step for reading image information of pixel P of line sensor L in TDI sensor 1 into temporary data storage unit 2, and is the same as step ST1 in the first embodiment, so a description thereof will be omitted.

[0120] Step ST1A is a smoothing processing step in which the smoothing processing unit 40 performs smoothing processing in the time direction using the above equation (3) on the digital pixel data x(i, j, k) based on the pixel information of all pixels P of the multiple stages of line sensors L of the TDI sensor 1, to obtain smoothed pixel data A(i, J, k). That is, the smoothing processing unit 40 reads out the continuously acquired image data (N×M×T pixels) stored in the data temporary storage unit 2, and performs a smoothing process with a window size w in the time direction on the digital pixel data x(i, j, k) in the continuously acquired image data (N×M×T), thereby obtaining smoothed pixel data A(i, J, k).

[0121] Step ST2 is an extraction step of image pixel data in which the outlier detection unit 41 reads the continuously acquired image data (N×M×T) smoothed by the smoothing processing unit 40 and extracts digital pixel data for each horizontal coordinate i and time coordinate j. That is, the only difference between step ST2 in embodiment 1 and step ST2 in embodiment 1 is whether the data handled by the outlier detection unit 41 is continuously acquired image data (N×M×T pixels) stored in the data temporary storage unit 2, or continuously acquired image data (N×M×T) that has been smoothed by the smoothing processing unit 40 from the continuously acquired image data (N×M×T pixels) stored in the data temporary storage unit 2; since this is essentially the same as step ST2 in embodiment 1, detailed explanation will be omitted.

[0122] Step ST3 is an outlier detection step in which the outlier detection unit 41 obtains an outlier detection result, step ST4 is a time-direction majority voting processing step in which a majority voting processing result is obtained, and step ST5 is a step in which the coordinates of the pixel where the sudden noise occurred are obtained, and further, the output unit 43 provides the coordinates (i, j, k) of the pixel where the sudden noise occurred to the accumulation processing unit 3, and the output unit 43 outputs the majority voting processing result F(i, j, k) to the display unit 5. These steps are the same as steps ST3, ST4, and ST5 in the first embodiment, and therefore will not be described again.

[0123] Furthermore, the image data generating step for generating cumulative pixel data and the display image data generating step for generating image data for display are the same as the image data generating step and the display image data generating step in the first embodiment. The steps up to the step of generating image data for display are the method for generating display image data.

[0124] The sudden noise detection method for detecting sudden noise occurring in the TDI sensor 1 in steps ST1A to ST5 is performed by the processor 100 executing processes in accordance with a program stored in the ROM that constitutes the memory 200.

[0125] That is, the program stored in the ROM includes a step of performing a smoothing process in the time direction on digital pixel data based on pixel information of a plurality of pixels P in the plurality of sensors that have captured an image of an object from an optical sensor having a plurality of sensors in each of which a single line of a plurality of pixels P is arranged in a straight line, a step of extracting smoothed pixel data that has been subjected to the smoothing process, and a step of detecting whether or not a data value of each of the plurality of smoothed pixel data based on pixel information of a plurality of pixels P in the plurality of sensors that have captured an image of the same portion of the object and that has been extracted by the extraction step is an outlier, and The method includes an outlier detection procedure for determining whether an outlier is present and obtaining an outlier detection result indicating whether an outlier is present; a majority voting procedure for performing a majority voting process based on the outlier detection result obtained by the outlier detection procedure on a plurality of smoothed pixel data based on a plurality of pixel information for each of a plurality of pixels P in each of a plurality of sensor stages in the optical sensor within a certain period of time traveling in the relative traveling direction of the optical sensor and the subject, extracted by the extraction procedure, to obtain a majority voting result; and a pixel coordinate acquisition procedure for acquiring, when the majority voting procedure determines that abrupt noise has occurred as the pixel coordinates where the abrupt noise has occurred, the pixel coordinates indicated by the majority voting result that indicates that abrupt noise has occurred.

[0126] As described above, the sudden noise detection device in the imaging apparatus according to the second embodiment includes a smoothing processing unit 40 that performs smoothing processing in the time direction on a plurality of digital pixel data based on pixel information of a plurality of pixels P in the multi-stage line sensors L that capture an image of an object from the TDI sensor 1 having the multi-stage line sensors L, and obtains smoothed pixel data; an outlier detection unit 41 that performs outlier detection processing that detects whether or not each data value of a plurality of smoothed pixel data based on pixel information of a plurality of pixels P in the multi-stage line sensors L that capture an image of the same location of the object from the TDI sensor 1 is an outlier, determines whether or not the smoothed pixel data is an outlier, and obtains an outlier detection result indicating whether or not the smoothed pixel data is an outlier; and a smoothing processing unit 42 that performs smoothing processing on a plurality of smoothed pixel data based on pixel information for a plurality of pixels P in the line sensors L of each stage in the TDI sensor 1, within a certain time traveling in the relative traveling direction of the TDI sensor 1 and the object. The system is provided with an abrupt-noise-occurrence pixel coordinate detection unit 42 that performs majority voting on the raw data based on the outlier detection result outlier(i, j, k) obtained by the outlier detection unit 41, and, if the majority voting result F(i, j, k) indicates the presence of an outlier, outputs the pixel coordinates (i, j, k) of the smoothed pixel data for the majority voting result F(i, j, k) that indicates the presence of an outlier as the pixel coordinates where abrupt noise has occurred. Therefore, the smoothing process by the smoothing processing unit 40 makes it easy to distinguish between abrupt noise and random noise, and the outlier detection unit 41 performs primary detection of the occurrence of abrupt noise, and even if no abrupt noise is detected by the primary detection, the abrupt-noise-occurrence pixel coordinate detection unit 42 performs secondary detection of the occurrence of abrupt noise. This reduces false positives, in which the occurrence of abrupt noise is not detected, and enables more accurate detection of the occurrence of abrupt noise in a TDI sensor that can achieve high-speed, high-sensitivity, and high-resolution readout.

[0127] Embodiment 3 An imaging device including a sudden noise detection device according to the third embodiment will be described with reference to FIGS. An imaging device equipped with the abrupt noise detection device according to embodiment 3 differs from the imaging device equipped with the abrupt noise detection device according to embodiment 1 in that the abrupt noise occurrence pixel coordinate detection unit 42 in the imaging device equipped with the abrupt noise detection device according to embodiment 1 is configured with a majority processing unit 421, whereas the abrupt noise occurrence pixel coordinate detection unit 42 is equipped with a total outlier number calculation unit 422 and a second outlier detection unit 423; the other components are the same or equivalent components. 12 and 13, the same reference numerals as those shown in FIGS. 1 to 7 indicate the same or corresponding parts.

[0128] The imaging device according to the third embodiment includes an optical sensor 1, a temporary data storage unit 2, a cumulative processing unit 3, an abrupt noise detection unit 4 which is an abrupt noise detection device, and a display unit 5. The optical sensor 1, data temporary storage unit 2, cumulative processing unit 3, and display unit 5 are the same as the optical sensor 1 (TDI sensor 1), data temporary storage unit 2, cumulative processing unit 3, and display unit 5 in the imaging device of embodiment 1, so their explanation will be omitted. As shown in FIG. 12, the sudden noise detection unit 4 includes a first outlier detection unit 41, a sudden noise occurrence pixel coordinate detection unit 42, and an output unit 43. The first outlier detector 41 is the same as the outlier detector 41 in the imaging device according to the first embodiment, and therefore a description thereof will be omitted.

[0129] As shown in FIG. 12, the sudden noise occurrence pixel coordinate detection unit 42 includes an outlier total number calculation unit 422 and a second outlier detection unit 423. The total outlier count calculation unit 422 calculates the number of times that the outlier detection result outlier(i, j, k) exhibits an outlier, as the number of times S(i, k), based on the outlier detection result outlier(i, j, k) obtained by the first outlier detection unit 41, for multiple digital pixel data x(i, j, k) based on multiple pixel information for each of multiple pixels P in the line sensor L of each stage in the TDI sensor 1.

[0130] That is, for each of the multiple pixels P in the line sensor L at each stage in the TDI sensor 1, the total outlier count calculation unit 422 calculates the number of times the outlier detection result indicates an outlier based on the outlier detection results obtained by the first outlier detection unit 41 from multiple digital pixel data based on multiple pixel information for each of the multiple pixels P in the line sensor L at each stage, that is, based on the outlier detection results obtained by the first outlier detection unit 41 from multiple digital pixel data based on multiple pixel information for the pixels P in each column in the line sensor L at each stage.

[0131] In short, based on the outlier detection results obtained by the first outlier detection unit 41 for the digital pixel data x(i, j, k) in the continuously acquired image data (N×M×T), the total outlier count calculation unit 422 calculates the number of times that the outlier detection result showed an outlier among the T sampling times at each horizontal coordinate i for each line sensor L as the number of outliers.

[0132] Now, let the row of line sensor L in the TDI sensor 1 be line sensor coordinate k, which is the coordinate in the relative traveling direction between the TDI sensor 1 and the subject, and let outlier(i, j, k) be the outlier detection result obtained by the first outlier detection unit 41 at time direction coordinate j for pixel P in horizontal coordinate column i of the kth row of line sensor L. The total outlier count calculation unit 422 calculates the number of times that the outlier detection result outlier(i, j, k) obtained by the first outlier detection unit 41 showed an outlier based on image information from pixels P in the i-th row of horizontal coordinates in the k-th row of line sensors L, captured at T sampling timings, as the outlier count S(i, k).

[0133] In other words, the total outlier count calculation unit 422 calculates the number of times that the outlier detection result outlier(i, j, k) indicates an outlier for pixel P in the horizontal coordinate i column in the kth row of the line sensor L, where the time coordinate j is from 1 to T, as the outlier count S(i, k). The outlier total number calculation unit 422 calculates the number of outliers S(i, k) using the following equation (4).

[0134] TIFF0007770205000004.tif18166

[0135] The second outlier detection unit 423 compares the outlier count S(i, k) calculated by the outlier total count calculation unit 422 with a threshold value for each of the multiple pixels P in the line sensor L of each stage in the TDI sensor 1, and acquires the pixel coordinates of the pixel where the outlier count S(i, k) exceeds the threshold value as the sudden noise occurrence pixel coordinates (i, j, k) where the sudden noise has occurred.

[0136] The outlier count detection process by the second outlier detection unit 423 is performed using an outlier testing method such as the Smirnoff-Grubbs test, as in the first outlier detection unit 41. The threshold value for the number of outliers calculated by the outlier total number calculation unit 422 is calculated by calculating a reference value and standard deviation, which are the average or median of the number of outliers S(i, k) for each pixel P in one column of the line sensor L in each stage, and the standard deviation for the reference value is used as the threshold value.

[0137] That is, the second outlier detection unit 423 performs an outlier count detection process based on multiple outlier counts S(i, k) obtained sequentially for pixels P in the horizontal coordinate i column in the kth row of the line sensor L, and detects pixels (i, j, k) whose outlier count S(i, k) exceeds a threshold as pixels P where sudden noise is suspected to have occurred.

[0138] The pixel (i, j, k) where the outlier count S(i, k) exceeds the threshold is the pixel coordinate (i, j, k) where sudden noise occurs. The output unit 43 outputs the coordinates (i, j, k) of the pixel where the sudden noise occurs from the second outlier detection unit 423 to the accumulation processing unit 3 and the display unit 5.

[0139] During the occurrence of sudden noise represented by RTS, the output of the pixel P where the sudden noise occurred remains fluctuating at a constant value, and therefore the frequency with which a pixel (i, j, k) determined to be an outlier by the first outlier detection unit 41 is detected as an outlier increases as the time direction coordinate j ranges from 1 to T.

[0140] That is, a pixel (i, j, k) for which the outlier detection result outlier(i, j, k) indicates 1 is detected as an outlier more often than a pixel (i, j, k) for which the outlier detection result outlier(i, j, k) indicates 0. Therefore, by calculating the number of outliers S(i, k) for pixel P in horizontal coordinate column i in k-th row of line sensor L and performing outlier detection processing on the number of outliers S(i, k) for pixel P in column i, it is possible to confirm whether or not sudden noise has occurred. Therefore, the coordinates of the pixel where the sudden noise occurs may be coordinates (i, k).

[0141] Furthermore, in the imaging device according to the third embodiment, similarly to the first embodiment, the accumulation processing unit 3 performs accumulation processing by correcting the digital pixel data for the pixel identified as having abrupt noise based on the pixel coordinates (i, j, k) where the abrupt noise has occurred from the abrupt noise detection unit 4. The display unit 5 receives the image data from the accumulation processing unit 3 and the coordinates (i, j, k) where the sudden noise occurs from the sudden noise detection unit 4, and displays an image in which the pixel data indicated by the coordinates (i, j, k) where the sudden noise occurs in the image data has been corrected.

[0142] The sudden noise detection unit 4 includes a processor 100, a memory 200, an input interface unit 300, an output interface unit 400, and a signal path 500, as shown in FIG.

[0143] Next, the operation of the sudden noise detection unit 4 will be mainly described with reference to FIG. Before the processing by the sudden noise detection unit 4, there is a step in which the data temporary storage unit 2 temporarily stores image information of the pixel P of the line sensor L in the TDI sensor 1 as analog pixel data.

[0144] Step ST1, which is an image reading step, step ST2, which is an image pixel data extraction step, and step ST3, which is a first outlier detection step, are the same as steps ST1 to ST3 in the sudden noise detection unit 4 in embodiment 1, so their explanations are omitted.

[0145] Step ST4A is an outlier count calculation step in which the total outlier count calculation unit 422 calculates, for each of the multiple pixels P in the line sensor L of each stage in the TDI sensor 1, the number of times that the outlier detection result outlier(i, j, k) shows an outlier, as the outlier count S(i, k), based on the outlier detection result outlier(i, j, k) obtained by the first outlier detection unit 41 in step ST3 from multiple digital pixel data based on multiple pixel information for each of the multiple pixels P in the line sensor L of multiple stages.

[0146] That is, step ST4A is a step in which the total outlier count calculation unit 422 calculates the number of outlier detection results outlier(i, j, k) that indicate outliers for pixels P in the horizontal coordinate i column in the kth row of the line sensor L, where the time coordinate j ranges from 1 to T, as the outlier count S(i, k) using the above formula (3). In short, the outlier total number calculation unit 422 calculates the total number of outlier determinations for each of the plurality of pixels P in the line sensors L in the plurality of stages in the TDI sensor 1.

[0147] Specifically, the total outlier count calculation unit 422 calculates the number of times that all pixels P in the k-stage line sensor L are judged to be outliers by sequentially calculating the number of times that the outlier detection result outlier(i, j, k) indicating an outlier for the digital pixel data extracted by the first outlier detection unit 41 is found, for each horizontal coordinate i from 1 to N in the k-stage line sensor L.

[0148] In addition, the total outlier count calculation unit 422 sequentially calculates the number of times that outlier detection results outlier(i, j, k) indicating outliers for the digital pixel data extracted by the first outlier detection unit 41 are obtained for each line sensor coordinate k from 1 to M, thereby calculating the number of times that all pixels P in all M stages of line sensors L are determined to be outliers. In short, the total outlier count calculation unit 422 calculates the number of outlier counts S(i, k) for the (N×M) pixels P in the line sensors L at M stages in the TDI sensor 1.

[0149] Steps ST4B and ST5A are pixel coordinate acquisition steps in which the second outlier detection unit 423 compares the outlier count S(i, k) calculated by the total outlier count calculation unit 422 in step ST4A with a threshold value for each of the multiple pixels P in the line sensor L of each stage in the TDI sensor 1, and acquires the pixel coordinates of the pixel where the outlier count S(i, k) exceeds the threshold value as the pixel coordinates where sudden noise occurs.

[0150] Specifically, the second outlier detection unit 423 determines whether the number of outliers S(i, k) calculated by the total outlier calculation unit 422 exceeds a threshold value for each horizontal coordinate i from 1 to N in the k-th row of the line sensor L, detects the pixel (i, j, k) where the number of outliers S(i, k) exceeds the threshold value as a pixel P where sudden noise is suspected to have occurred, and obtains the pixel coordinates (i, j, k) where the sudden noise occurred.

[0151] In addition, the second outlier detection unit 423 sequentially detects pixels (i, j, k) for which the number of outliers S(i, k) calculated by the total outlier calculation unit 422 for each line sensor coordinate k from 1 to M exceeds a threshold, thereby determining the occurrence of sudden noise in all pixels P in all M stages of line sensors L.

[0152] In short, steps ST4B and ST5A are steps in which the second outlier detection unit 423 determines whether sudden noise has occurred for (N×M) pixels P in the M-stage line sensor L in the TDI sensor 1 using the outlier count S(i, k), and acquires the pixel (i, j, k) where the outlier count S(i, k) exceeds a threshold as the sudden noise occurring pixel coordinate (i, j, k).

[0153] In this case, the coordinates of the pixel where the sudden noise occurred may be coordinates (i, k) because the output of the pixel P where the sudden noise occurred, represented by RTS, remains fluctuating at a constant value while the sudden noise is occurring, and therefore the number of outliers S(i, k) of the pixel P where the sudden noise occurred is larger than that of the pixel P where the sudden noise did not occur.

[0154] Step ST5A is a step in which the output unit 43 outputs the coordinates (i, j, k) of the pixel where the sudden noise occurs to the accumulation processing unit 3 and the display unit 5. The accumulation processing unit 3 identifies the pixel where the sudden noise occurred based on the pixel coordinates (i, j, k) where the sudden noise occurred from the sudden noise detection unit 4, and corrects the digital pixel data corresponding to the identified pixel, that is, excludes it from the target of accumulation processing or accumulates the target of accumulation processing as corrected data. The display unit 5 generates image data based on the image data of the accumulated pixel data from the accumulation processing unit 3, and displays an image based on the image data. The steps up to the step of generating image data for display are the method for generating display image data.

[0155] The image data from the cumulative pixel data from the cumulative processing unit 3 is generated by an image data generation step in which the cumulative processing unit 3 reads out continuously acquired image data (N×M×T pixels) of an object photographed on the TDI sensor 1 that was temporarily stored in the temporary storage step, accumulates digital pixel data x(i, j, k) based on pixel information of column i corresponding to each of N columns in the M-stage line sensor L that photographed the same line of the object, obtains cumulative pixel data for each of i from 1 to N to obtain cumulative data for each pixel of one line, and further obtains cumulative pixel data in the same manner for each of time direction coordinates j from 1 to T to generate cumulative pixel data for one image.

[0156] The image data generation step by the accumulation processing unit 3 involves performing an accumulation process in which the accumulation processing unit 3 deletes digital pixel data for pixels identified as pixels where sudden noise has occurred based on the pixel coordinates (i, j, k) from the sudden noise detection unit 4 where the sudden noise has occurred, to obtain accumulated pixel data, or performs an accumulation process in which the digital pixel data for the identified pixels is converted into digital correction data and added to obtain accumulated pixel data.

[0157] The sudden noise detection method for detecting sudden noise occurring in the TDI sensor 1 in steps ST2 to ST4B is performed by the processor 100 executing processes in accordance with a program stored in the ROM that constitutes the memory 200.

[0158] That is, the program stored in the ROM includes a procedure for extracting digital pixel data based on pixel information of a plurality of pixels P in the sensors of the plurality of stages that have captured an image of an object from an optical sensor having a plurality of stages in which sensors, each of which has a single line of a plurality of pixels P arranged in a straight line, are arranged in a plurality of stages, and each sensor captures an image of the same location of the object; and an outlier detection procedure for detecting whether or not each of the plurality of digital pixel data based on the pixel information of the plurality of pixels P in the sensors of the plurality of stages that have captured an image of the same location of the object extracted by the extraction procedure is an outlier, and determining whether or not the digital pixel data is an outlier. an outlier detection procedure for obtaining a result of the outlier detection; an outlier count calculation procedure for calculating, for each of a plurality of pixels in each stage of the sensors of the multiple stages in the optical sensor, the number of times that the outlier detection result indicates an outlier based on the outlier detection result obtained by the outlier detection procedure from a plurality of digital pixel data based on a plurality of pixel information for each of the plurality of pixels in the sensors of the multiple stages; and a pixel coordinate acquisition procedure for comparing, for each of the plurality of pixels in each stage of the sensors of the multiple stages in the optical sensor, the number of times the outlier detection result indicates an outlier, the outlier count calculated by the outlier count calculation procedure with a threshold, and acquiring, as the sudden noise occurrence coordinates, the pixel coordinates of the pixel whose outlier count exceeds the threshold.

[0159] As described above, the sudden noise detection device in the imaging device according to the third embodiment includes a first outlier detection unit 41 that performs an outlier detection process to detect whether or not each of a plurality of digital pixel data values ​​is an outlier from a plurality of digital pixel data based on pixel information of a plurality of pixels P in a plurality of line sensors L that have been used to capture an image of the same location of a subject using the TDI sensor 1 that has multiple line sensors L, to determine whether or not the digital pixel data is an outlier, and obtain an outlier detection result indicating whether or not the digital pixel data is an outlier; and a first outlier detection unit 41 that performs an outlier detection process to determine whether or not the digital pixel data is an outlier from the plurality of digital pixel data based on pixel information for each of the plurality of pixels in the line sensors L at each line sensor L in the TDI sensor 1, to determine whether or not the outlier detection result outlier(i, j, k) is an outlier. and a sudden-noise-occurrence pixel coordinate detection unit 42 which includes a total outlier count calculation unit 422 which calculates the number of times that the number of times above 1000 is shown as the outlier count S(i, k), and a second outlier detection unit 423 which compares the outlier count S(i, k) calculated by the total outlier count calculation unit 422 with a threshold for each of the plurality of pixels P in the line sensor L of each stage, and acquires the pixel coordinates of the pixel P where the outlier count S(i, k) exceeds the threshold as the sudden-noise-occurrence pixel coordinate (i, k) where the sudden noise has occurred. Therefore, primary detection of the occurrence of sudden noise is performed by the outlier detection unit 41, and even if no sudden noise is detected by the primary detection, secondary detection of the occurrence of sudden noise is performed by the sudden-noise-occurrence pixel coordinate detection unit 42, thereby reducing erroneous detections in which the occurrence of sudden noise is not detected, and enabling accurate detection of the occurrence of sudden noise in a TDI sensor which can achieve high-speed, high-sensitivity, and high-resolution readout.

[0160] Embodiment 4 An imaging device including a sudden noise detection device according to the fourth embodiment will be described with reference to FIGS. An imaging device equipped with the abrupt noise detection device according to embodiment 4 differs from the abrupt noise detection device according to embodiment 3 in that a smoothing processing unit 40 is placed before the first outlier detection unit 41, and the other components are the same or equivalent. 14 and 15, the same reference numerals as those shown in FIGS. 1 to 13 indicate the same or corresponding parts.

[0161] As shown in FIG. 14, the imaging device according to the fourth embodiment includes an optical sensor 1, a temporary data storage unit 2, a cumulative processing unit 3, an abrupt noise detection unit 4 which is an abrupt noise detection device, and a display unit 5. The optical sensor 1, data temporary storage unit 2, cumulative processing unit 3, and display unit 5 are the same as the optical sensor 1 (TDI sensor 1), data temporary storage unit 2, cumulative processing unit 3, and display unit 5 in the imaging device of embodiment 1, as in the imaging device of embodiment 3.

[0162] The sudden noise detection unit 4 includes a smoothing processing unit 40 , a first outlier detection unit 41 , a sudden noise occurrence pixel coordinate detection unit 42 , and an output unit 43 . The smoothing processor 40 performs smoothing processing in the time direction on digital pixel data based on pixel information of all pixels of the pixels of the multiple stages of sensors of the TDI sensor 1, to obtain smoothed pixel data. The smoothing processing unit 40 is the same as the smoothing processing unit 40 in the second embodiment, and a detailed description thereof will be omitted.

[0163] While the first outlier detection unit 41 in embodiment 3 obtains the outlier detection result outlier(i, j, k) by directly using the digital pixel data x(i, j, k) from the data temporary storage unit 2, like the outlier detection unit 41 in embodiment 1, the first outlier detection unit 41 obtains the outlier detection result outlier(i, j, k) by using smoothed pixel data A(i, J, k) obtained by smoothing the digital pixel data x(i, j, k) in the continuously acquired image data (N×M×T) from the data temporary storage unit 2 by the smoothing processing unit 40.

[0164] The first outlier detection unit 41 obtains the outlier detection result outlier(i, j, k) using the smoothed pixel data A(i, J, k). However, the processing itself is essentially the same as the processing of the outlier detection unit 41 in the first embodiment, as well as the processing of the first outlier detection unit 41 in the third embodiment, and therefore a detailed description thereof will be omitted.

[0165] The abrupt noise occurrence pixel coordinate detection unit 42 includes an outlier total number calculation unit 422 and a second outlier detection unit 423, similar to the abrupt noise occurrence pixel coordinate detection unit 42 in the third embodiment. The outlier total number calculation unit 422 calculates the number of times that the outlier detection result outlier(i, j, k) indicates an outlier based on the outlier detection result outlier(i, j, k) obtained by the first outlier detection unit 41 as the outlier count S(i, k). The outlier total number calculation unit 422 is the same as the outlier total number calculation unit 422 in the third embodiment, and therefore a detailed description thereof will be omitted.

[0166] The second outlier detection unit 423 compares the outlier count S(i, k) calculated by the outlier total count calculation unit 422 with a threshold, and acquires the pixel coordinates of the pixel where the outlier count S(i, k) exceeds the threshold as the sudden noise occurrence pixel coordinates (i, j, k) where the sudden noise has occurred. Second outlier detection section 423 is the same as second outlier detection section 423 in the third embodiment, and therefore a detailed description thereof will be omitted.

[0167] The output unit 43 outputs the coordinates (i, j, k) of the pixel where the sudden noise occurs from the second outlier detection unit 423 to the accumulation processing unit 3 and the display unit 5. In the imaging device of embodiment 4, similar to embodiment 3, the accumulation processing unit 3 performs accumulation processing by correcting the digital pixel data for the pixel identified as having abrupt noise based on the abrupt noise occurrence pixel coordinates (i, j, k) from the abrupt noise detection unit 4. The display unit 5 receives the image data from the accumulation processing unit 3 and the coordinates (i, j, k) where the sudden noise occurs from the sudden noise detection unit 4, and displays an image in which the pixel data indicated by the coordinates (i, j, k) where the sudden noise occurs in the image data has been corrected.

[0168] The sudden noise detection unit 4 includes a processor 100, a memory 200, an input interface unit 300, an output interface unit 400, and a signal path 500, as shown in FIG. 6 which explains the sudden noise detection unit 4 in embodiment 1, similar to the sudden noise detection unit 4 in embodiment 3.

[0169] Next, the operation of the sudden noise detection unit 4 will be mainly described with reference to FIG. Before the processing by the sudden noise detection unit 4, there is a step in which the data temporary storage unit 2 temporarily stores image information of the pixel P of the line sensor L in the TDI sensor 1 as analog pixel data.

[0170] Step ST1 is an image reading step for reading image information of pixel P of line sensor L in TDI sensor 1 into temporary data storage unit 2, and is the same as step ST1 in the first embodiment as well as step ST1 in the third embodiment. Step ST1A is a smoothing processing step in which the smoothing processing unit 40 performs smoothing processing in the time direction using the above equation (3) on the digital pixel data based on the pixel information of all pixels P of the multiple stages of line sensors L of the TDI sensor 1, to obtain smoothed pixel data A(i, J, k). Step ST1A is the same as step ST1A in the second embodiment, and therefore a description thereof will be omitted.

[0171] Step ST2 is an extraction step of the captured pixel data in which the outlier detection unit 41 extracts smoothed pixel data A(i, J, k) based on pixel information of multiple pixels P in multiple stages of line sensors L that have captured the subject, via the smoothing processing unit 40. In step ST2, the data handled by the first outlier detection unit 41 is smoothed pixel data A(i, J, k) obtained by smoothing the digital pixel data from the data temporary storage unit 2 by the smoothing processing unit 40, whereas in embodiment 3 the data handled by the first outlier detection unit 41 is digital pixel data from the data temporary storage unit 2, and the data processing itself is the same. Step ST2 in the third embodiment is substantially the same as step ST2 in the first embodiment, which step ST2 in the third embodiment refers to.

[0172] Step ST3 is an outlier detection step in which the outlier detection unit 41 obtains an outlier detection result, step ST4A is an outlier count calculation step in which the total outlier count calculation unit 422 calculates the outlier count S(i, k), steps ST4B and ST5A are pixel coordinate acquisition steps in which the second outlier detection unit 423 acquires pixel coordinates where sudden noise occurs, and step ST5A is a step in which the sudden noise detection unit 4 outputs the pixel coordinates (i, j, k) where sudden noise occurs to the accumulation processing unit 3 and the display unit 5, and these are respectively the same as step ST3, step ST4A, step ST4B, and step ST5A in embodiment 3.

[0173] The accumulation processing unit 3 identifies the pixel where the sudden noise occurred based on the pixel coordinates (i, j, k) where the sudden noise occurred from the sudden noise detection unit 4, and corrects the digital pixel data corresponding to the identified pixel, that is, excludes it from the target of accumulation processing or accumulates the target of accumulation processing as corrected data. The display unit 5 generates image data based on the image data of the accumulated pixel data from the accumulation processing unit 3, and displays an image based on the image data.

[0174] In addition, the image data generation step for generating cumulative pixel data and the display image data generation step for generating image data for display are the same as the image data generation step and the display image data generation step in embodiment 1, as in embodiment 3. The steps up to the step of generating image data for display are the method for generating display image data.

[0175] The sudden noise detection method for detecting sudden noise occurring in the TDI sensor 1 in steps ST1A to ST4B is performed by the processor 100 executing processes in accordance with a program stored in the ROM that constitutes the memory 200.

[0176] That is, the program stored in the ROM includes a procedure for performing a smoothing process in the time direction on digital pixel data based on pixel information of a plurality of pixels P in the plurality of sensors that have captured an image of an object from an optical sensor having a plurality of sensors in each of which a single line of a plurality of pixels P is arranged in a straight line, a procedure for extracting smoothed pixel data that has been subjected to the smoothing process, and a procedure for detecting whether or not a data value of each of the plurality of smoothed pixel data based on pixel information of a plurality of pixels P in the plurality of sensors that captured an image of the same portion of the object and that has been extracted by the extraction procedure is an outlier, thereby determining whether or not the smoothed pixel data is an outlier. the optical sensor includes an outlier detection procedure for obtaining an outlier detection result indicating whether an outlier is detected; an outlier count calculation procedure for calculating, for each of a plurality of pixels in each stage of the sensors of the multiple stages in the optical sensor, the number of times the outlier detection result indicates an outlier based on the outlier detection result obtained by the outlier detection procedure from a plurality of smoothed pixel data based on a plurality of pixel information for each of the plurality of pixels in the sensors of the multiple stages; and a pixel coordinate acquisition procedure for comparing, for each of the plurality of pixels in each stage of the sensors of the multiple stages in the optical sensor, the number of times the outlier detection result indicates an outlier, the outlier count calculated by the outlier count calculation procedure with a threshold, and acquiring, as the sudden noise occurrence coordinates, the pixel coordinates of the pixel whose outlier count exceeds the threshold.

[0177] As described above, the sudden noise detection device in the imaging device according to the fourth embodiment includes a smoothing processing unit 40 that performs a smoothing process in the time direction on a plurality of digital pixel data based on pixel information of a plurality of pixels P in the multi-stage line sensors L that have been used to capture an image of an object from the TDI sensor 1 that has the multi-stage line sensors L, and obtains smoothed pixel data; a first outlier detection unit 41 that performs an outlier detection process that detects whether or not a data value of each of a plurality of smoothed pixel data based on pixel information of a plurality of pixels P in the multi-stage line sensors L that have been used to capture an image of the same part of the object from the TDI sensor 1 is an outlier, and determines whether or not the smoothed pixel data is an outlier, and obtains an outlier detection result indicating whether or not the smoothed pixel data is an outlier; and a first outlier detection unit 41 that performs an outlier detection process that detects whether or not a data value of each of a plurality of smoothed pixel data based on pixel information of a plurality of pixels P in the multi-stage line sensors L that have been used to capture an image of the same part of the object from the TDI sensor 1 is an outlier, and obtains an outlier detection result indicating whether or not the smoothed pixel data is an outlier; and a second outlier detection unit 423 that compares the outlier count S(i, k) calculated by the outlier count calculation unit 422 with a threshold value for each of a plurality of pixels P in the line sensor L at each stage, and acquires the pixel coordinates of the pixel P where the outlier count S(i, k) exceeds the threshold value as the abrupt noise occurrence pixel coordinates (i, k) where the abrupt noise has occurred. As a result, the smoothing process by the smoothing processing unit 40 makes it easy to distinguish between sudden noise and random noise, and the outlier detection unit 41 performs primary detection of the occurrence of sudden noise, and even if no sudden noise is detected by the primary detection, the sudden-noise-occurrence pixel coordinate detection unit 42 performs secondary detection of the occurrence of sudden noise. This reduces false detections in which the occurrence of sudden noise is not detected, and makes it possible to more accurately detect the occurrence of sudden noise in a TDI sensor that can achieve high-speed, high-sensitivity, and high-resolution readout.

[0178] It should be noted that the embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted. [Industrial Applicability]

[0179] An imaging device equipped with the sudden noise detection device of the present disclosure is applied to an imaging device in which an optical sensor moves relative to a subject, and is suitable for, for example, a fixed camera that captures images of a constantly moving product, or a camera with a moving platform where the subject is fixed and is mounted on a moving body such as a drone, aircraft, or artificial satellite. [Explanation of symbols]

[0180] 1 optical sensor (TDI sensor), 2 data temporary storage unit, 21 line memory unit, 22 A / D converter, 3 accumulation processing unit, 4 sudden noise detection unit (sudden noise detection device), 40 smoothing processing unit, 41 (first) outlier detection unit, 42 sudden noise occurrence pixel coordinate detection unit, 421 majority processing unit, 422 outlier total number calculation unit, 423 second outlier detection unit, 43 output unit, 5 display unit.

Claims

1. an outlier detection unit that performs an outlier detection process, in which sensors each having one line of a plurality of pixels arranged in a straight line are arranged in a plurality of stages, and each of the sensors arranged in a plurality of stages is an optical sensor that photographs the same location of the subject, and detects whether or not each of the data values ​​of the plurality of digital pixel data is an outlier from pixel information of the plurality of pixels in the sensors in the plurality of stages that photograph the same location of the subject, and determines whether or not the digital pixel data is an outlier, and obtains an outlier detection result that indicates whether or not the digital pixel data is an outlier; an abrupt noise detection unit that performs a majority vote process based on an outlier detection result obtained by the outlier detection unit on a plurality of digital pixel data based on a plurality of pixel information for a plurality of pixels in each of a plurality of stages of sensors in the optical sensor within a certain period of time traveling in a relative traveling direction between the optical sensor and the subject, and when the majority vote process result indicates the presence of an outlier, acquires pixel coordinates of the digital pixel data corresponding to the majority vote process result indicating the presence of an outlier as abrupt noise occurrence pixel coordinates where abrupt noise has occurred; a sudden noise detection device for detecting sudden noise occurring in the optical sensor, comprising:

2. a smoothing processing unit that performs a smoothing process in a time direction on digital pixel data based on pixel information of all pixels of the pixels of the plurality of stages of sensors of the optical sensor, and obtains smoothed pixel data; 2. The sudden noise detection device according to claim 1, wherein the digital pixel data on which the outlier detection unit performs the outlier detection process is smoothed pixel data that has been smoothed by the smoothing unit.

3. a first outlier detection unit that performs an outlier detection process in which sensors for one line, each having a plurality of pixels arranged in a straight line, are arranged in a plurality of stages, and each of the sensors arranged in a plurality of stages is an optical sensor that photographs the same location of the subject, and detects whether or not each of the data values ​​of the plurality of digital pixel data is an outlier from the pixel information of the plurality of pixels in the sensors in the plurality of stages that photograph the same location of the subject, thereby determining whether or not the digital pixel data is an outlier, and obtaining an outlier detection result that indicates whether or not the digital pixel data is an outlier; an outlier total number calculation unit that calculates, for each of a plurality of pixels in each stage of sensors in the optical sensor, the number of times that the outlier detection result indicates an outlier based on the outlier detection result obtained by the first outlier detection unit from a plurality of digital pixel data based on a plurality of pixel information for each of the plurality of pixels in the plurality of stages of sensors, as an outlier count; and a second outlier detection unit that compares, for each of a plurality of pixels in each stage of sensors in the optical sensor, the outlier count calculated by the outlier total number calculation unit with a threshold, and acquires pixel coordinates of pixels where the outlier count exceeds the threshold as pixel coordinates where the abrupt noise has occurred. a sudden noise detection device for detecting sudden noise occurring in the optical sensor, comprising:

4. a smoothing processing unit that performs a smoothing process in a time direction on digital pixel data based on pixel information of all pixels of the pixels of the plurality of stages of sensors of the optical sensor, and obtains smoothed pixel data; 4. The sudden noise detection device according to claim 3, wherein the digital pixel data on which the first outlier detection unit performs the outlier detection process is smoothed pixel data that has been smoothed by the smoothing unit.

5. 5. The sudden noise detection device according to claim 1, wherein the outlier detection unit detects an outlier by calculating a reference value, which is an average or median of data values ​​of pixel data based on pixel information of a plurality of pixels in a plurality of stages of sensors that photograph the same location of the subject, and a standard deviation, and determining that the pixel data value based on pixel information of a pixel of interest among the plurality of pixels has a difference from the reference value by a real number multiple of the standard deviation or more, and detecting the outlier.

6. an optical sensor in which sensors each having one line of a plurality of pixels arranged in a straight line are arranged in a plurality of stages, and each of the sensors arranged in a plurality of stages captures an image of the same location on the subject; a data temporary storage unit that temporarily stores pixel data based on pixel information of pixels photographed by sensors at each stage of the optical sensor; 6. The sudden noise detection device according to claim 1, wherein pixel data stored in the temporary data storage unit is extracted and processing is performed based on the extracted pixel data; an accumulation processing unit that corrects and accumulates pixel data indicated by the coordinates at which the sudden noise occurs, for data for a plurality of pixels photographed along the same line of the subject by the optical sensor and stored in the data temporary storage unit, obtains accumulated pixel data for pixels of one line of the subject, and outputs the accumulated pixel data as image data; a display unit that displays an image based on the image data from the cumulative processing unit; An imaging device comprising:

7. an outlier detection unit extracting digital pixel data based on pixel information of a plurality of pixels in a plurality of rows of sensors that have captured an image of the subject from an optical sensor in which sensors each having one line of a plurality of pixels arranged in a straight line are arranged in a plurality of rows, and each of the sensors arranged in a plurality of rows captures an image of the same location of the subject; the outlier detection unit detects whether or not each of the extracted digital pixel data values ​​is an outlier from the plurality of digital pixel data based on pixel information of a plurality of pixels in a plurality of stages of sensors that have captured an image of the same location of the subject, and determines whether or not the digital pixel data is an outlier, thereby obtaining an outlier detection result indicating whether or not the digital pixel data is an outlier; a step in which a sudden noise detection unit performs majority voting processing based on the outlier detection result obtained by the outlier detection unit for a plurality of digital pixel data based on a plurality of pixel information for a plurality of pixels in each stage of sensors in the optical sensor during a certain period of time extracted by the outlier detection unit in the relative traveling direction of the optical sensor and the subject, and obtains a majority voting processing result; When the result of the majority decision process determines that abrupt noise has occurred, the sudden noise detection unit acquires pixel coordinates indicated by the result of the majority decision process that determined that abrupt noise has occurred as sudden noise-occurring pixel coordinates; a sudden noise detection method for detecting sudden noise occurring in the optical sensor, comprising:

8. a smoothing processing step in which a smoothing processing unit performs a smoothing process in a time direction on digital pixel data based on pixel information of all pixels of the pixels of the sensors at multiple stages of the optical sensor, to obtain smoothed pixel data; 8. The method for detecting sudden noise according to claim 7, wherein the digital pixel data extracted in the pixel data extracting step is smoothed pixel data that has been subjected to the smoothing process in the smoothing process step.

9. a step in which a first outlier detection unit extracts digital pixel data based on pixel information of a plurality of pixels in a plurality of rows of sensors that have captured an image of the subject from an optical sensor in which sensors each having one line of a plurality of pixels arranged in a straight line are arranged in a plurality of rows, and each of the sensors arranged in a plurality of rows captures an image of the same location of the subject; the first outlier detection unit detects whether or not each data value of a plurality of digital pixel data based on pixel information of a plurality of pixels in a plurality of stages of sensors that have captured an image of the same location of the extracted subject is an outlier, and determines whether or not the digital pixel data is an outlier, thereby obtaining an outlier detection result indicating whether or not the digital pixel data is an outlier; an outlier count calculation step in which an outlier total count calculation unit in the sudden noise detection unit calculates, for each of a plurality of pixels in each stage of sensors in the multiple stages of sensors in the optical sensor, the number of times that the outlier detection result shows an outlier based on the outlier detection result obtained by the first outlier detection unit from a plurality of digital pixel data based on a plurality of pixel information for each of the plurality of pixels in the multiple stages of sensors; a step in which a second outlier detection unit in the sudden noise detection unit compares the number of outliers calculated by the total outlier number calculation unit with a threshold value for each of a plurality of pixels in each stage of sensors in the multiple stages of sensors in the optical sensor, and acquires pixel coordinates of pixels where the number of outliers exceeds the threshold value as sudden noise occurrence coordinates; a sudden noise detection method for detecting sudden noise occurring in the optical sensor, comprising:

10. a smoothing processing step in which a smoothing processing unit performs a smoothing process in a time direction on digital pixel data based on pixel information of all pixels of the pixels of the sensors at multiple stages of the optical sensor, to obtain smoothed pixel data; 10. The method for detecting sudden noise according to claim 9, wherein the digital pixel data extracted in the pixel data extracting step is smoothed pixel data that has been smoothed in the smoothing step.

11. a temporary storage step in which the data temporary storage unit temporarily stores pixel data based on pixel information of pixels captured by sensors in each stage of an optical sensor in which sensors for one line, each having a plurality of pixels arranged in a straight line, are arranged in a plurality of stages, and each of the sensors arranged in a plurality of stages captures the same location of the subject; a sudden noise detection step according to the sudden noise detection method of any one of claims 7 to 10, in which the sudden noise detection device extracts the pixel data temporarily stored in the temporary storage step, processes the extracted pixel data, and acquires coordinates of pixels where sudden noise occurs; an image data generating step in which an accumulation processing unit corrects and accumulates pixel data indicated by the coordinates at which the sudden noise occurs, for data for a plurality of pixels photographing the same line of the subject in the optical sensor, which data has been temporarily stored in the temporary storing step, to obtain accumulated pixel data for pixels of one line of the subject, and to generate image data; a display image data generating step in which a display unit generates image data for display based on the image data generated in the image data generating step; A display image data generating method comprising:

12. a step of extracting digital pixel data based on pixel information of a plurality of pixels in a plurality of rows of sensors that have captured an image of the subject from an optical sensor in which sensors each having one line of a plurality of pixels arranged in a straight line are arranged in a plurality of rows, and each of the sensors arranged in a plurality of rows captures an image of the same location of the subject; an outlier detection procedure for detecting whether or not each data value of a plurality of digital pixel data based on pixel information of a plurality of pixels in a plurality of stages of sensors that photographed the same location of the subject extracted by the extraction procedure is an outlier, determining whether or not the digital pixel data is an outlier, and obtaining an outlier detection result indicating whether or not the digital pixel data is an outlier; a majority decision processing procedure for performing a majority decision processing based on the outlier detection result obtained by the outlier detection procedure on a plurality of digital pixel data based on a plurality of pixel information for a plurality of pixels in each stage of sensors in the plurality of stages of sensors in the optical sensor, extracted by the extraction procedure, within a certain period of time traveling in the relative traveling direction of the optical sensor and the subject, to obtain a majority decision processing result; a pixel coordinate acquisition step of acquiring, when the majority decision processing step determines that abrupt noise has occurred as a result of the majority decision processing, pixel coordinates indicated by the majority decision processing result that determined that abrupt noise has occurred as abrupt noise occurrence coordinates; a sudden noise detection program for detecting a sudden noise occurring in the optical sensor, the program causing a computer to execute the above steps.

13. a step of extracting digital pixel data based on pixel information of a plurality of pixels in a plurality of rows of sensors that have captured an image of the subject from an optical sensor in which sensors each having one line of a plurality of pixels arranged in a straight line are arranged in a plurality of rows, and each of the sensors arranged in a plurality of rows captures an image of the same location of the subject; an outlier detection procedure for detecting whether or not each data value of a plurality of digital pixel data based on a plurality of pixel information for each of a plurality of pixels in a plurality of stages of sensors that photographed the same location of the subject extracted by the extraction procedure is an outlier, determining whether or not the digital pixel data is an outlier, and obtaining an outlier detection result indicating whether or not the digital pixel data is an outlier; a majority decision processing procedure for performing a majority decision processing based on the outlier detection result obtained by the outlier detection procedure on a plurality of digital pixel data based on pixel information of a plurality of pixels in each stage of sensors in the plurality of stages of sensors in the optical sensor, extracted by the extraction procedure, within a certain period of time traveling in the relative traveling direction of the optical sensor and the subject, to obtain a majority decision processing result; a pixel coordinate acquisition step of acquiring, when the majority decision processing step determines that abrupt noise has occurred as a result of the majority decision processing, pixel coordinates indicated by the majority decision processing result that determined that abrupt noise has occurred as abrupt noise occurrence coordinates; A recording medium storing a program for detecting sudden noise occurring in the optical sensor, the program causing a computer to execute the above.

Images