Processing apparatus, processing system, processing method, and program

Abstract

To reduce the time for detecting defective pixels.SOLUTION: A processing apparatus includes: a first detection processing unit which performs first detection processing to detect a first defective pixel by comparing a first threshold with first pixel information obtained from signals output from pixels of an image sensor; a first threshold modifying unit which repeatedly modifies the first threshold, using a first modification amount set at a predetermined ratio for an initial value of the first threshold until a first number of the detected first defective pixels becomes a predetermined value or lower, in the first detection processing; a second detection processing unit which performs second detection processing to detect a second defective pixel, which is different from the first defective pixel, by comparing second pixel information obtained from the output signals, the second pixel information being different from the first pixel information, with a second threshold obtained by using a second modification amount and the number of times that the first threshold is repeatedly modified; and a second threshold modifying unit which modifies the second threshold, using the second modification amount set at a predetermined ratio for the initial value of the first threshold until information on the sum of the first number of the detected first defective pixels and a second number of the detected second defective pixels becomes the predetermined value or lower.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The disclosed technology relates to a processing device, a processing system, a processing method, and a program. [Background technology]

[0002] An imaging device includes an image sensor (e.g., a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor), and obtains an image by performing photoelectric conversion using the image sensor. Normally, if the image sensor contains defective pixels, the image obtained as a video may be degraded. For this reason, image processing that detects and corrects defective pixels has been widely used in the past.

[0003] Patent document 1 describes a technology in which the signal output value of each pixel is compared with a threshold value to determine whether the output value is higher or lower than the threshold value, and the threshold value is changed until the number of determined pixels becomes a predetermined number or less. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-124036 A Summary of the Invention [Problem to be solved by the invention]

[0005] Nowadays, the number of pixels in an image sensor is increasing in order to obtain higher resolution images. In proportion to this, the number of defective pixels in an image sensor also tends to increase. However, since the amount of data that can be stored in memory for addresses determined to be defective pixels is limited, thorough examination is required to determine whether a pixel is defective, and the processing required can be very time-consuming.

[0006] The disclosed technology has been made to solve such problems in the past, and aims to provide a technology that makes it possible to shorten the time required to detect defective pixels. [Means for solving the problem]

[0007] A processing device according to one aspect of the disclosed technology includes: a first detection processing unit that performs a first detection process to detect a first defective pixel by comparing first pixel information obtained from an output signal of each pixel of an image sensor with a first threshold value; a first threshold value changing unit that repeatedly changes the first threshold value by using a first change amount that is set at a predetermined rate with respect to an initial value of the first threshold value, until a first detection number of the first defective pixels becomes a predetermined number or less in the first detection process; a second detection processing unit that performs a second detection process to detect a second defective pixel different from the first defective pixel by comparing second pixel information obtained from the output signal and different from the first pixel information with a second threshold value obtained by using the number of times the first threshold value is repeatedly changed and a second change amount; a second threshold change unit that changes the second threshold by using a second change amount set at a predetermined rate with respect to an initial value of the second threshold until added information of the first detection number of the first defective pixel and the second detection number of the second defective pixel becomes equal to or less than the predetermined number. Effect of the Invention

[0008] According to the disclosed technique, the processing time required to detect defective pixels can be reduced. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a configuration of a processing apparatus according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a processing apparatus according to a first embodiment. [Diagram 3] FIG. 11 is a block diagram showing the configuration of a processing apparatus according to a second embodiment. [Figure 4] FIG. 11 is a block diagram showing a modified example of the processing apparatus of the second embodiment. [Diagram 5]FIG. 13 is a block diagram showing the configuration of a processing apparatus according to a third embodiment. [Figure 6] 5 is a diagram for explaining the flow of defective pixel detection processing according to the first embodiment. [Figure 7] 13 is a diagram for explaining the flow of defective pixel detection processing according to the second embodiment. [Figure 8] 13 is a diagram for explaining the flow of defective pixel detection processing according to the third embodiment. [Figure 9] 13 is a diagram for explaining the flow of defective pixel detection processing according to the third embodiment. [Figure 10] FIG. 13 is a diagram for explaining the flow of defective pixel detection processing according to the third embodiment. [Figure 11] FIG. 4 is a diagram illustrating the process of a multi-frame smoothing unit. [Figure 12] FIG. 11 is a diagram showing a schematic diagram of a process of a multi-frame union unit. [Figure 13] FIG. 1 is a diagram showing a schematic configuration of a processing system including processing apparatuses according to first to third embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] (First embodiment) FIG. 1 is a block diagram showing the configuration of the processing device of the first embodiment. In FIG. 1, 100 is a camera head, 110 is a camera control unit, and 130 is a display device. FIG. 13 is a diagram showing a schematic configuration of a processing system STM including the processing device according to the first embodiment. The configuration of the processing system STM can be similarly applied to the second and third embodiments described later. The processing device 1 has a camera head 100 and a camera control unit 110. The display device 130 is configured to be able to display a signal output from the camera control unit 110. The light source 1301 irradiates a subject P with a predetermined light. The light source 1301 may include, for example, a white light source that irradiates visible light or an excitation light source that irradiates excitation light used in fluorescence observation. The camera head 100 is configured to receive reflected light from the subject P and perform imaging processing. The camera control unit 110 processes the imaging signal obtained from the camera head 100, and can also control the output (irradiation timing and intensity of irradiation light) of the light source 1301.

[0012] (camera head 100) Camera head 100 has an image sensor 101, and a signal output from image sensor 101 is input to camera control unit 110. As image sensor 101, for example, a CMOS (Complementary Metal Oxide Semiconductor) in which pixels are arranged two-dimensionally is assumed to be used, but other imaging devices such as a CCD (Charge Coupled Device) may also be used.

[0013] (Camera control unit 110) The camera control unit 110 has a signal processing unit 111 that processes a signal (hereinafter also referred to as an imaging signal) input from the camera head 100, a transmission unit 123 (output unit) that acquires information processed by the signal processing unit 111 and outputs it to the outside, and a storage unit 122 that stores the information processed by the signal processing unit 111. The storage unit 122 can store information set by a user via an operation unit in addition to storing information processed by the signal processing unit 111. The storage unit 122 can use various memories that can input (write) and output (read) information, and may be, for example, a RAM (Random Access Memory).

[0014] (Signal processing unit 111) A signal input from camera head 100 is input to signal processing unit 111. Signal processing unit 111 has a defective pixel correction unit 112 and a defective pixel detection unit 113, and the input signal is input to both defective pixel detection unit 113 and defective pixel correction unit 112. Defective pixel detection unit 113 performs a process of detecting defective pixels included in image sensor 101 based on the input signal, and defective pixel correction unit 112 performs a process of correcting an output signal for a defective pixel detected by defective pixel detection unit 113.

[0015] (Defective pixel detection unit 113) The defective pixel detection unit 113 includes a multi-frame smoothing unit 114 , a multi-frame union unit 115 , a first defective pixel detection unit 117 , a second defective pixel detection unit 116 , and a detected address generation unit 121 .

[0016] A signal input to the signal processing unit 111 at a predetermined frame rate is input to the multiple frame smoothing unit 114 and the multiple frame summing unit 115. For example, if the frame rate is N (fps: frame per second), a signal (imaging signal) of N frames is input to the multiple frame smoothing unit 114 and the multiple frame summing unit 115 as multiple frames.

[0017] (Multi-frame smoothing unit 114) 11 is a diagram illustrating a schematic diagram of the process of the multi-frame smoothing unit 114. The multi-frame smoothing unit 114 generates a smoothed image by smoothing pixel information output from each pixel over multiple frames (e.g., N frames), and outputs the generated smoothed image to the first defective pixel detection unit 117.

[0018] The multi-frame smoothing unit 114 acquires pixel information by taking an arithmetic average of pixel information (e.g., pixel values) output from each pixel for the input multiple frames. The multi-frame smoothing unit 114 then acquires the averaged pixel information as smoothed pixel information, and acquires (generates) a smoothed image in which the smoothed pixel information acquired at each pixel position is set in one frame. The multi-frame smoothing unit 114 then outputs the acquired smoothed image to the first defective pixel detection unit 117.

[0019] For example, in Fig. 11, pixel information at pixel position (x, y) in the i-th frame (Fi) is denoted as Pi. The multi-frame smoothing unit 114 acquires pixel information (P1 to Pn) at pixel position (x, y) in multiple frames (F1 to Fn), and acquires image information Pave(x, y) obtained by averaging the pixel information (P1 to Pn) as smoothed pixel information at the pixel position (x, y). The multi-frame smoothing unit 114 then acquires a smoothed image 1101 in which the smoothed pixel information is arranged in one frame.

[0020] A plurality of frames may include frames with high noise components and frames with low noise components, but unless a pixel is a defective pixel, the smoothed pixel information will eventually converge to a level within a predetermined range.

[0021] However, if a pixel is defective, the signal level will be offset by the effect of the pixel defect (flaw), and when an arithmetic average is taken, the signal level will be high or low for only that defective pixel.

[0022] A first defective pixel detection unit 117, which will be described later, detects defective pixels by comparing smoothed pixel information of each pixel in the smoothed image 1101 with a threshold value (first threshold value).

[0023] By acquiring smoothed images over multiple frames in the multi-frame smoothing unit 114, for example, defective pixels that have a high signal level and always exist as bright spots can be more easily detected by the first defective pixel detection unit 117 described later.

[0024] (Multiple Frame Union Part 115) 12 is a diagram showing a schematic diagram of the process of the multiple frame union unit 115 (hereinafter also referred to as a maximum pixel value acquisition unit). The multiple frame union unit 115 acquires image information (e.g., pixel value) of each pixel output from the image sensor 101 over multiple frames, and acquires (generates) a maximum pixel information image (hereinafter also referred to as a maximum pixel value image) of one frame in which the maximum value of the image information of each pixel is set at the pixel position of each pixel. Then, the multiple frame union unit 115 outputs the acquired maximum pixel information image to the second defective pixel detection unit 116.

[0025] For example, in FIG. 12, pixel information at pixel position (x, y) in the i-th frame (Fi) is Pi. The multiple frame summing unit 115 compares pixel information (P1 to Pn) at pixel positions (x, y) across multiple frames. If the pixel information (Pi) of the i-th frame (Fi) is the maximum value, the multiple frame summing unit 115 sets the pixel information (Pi) at pixel position (x, y) of the i-th frame (Fi) as maximum pixel information Pmax(x, y). The multiple frame summing unit 115 similarly compares image information at other pixel positions, and acquires a maximum pixel information image 1201 of one frame in which the maximum pixel information Pmax is set at each pixel position.

[0026] A second defective pixel detection unit 116, which will be described later, detects defective pixels by comparing the maximum pixel information of each pixel in the maximum pixel information image 1201 with a threshold value (a second threshold value).

[0027] By obtaining the maximum pixel information image among multiple frames in the multiple frame union unit 115, it becomes easier for the second defective pixel detection unit 116, which will be described later, to detect, for example, a defective pixel that has the highest signal level among the multiple frames and that blinks at a given time.

[0028] (First defective pixel detection unit 117) The first defective pixel detection unit 117 includes a threshold setting change unit 119a (hereinafter also referred to as a first threshold setting change unit) and a detection processing unit 120a (hereinafter also referred to as a first detection processing unit).

[0029] The first defective pixel detector 117 detects defective pixels by comparing the input signal (smoothed pixel information of each pixel) with a threshold value (first threshold value), and outputs the defective pixel detection result to the detected address generator 121.

[0030] The detection processing unit 120a (first detection processing unit) performs a first detection process to detect, as a first defective pixel, a pixel for which first pixel information (smoothed pixel information) obtained from an output signal of each pixel of the image sensor 101 exceeds a first threshold value THA. The first defective pixel is, for example, a defective pixel that always exists as a bright spot, and will also be referred to as a defective pixel A in the following description.

[0031] Threshold setting change unit 119a (first threshold setting change unit) repeatedly changes the setting of the first threshold by the first detection process in which the first threshold is changed, until the first detection number NA of first defective pixels becomes equal to or less than a predetermined number.

[0032] Here, the change in the first threshold is performed by setting the initial first threshold to THA and changing the threshold by ΔT A Let the number of times the change is repeated be N1. If the change is repeated N1 times, the cumulative change amount of the first threshold for N1 times is ΔT A ×N1.

[0033] In this case, the first threshold value can be expressed by the following formula (1).

[0034] THA = THA (initial value) + ΔT A ×N1 (1) Here, the change amount ΔT of the first threshold A can be set to a predetermined ratio (for example, m1% of the first threshold THA (initial value)) with respect to the first threshold THA (initial value).

[0035] In the memory unit 122 (RAM), the number of detections in which address information of defective pixels that can be stored is denoted as S1. The threshold setting change unit 119a repeatedly changes the setting of the first threshold until the first detection number NA, which is the number of detections of defective images detected by the first detection process in which the first threshold has been changed, becomes equal to or less than a predetermined number (equal to or less than S1).

[0036] (Second defective pixel detection unit 116) The second defective pixel detection section 116 includes a threshold setting change section 119b (hereinafter also referred to as a second threshold change section) and a detection processing section 120b (hereinafter also referred to as a second detection processing section).

[0037] The second defective pixel detector 116 detects defective pixels by comparing the input signal (maximum pixel information of each pixel) with a threshold value (second threshold value), and outputs the defective pixel detection result to the detected address generator 121.

[0038] The detection processing unit 120b (second detection processing unit) detects that second pixel information (maximum pixel information) obtained from the output signal of the image sensor 101 and different from the first pixel information (smoothed pixel information) is the first threshold change information (ΔT A Then, a second detection process is performed to detect pixels exceeding a second threshold value THB obtained by using the second threshold value THB × N1 as second defective pixels different from the first defective pixels. The second defective pixel is, for example, a defective pixel that blinks at any time, and will be referred to as defective pixel B in the following description.

[0039] The threshold setting change unit 119b (second threshold change unit) further changes the first threshold THA to (THA+ΔT A ) the first detection number (NA_A) obtained by the first detection process and the change amount ΔT B Add further to change the second threshold THB (THB+ΔTB The setting of the second threshold is repeatedly changed until the sum information (total number of defective pixels) of the second detection number (NB_B) obtained by the second detection process (NB_B) and the sum information (total number of defective pixels) becomes a predetermined number or less (for example, a predetermined number S2 or less).

[0040] Here, the change in the second threshold is performed by setting the initial second threshold to THB and changing the threshold by ΔT B If the number of times the change is repeated is N2, then the change amount of the second threshold for N2 times is ΔT B ×N2.

[0041] In this case, the second threshold value can be expressed by the following formula (2).

[0042] THB = THB (initial value) + ΔT B ×N2 (2) Change in the second threshold ΔT B can be set to a predetermined ratio (e.g., m2% of the second threshold THB (initial value)) with respect to the second threshold THB (initial value). In addition, in the storage unit 122 (RAM), the number of detections in which address information of defective pixels can be stored is set to a predetermined number S2 (≦S1).

[0043] The threshold setting change unit 119b changes the first threshold THA by a second detection number NB_B, which is the number of defect images detected by the second detection process in which the second threshold is changed, and the first detection number NB_B by a further change (THA+ΔT A The setting and changing of the second threshold is repeated until the total number of defective pixels, calculated by adding the first detection number NA_A, which is the number of detections of the defective images obtained by the first detection process performed in the above step 121, becomes equal to or less than the detection number S2 at which address information can be stored in the memory unit 122.

[0044] In order to make the sensitivity of defective pixels uniform, when the second threshold THB is changed and the second detection process is repeated in the threshold setting change unit 119b, the first threshold THA is changed and the first detection process is repeated in the threshold setting change unit 119a.

[0045] As the first threshold THA and the second threshold THB are changed and increased, the total number of defective pixels detected that exceed each threshold decreases. In order to make the sensitivity of defective pixels uniform, the first threshold THA and the second threshold THB are changed in conjunction with each other. The setting change of the first threshold THA and the second threshold THB and the first detection process and the second detection process are repeated until the address information of all detected defective pixels converges to a detection number Nmax or less that can be stored in the memory unit 122.

[0046] The storage unit 122 (RAM) stores address information of pixels determined to be defective pixels, which is generated by the detected address generation unit 121 .

[0047] Defective pixel correction unit 112 corrects the signal output from image sensor 101 based on the defective pixel detection result (address information) output from defective pixel detection unit 113. The defective pixel correction process by defective pixel correction unit 112 may be a process of interpolating pixel information (pixel value) of a pixel detected as a defective pixel using pixel information (pixel values) of surrounding normal pixels, or of subtracting a predetermined offset value. Defective pixel correction unit 112 outputs the corrected signal to transmission unit 123.

[0048] The video signal output from the signal processing unit 111 is output as an output signal of the camera control unit 110 via the transmission unit 123, and is input to the display device 130 for display.

[0049] (Configuration of display device 130) The camera control unit 110 of this embodiment is connected to a display device 130 via a transmission unit 123. The display device 130 has a display unit 131, a display control unit 132, and an operation unit 133. The display unit 131 is configured with, for example, a liquid crystal display or an organic EL display.

[0050] The display control unit 132 performs display control to cause the display unit 131 to display various information including the output signal acquired from the camera control unit 110. For example, the display control unit 132 performs display control to cause the display unit 131 to display image information captured by the camera head 100 based on the output signal acquired from the camera control unit 110.

[0051] The operation unit 133 is configured with, for example, a dial, a mouse, operation buttons, etc., and inputs various instructions from the user. The display unit 131 and the operation unit 133 may be realized as an integrated touch panel.

[0052] The display control unit 132 can also perform display control to display a screen (user interface screen) for setting external setting parameters for the camera control unit 110 on the display unit 131, based on a user's input via the operation unit 133. The external setting parameters include, for example, an initial value of the threshold (first threshold THA and second threshold THB), a rate of change in the threshold (m1% of the first threshold THA (initial value), m1% of the second threshold THB (initial value)), etc., and the user can set various parameters via the operation unit 133.

[0053] (Relationship between defective pixels and threshold) Next, a description will be given of the defective pixel detection unit 113 and the threshold value. A defective pixel is a pixel that does not output a normal signal, such as a pixel that outputs a signal of a fixed level that is added regardless of the subject, or a pixel whose output differs from a predetermined level for a uniform amount of light.

[0054] Therefore, in the case where the defective pixel detection unit 113 (first defective pixel detection unit 117, second defective pixel detection unit 116) is targeting, for example, bright spots where the output from the pixel is greater than a predetermined level, the higher the threshold value set, the more difficult it may be to detect defective pixels with low signal levels. As a result, the number of detected defective pixels decreases.

[0055] It is required to set the threshold as low as possible so that as many defective pixels as possible can be detected so that they are not recognized as defective pixels in the output image displayed on the display device 130. On the other hand, the memory area of ​​the storage unit 122 is finite, and if the threshold is set low, the number of pixels detected as defective pixels may exceed a predetermined number, and the address information of the detected defective pixels stored in the storage unit 122 may exceed the memory area of ​​the storage unit 122. In such a case, it is required to set the threshold so that as many defective pixels as possible can be detected without exceeding the memory area.

[0056] The above-mentioned defective pixels are not limited to bright spots whose output signal from the pixel is greater than a predetermined level, but may be black spots whose output signal from the pixel is smaller than a predetermined level, in which case the number of defective pixels detected as defective pixels may increase if the threshold is set higher. Therefore, when black spots are targeted as defective pixels, the threshold during repeating may be changed to lower the threshold setting by subtracting a predetermined change amount.

[0057] (Detection according to type of defective pixel) Next, the processing of defective pixel detection unit 113 according to the type of defective pixel will be described. Assume that defective pixel A (hereinafter also referred to as a first defective pixel) and defective pixel B (hereinafter also referred to as a second defective pixel) of different types are present in the imaging signal of each pixel of image sensor 101 in a light-shielded state. For example, assume that defective pixel A is a defective pixel that always exists as a bright spot, and defective pixel B is a defective pixel that flashes at any time.

[0058] The first defective pixel detection unit 117 of this embodiment has the detection processing unit 120a and threshold setting change unit 119a described above, and inputs the smoothed image 1101 generated by the multi-frame smoothing unit 114, and performs a process (hereinafter also referred to as the first detection process) of detecting defective pixel A (first defective pixel) by comparing the smoothed pixel information (pixel value) of each pixel in the smoothed image 1101 with a first threshold value THA.

[0059] The second defective pixel detection unit 116 has the detection processing unit 120b and threshold setting change unit 119b described above, and inputs the maximum pixel information image 1201 generated by the multiple frame union unit 115, and performs a process (hereinafter also referred to as the second detection process) of detecting defective pixel B (second defective pixel) by comparing the maximum pixel information (pixel value) of each pixel in the maximum pixel information image 1201 with a second threshold value THB.

[0060] The output level of defective pixels is not output as is as a video output, but can be affected by various other image processing settings (for example, gain processing or edge enhancement processing). Also, the level at which a person can see a defective pixel also differs depending on the environment (for example, the visibility level also changes depending on color information such as R (red), G (green), and B (blue)). Here, the relationship between visibility and defective pixels is called sensitivity.

[0061] In order to equalize the sensitivity at which a signal is determined to be a defective pixel, the thresholds THA and THB used in the first detection process and the second detection process must be set to have equal sensitivities. Therefore, when the first detection process is performed by the first defective pixel detection unit 117, if the number of detected defective pixels A exceeds the upper limit of the number of data items (hereinafter referred to as the "predetermined number") that can be stored in the storage unit 122 and defective pixels A are acquired, the threshold THA is changed by a certain amount and the first detection process is performed again by the first defective pixel detection unit 117. At this time, in order to keep the sensitivity equal, the threshold THB must also be changed by a certain amount and the second detection process is performed by the second defective pixel detection unit 116.

[0062] (Processing flow) 6 is a flowchart illustrating the flow of the defective pixel detection process according to the first embodiment. Hereinafter, the flow of the defective pixel detection process according to the first embodiment will be described with reference to FIG.

[0063] In step S201, the number of times N1 to repeat the defective pixel detection process is initialized to 0. Here, the initial values ​​of the first threshold THA and the second threshold THB, the change amount ΔT A , ΔT BThese parameters may be stored in advance in the storage unit 122, or may be set by the user via the operation unit 133.

[0064] In step S202, detection processing unit 120a (first detection processing unit) executes a first detection process, and obtains the number NA of defective pixels A and position information of the detected defective pixels A.

[0065] In step S203, threshold setting change unit 119a (first threshold setting change unit) determines whether the number of defective pixels NA detected in step S202 exceeds a predetermined number S1. If the number of defective pixels NA exceeds the predetermined number S1 (S203-NO), the process proceeds to step S204. Here, the predetermined number S1 is the upper limit number of data that can be stored in storage unit 122.

[0066] In step S204, the threshold setting change unit 119a changes the first change amount (ΔT A The threshold setting change unit 119a repeatedly changes the first threshold THA by a change amount ΔT A is added to increase the setting of the first threshold THA.

[0067] In step S205, the threshold setting change unit 119a increments the number of repetitions N1 by 1 (N1=N1+1). After that, the process returns to step S202, and the detection processing unit 120a executes the first detection process again. Steps S202 to S205 are repeated until the number of defective pixels NA becomes equal to or less than the predetermined number S1 in step S203 (S203-YES).

[0068] In step S203, if the number of defective pixels NA is equal to or smaller than the predetermined number S1 (S203-YES), the threshold setting change unit 119a advances the process to step S206.

[0069] In step S206, the threshold setting change unit 119b (second threshold setting change unit) calculates the number of repetitions N1 (number of repetitions N1) performed in the first detection process and the change amount ΔT B The change amount using (repeat count N1 × ΔT B ) is added to the second threshold THB to change the setting of the second threshold THB.

[0070] By setting the second threshold THB using the number of repetitions N1 performed in the first detection process, the number of times the second detection process is performed corresponding to the cumulative change amount of the second threshold THB can be reduced in the defective pixel detection process by the detection processing unit 120b. Also, by setting the second threshold THB using the number of repetitions N1 performed in the first detection process, the sensitivity for determining a defective pixel can be aligned with the first threshold THA.

[0071] Thereafter, in step S207, detection processor 120b (second detection processor) executes a second detection process to obtain the number NB of defective pixels B and position information of the detected defective pixels B.

[0072] In step S208, threshold setting change unit 119b obtains the total number of defective pixels by adding the number NB of defective pixels B detected by detection processing unit 120b and the number NA of defective pixels A detected by detection processing unit 120a. First defective pixel detection unit 117 and second defective pixel detection unit 116 are connected to each other so that they can communicate with each other, and can share threshold settings and defective pixel detection results with each other.

[0073] In step S209, threshold setting change unit 119b determines whether the total number of defective pixels acquired in step S208 exceeds predetermined number S2. If the total number of defective pixels exceeds predetermined number S2 (S209-NO), the process proceeds to step S210. Here, predetermined number S2 may be equal to predetermined number S1 or may be smaller than predetermined number S1, so long as it is equal to or smaller than the upper limit number of data (predetermined number S1) that can be stored in storage unit 122.

[0074] In step S210, the threshold setting change unit 119a calculates the change amount (ΔT A ) is added to the first threshold THA to change the setting of the first threshold THA.

[0075] In step S211, detection processing unit 120a executes a first detection process to obtain the number NA of defective pixels A and position information of the detected defective pixels A.

[0076] In step S212, the threshold setting change unit 119b changes the second threshold by a predetermined ratio (ΔT B ) to change the second threshold value. The threshold value setting change unit 119b changes the second threshold value by a change amount ΔT B to the second threshold THB to change the setting of the second threshold THB.

[0077] In step S213, detection processor 120b (second detection processor) executes a second detection process, and obtains the number NB of defective pixels B and position information of the detected defective pixels B.

[0078] Steps S208 to S213 are repeated until the total number of defective pixels becomes equal to or less than the predetermined number S2 in step S209.

[0079] In step S209, the threshold setting change unit 119b determines whether the total number of defective pixels acquired in step S208 exceeds a predetermined number S2, and if the total number of defective pixels is equal to or less than the predetermined number S2 (S209-YES), the process proceeds to step S214.

[0080] In step S214, detection address generation unit 121 acquires position information of defective pixel A detected by detection processing unit 120a and position information of defective pixel B detected by detection processing unit 120b, and generates address information combining the position information of defective pixel A and the position information of defective pixel B.

[0081] The detection address generation unit 121 may add identification information indicating the type of defective pixel to the address information so that the type of defective pixel A of the position information obtained from the detection processing unit 120a can be distinguished from the type of defective pixel B of the position information obtained from the detection processing unit 120b.

[0082] By referring to the identification information in the address information, the defective pixel correction unit 112 can identify whether the pixel is defective pixel A (a defective pixel that always exists as a bright spot) or defective pixel B (a defective pixel that blinks at any time).

[0083] In step S215, defective pixel correction unit 112 acquires the address information generated in step S214 and identifies the position of the defective pixel. Then, defective pixel correction unit 112 corrects the output signal of the pixel detected as a defective pixel, and ends the process. Note that when correcting the output signal of the defective pixel, defective pixel correction unit 112 may identify the type of the defective pixel by referring to the identification information, and correct the output signal of the defective pixel according to the type of the defective pixel.

[0084] In the process flow of FIG. 6, the flow when a bright spot occurs as a defective pixel is explained, but the process flow is similar when a black spot occurs as a defective pixel. In that case, the threshold value at the time of repeat is a predetermined change amount (ΔT A , ΔT B ) to change the settings of the thresholds (first threshold THA, second threshold THB) to lower values, and the light irradiated onto the subject may be uniform light.

[0085] In the conventional technology, when the total number of detected defective pixels exceeds a predetermined number S2, the processing in first defective pixel detection unit 117 and second defective pixel detection unit 116 is repeated using the same threshold change amount Δ until the total number of detected defective pixels becomes equal to or less than the predetermined number S2.

[0086] Here, as in the conventional technology, the time required for the multi-frame smoothing unit 114 to execute the first detection process on the image acquired using the same threshold change amount Δ is denoted as TA, and the time required for the multi-frame union unit 115 to execute the second detection process is denoted as TB. If the number of repetitions is Nrep, then the time T1 required for the total number of defective pixels to become equal to or less than a predetermined number S2 can be expressed by the following formula (3).

[0087] T1 = (TA + TB) × Nrep (3) In the defect detection process in the first embodiment, when the number of defective pixels detected by the first detection process exceeds a predetermined number S1, the first detection process is repeatedly performed by the first defective pixel detection unit 117 until the number of defective pixels detected falls below the predetermined number S1, and the number of repetitions (repeat count) N1 is obtained.

[0088] When the second defective pixel detection unit 116 performs the second detection process, the second threshold THB is set to a cumulative change amount (ΔT B ×N1), the second defective pixel detection unit 116 starts the second detection process. This enables the second defective pixel detection unit 116 to reduce the number of times the second detection process is performed, which corresponds to the accumulated change amount of the second threshold THB. In the defective pixel detection process of this embodiment, the time T2 required for defective pixel detection using a combination of the first defective pixel detection unit 117 and the second defective pixel detection unit 116 can be expressed by the following equation (4).

[0089] T2 = TA × N1 + (TA + TB) × N2 (4) Here, when the first defective pixel detector 117 is executed, if the predetermined number S1 is set to be equal to or larger than the predetermined number S2 (S1≧S2), the number of repetitions N1 becomes equal to or smaller than the number of repetitions N in the conventional technique (N1≦N).

[0090] In the second defective pixel detection unit 116, the second threshold THB at the time of starting the second detection process is set to a value obtained by performing the second detection process N1 times (=amount of change ΔT BThe total number of repeats Nsum when the total number of detected defective pixels becomes equal to or smaller than the predetermined number S2 satisfies the relationship of the following formula (5).

[0091] Nsum = N1 + N2 (5) According to the first embodiment, the number of times the second detection process is performed in the second defective pixel detection unit 116 can be reduced by the number of times N1 the process was repeated in the first defective pixel detection unit 117, so that T2≦T1 is satisfied, and the total defect detection time can be shortened.

[0092] It should be noted that the present embodiment is not limited to the configuration of the above-described embodiment, and the components can be modified and embodied without departing from the spirit and scope of the present invention.

[0093] In addition, in this embodiment, two types of defective pixels, defective pixel A and defective pixel B, have been described as examples of defective pixel types. However, the configuration of this embodiment is not limited to two types of defective pixels, and the defective pixel detection process of this embodiment can be similarly applied to two or more types of defective pixels.

[0094] Fig. 2 is a block diagram showing the configuration of a modified example of the processing device of the first embodiment. As shown in Fig. 2, by providing Nset or more (Nset≧2) combinations of a multiframe smoothing unit 114 and a first defective pixel detection unit 117, and a multiframe union unit 115 and a second defective pixel detection unit 116 inside the defective pixel detection unit 113, it is possible to perform processing corresponding to various types of defective pixels.

[0095] Second embodiment 3 is a block diagram showing the configuration of the second embodiment. The camera head 100 of the second embodiment has an image sensor R 102, an image sensor G 103, and an image sensor B 104. Signals are output from each image sensor and input to a signal processing unit 111.

[0096] The camera head 100 is provided with a prism (not shown), which splits (separates) light input to the camera head 100 (for example, reflected light from the subject P) into the three primary colors of red light, green light, and blue light. Hereinafter, red light will be referred to as R (red) light, green light as G (green) light, and blue light as B (blue) light. The image sensor R102 photoelectrically converts the R (red) light to generate an R imaging signal, which is an electrical signal. The image sensor G103 photoelectrically converts the G (green) light to generate a G imaging signal, which is an electrical signal. The image sensor B104 photoelectrically converts the B (blue) light to generate a B imaging signal, which is an electrical signal. Hereinafter, the R imaging signal, G imaging signal, and B imaging signal will be collectively referred to as RGB imaging signals.

[0097] In this embodiment, the imaging mode MR is a mode in which imaging is performed by the image sensor R102, the imaging mode MG is a mode in which imaging is performed by the image sensor G103, and the imaging mode MB is a mode in which imaging is performed by the image sensor B 104. The multiple imaging modes include an imaging mode for observing a subject using RGB imaging signals separated from visible light.

[0098] It should be noted that this embodiment is not limited to the configuration of the embodiment described above. For example, the multiple imaging modes are not limited to three imaging modes as in the case of RGB imaging signals, and for example, an image sensor IR105 for fluorescence observation may be provided in the camera head 100 as shown in Fig. 4, and four imaging modes may be provided. That is, an imaging mode (fluorescence observation mode) for observing fluorescence emitted from a fluorescent substance present in a subject is also included.

[0099] The image sensors 102 to 105 output imaging signals corresponding to a plurality of imaging modes as output signals. The threshold setting change unit 119a (first threshold changing unit) sets a different first threshold for each imaging signal, and the threshold setting change unit 119b (second threshold changing unit) sets a different second threshold for each imaging signal.

[0100] 13, for example, in addition to a white light source that irradiates visible light, an excitation light source (light source for fluorescence observation) for observing fluorescence emitted from a fluorescent substance present in the subject P may be provided. The subject P may be irradiated with light from the excitation light source, and the fluorescence generated from the fluorescent substance present in the subject P may be processed by the image sensor IR105.

[0101] As an imaging mode, when an image sensor IR105 for fluorescence observation is provided as shown in FIG. 4, different sensitivity modes can be set for the image sensor IR105 in the same manner.

[0102] The camera head 100 inputs an RGB imaging signal to a signal processing unit 111. In the signal processing unit 111, the RGB imaging signal is input to a defective pixel detection unit 113 and a defective pixel correction unit 112. In this embodiment, it is assumed that bright points are determined to be defective pixels. The functional configuration of the signal processing unit 111 is the same as in the first embodiment, so a duplicated description will be omitted.

[0103] In this embodiment, the defective pixel detection unit 113 performs a defective pixel detection process for each imaging signal based on the input RGB imaging signal. That is, the multi-frame smoothing unit 114 obtains pixel information by averaging pixel information (e.g., pixel values) output from each pixel for multiple frames of each imaging signal of the input RGB imaging signal. The pixel information obtained by averaging is used as smoothed pixel information. Then, a smoothed image is obtained (generated) by setting the smoothed pixel information obtained at each pixel position.

[0104] In addition, the multiple frame union unit 115 acquires image information (pixel value) of each pixel across multiple frames of each of the input RGB imaging signals, and acquires (generates) a maximum pixel information image (maximum pixel value image) in which the maximum value of the image information of each pixel is set at the pixel position of each pixel.

[0105] (Process flow 2) 7 is a flowchart illustrating the flow of defective pixel detection processing according to the second embodiment. The flow of defective pixel detection processing according to the second embodiment will be described below with reference to FIG.

[0106] In step S301, the defective pixel detection unit 113 executes the defective pixel detection process R in the imaging mode MR to obtain the number of detected defective images Nr. In the defective pixel detection process R, the detection processing unit 120a and the detection processing unit 120b execute the processes (first detection process and second detection process) described in Fig. 6 on the R imaging signal to obtain the number of detected defective images Nr.

[0107] In step S302, the defective pixel detection unit 113 executes the defective pixel detection process G in the imaging mode MG to obtain the number of detected defective images Ng. In the defective pixel detection process R, the detection processing unit 120a and the detection processing unit 120b execute the processes (first detection process and second detection process) described in Fig. 6 on the G imaging signal to obtain the number of detected defective images Ng.

[0108] In step S303, the defective pixel detection unit 113 executes the defective pixel detection process B in the imaging mode MB to obtain the number of detected defective images Nb. In the defective pixel detection process B, the detection processing units 120a and 120b execute the processes (first detection process and second detection process) described in Fig. 6 on the B imaging signal to obtain the number of detected defective images Nb.

[0109] Next, in step S304, the threshold setting change section 119b (second threshold change section) acquires the result of adding up the numbers of defective pixels Nr, Ng, and Nb in each imaging mode at each pixel position as the total number of defective pixels.

[0110] In step S305, the threshold setting change unit 119b determines whether the total number of defective pixels is equal to or less than a predetermined number (e.g., the predetermined number S2). If the total number of defective pixels exceeds the predetermined number (S305-NO), the threshold setting change unit 119b advances the process to step S306.

[0111] In step S306, the threshold setting change unit 119b judges whether the imaging mode with the largest number of defective pixels detected among the numbers of defective pixels Nr, Ng, and Nb is the imaging mode MR. If the imaging mode with the largest number of defective pixels is the imaging mode MR (S306-YES), the threshold setting change unit 119b advances the process to step S307. The threshold setting change unit 119b changes the threshold value THr of the imaging mode MR by the change amount ΔT r Here, the threshold value THr of the imaging mode MR is the threshold value THr (the first threshold value THr and the second threshold value THr) at the end of step S301.

[0112] In step S308, the detection processing units 120a and 120b execute the defective pixel detection process R in the imaging mode MR once using the changed threshold value THr, and the threshold setting change unit 119b obtains the number of defective pixels Nr based on the defective pixel detection process R that has been executed again, and returns the process to step S304.

[0113] On the other hand, if it is determined in the determination process of step S306 that the imaging mode with the largest number of detected defective pixels Nr, Ng, Nb is not the imaging mode MR (S306-NO), the defective pixel detection unit 113 advances the process to step S309.

[0114] In step S309, the threshold setting change unit 119b judges whether the imaging mode with the largest number of defective pixels detected among the numbers of defective pixels Nr, Ng, and Nb is the imaging mode MG. If the imaging mode with the largest number of defective pixels is the imaging mode MR (S309-YES), the threshold setting change unit 119b advances the process to step S310. The threshold setting change unit 119b changes the threshold THg of the imaging mode MG by the change amount ΔT g Here, the threshold value THg of the imaging mode MG is the threshold value THg (the first threshold value THg and the second threshold value THg) at the end of step S302.

[0115] In step S311, the detection processing unit 120a and the detection processing unit 120b execute the defective pixel detection process G in the imaging mode MG once again using the changed threshold value THg, and the threshold setting change unit 119b obtains the number of defective pixels Ng based on the defective pixel detection process G that has been executed again, and returns the process to step S304.

[0116] On the other hand, if it is determined in the determination process of step S309 that the imaging mode with the largest number of detected defective pixels Nr, Ng, Nb is not the imaging mode MG (S309-NO), the threshold setting change unit 119b advances the process to step S312.

[0117] In step S312, the threshold setting change unit 119b changes the threshold value THb of the imaging mode MB by a change amount ΔT b Here, the threshold value THb of the imaging mode MB is the threshold value THb (the first threshold value THb and the second threshold value THb) at the end of step S303.

[0118] In step S313, the detection processing units 120a and 120b execute the defective pixel detection process B in the imaging mode MB once again using the changed threshold value THb, and the threshold setting change unit 119b obtains the number of defective pixels Nb based on the defective pixel detection process B that has been executed again, and the process returns to step S304.

[0119] In step S305, if the total number of defective pixels is equal to or less than the predetermined number (predetermined number S2) (S305-YES), the threshold setting change unit 119b advances the process to step S314.

[0120] In step S314, the detection address generation unit 121 obtains position information of a defective pixel A(r) in the R imaging signal, a defective pixel A(g) in the G imaging signal, and a defective pixel A(b) in the B imaging signal, which are detected by the detection processing unit 120a. Here, the defective pixels A(r), A(g), and A(b) refer to defective pixels that always exist as bright points in the RGB imaging signals.

[0121] Furthermore, the detection address generation unit 121 acquires position information of a defective pixel B(r) in the R imaging signal, a defective pixel B(g) in the G imaging signal, and a defective pixel B(b) in the B imaging signal, which are detected by the detection processing unit 120b. Here, the defective pixels B(r), B(g), and B(b) indicate defective pixels that blink at any time in the RGB imaging signals.

[0122] In step S315, defective pixel correction unit 112 acquires the address information generated in step S314, and identifies the position of the defective pixel by referring to the position information of the defective pixel. Then, defective pixel correction unit 112 corrects the output signal of the pixel detected as the defective pixel, and ends the process.

[0123] When correcting the output signal of a defective pixel, the defective pixel correction unit 112 may identify the type of the defective pixel by referring to the identification information, and correct the output signal of the defective pixel according to the type of the defective pixel.

[0124] In the process flow 2 in Fig. 7, the process flow when a bright spot occurs as a defective pixel is explained, but in the case of a black spot, the process flow is similar to the process flow 2. In that case, the threshold value during the repeat can be changed to a lower threshold value by subtracting a predetermined change amount.

[0125] In the second embodiment, when the imaging signal output from camera head 100 includes multiple imaging signals, defective pixel detection processing (first detection processing, second detection processing) is performed for each of the multiple imaging signals, and the numbers of defective pixels Nr, Ng, and Nb are obtained for each of the multiple imaging modes. When the total number of defective pixels is equal to or greater than a predetermined number, the imaging mode with the largest number of defective pixels is selected, a predetermined change amount is added to the threshold value in the defective pixel detection processing to raise the threshold value, the total number of defective pixels is recalculated, and the defective pixel detection processing (first detection processing, second detection processing) is repeatedly performed until the total number of defective pixels becomes equal to or less than the predetermined number.

[0126] According to the second embodiment, even if the total number of defective pixels exceeds a predetermined number, it is no longer necessary to repeat the defective pixel detection process in all imaging modes. Therefore, the time required for the defective pixel detection process can be reduced compared to the prior art, in which there was only one threshold value regardless of the imaging mode.

[0127] Furthermore, compared to the conventional technique in which a single threshold is used to determine whether the total number of defective pixels exceeds a predetermined value, the process of the second embodiment allows the threshold to be set finely. Therefore, the number of pixels determined to be defective can be reduced by raising the threshold for only a certain imaging mode, making it possible to prevent an excessive reduction in the number of defective pixels. Note that this embodiment is not limited to the configuration of the above-described embodiment, and the components can be modified and embodied without departing from the spirit and scope of the present invention.

[0128] The imaging mode selected from among the multiple imaging modes may be an imaging mode selected according to a predetermined order among imaging modes with one or more detections. For example, a user may set a priority order and select an imaging mode according to the set order. The imaging mode selected may be an imaging mode with the fewest number of detected defective pixels among the multiple imaging modes. Furthermore, the imaging mode selected may be an imaging mode with the largest number of detected unique defective pixels among the multiple imaging modes. Furthermore, the imaging mode selected may be an imaging mode with the smallest number of detected unique defective pixels among the multiple imaging modes.

[0129] Third embodiment 3 is a block diagram showing the configuration of the third embodiment. The camera head 100 of the third embodiment has an image sensor R 102, an image sensor G 103, and an image sensor B 104. Signals are output from each image sensor and input to a signal processing unit 111.

[0130] In the third embodiment, as in the second embodiment, the imaging mode MR is a mode in which imaging is performed by the image sensor R102, the imaging mode MG is a mode in which imaging is performed by the image sensor G103, and the imaging mode MB is a mode in which imaging is performed by the image sensor B104.

[0131] The multiple imaging modes may be multiple different sensitivity modes. Here, the sensitivity mode is a setting for the image sensor R102, the image sensor G103, and the image sensor B104, and changes the gain of the output signal (R imaging signal, G imaging signal, B imaging signal) output from each image sensor. In other words, even if an object of the same brightness is captured, the levels of the imaging signals output from the image sensors R102, G103, and B104 may differ by setting the gain differently depending on the sensitivity mode.

[0132] 5, signal processing unit 111 has defective pixel detection unit 113, defective pixel correction unit 112, and sensitivity mode selection unit 124. Sensitivity mode selection unit 124 selects a sensitivity mode for changing the gain of an output signal of an image sensor included in camera head 100, and outputs a sensitivity mode selection signal to camera head 100.

[0133] The sensitivity mode selection signal transmitted from the camera control unit 110 is input to the image sensor R102, the image sensor G103, and the image sensor B 104. The image sensor R102, the image sensor G103, and the image sensor B 104 set different sensitivity modes for each image sensor based on the input sensitivity mode selection signal.

[0134] In this embodiment, the sensitivity mode M1 (hereinafter also referred to as the first sensitivity mode) is a mode in which imaging is performed using a specific sensitivity mode in each image sensor. Also, the sensitivity mode M2 ​​(hereinafter also referred to as the second sensitivity mode) is a mode in which imaging is performed using a sensitivity mode different from the sensitivity mode M1 (first sensitivity mode) in each image sensor. In this embodiment, two sensitivity modes are described as an example of multiple sensitivity modes, but the present embodiment is not limited to this example, and even if multiple sensitivity modes can be selected, the present embodiment can be similarly applied.

[0135] The image sensors 102 to 105 output imaging signals based on the sensitivity modes as output signals. The threshold setting change unit 119a (first threshold changing unit) sets a different first threshold for each sensitivity mode, and the threshold setting change unit 119b (second threshold changing unit) sets a different second threshold for each sensitivity mode.

[0136] (Process flow 3) Fig. 8 is a flowchart illustrating the flow of defective pixel detection processing according to the third embodiment. Fig. 9 is a flowchart illustrating the flow of defective pixel detection processing in sensitivity mode M1, and Fig. 10 is a flowchart illustrating the flow of defective pixel detection processing in sensitivity mode M2. The flow of defective pixel detection processing according to the third embodiment will be described below with reference to Figs. 8 to 10.

[0137] In step S401 of FIG. 8, defective pixel detection processing is performed in sensitivity mode M1.

[0138] (Defective pixel detection process in sensitivity mode 1) Step S401 represents defective pixel detection processing in sensitivity mode 1, and the processing of this step corresponds to the processing of steps S501 to S513 in Fig. 9. Details of the processing of steps S501 to S513 in Fig. 9 will be described below.

[0139] In step S501, the number of times R1 to repeat the defective pixel detection process is initialized to zero.

[0140] In step S502, detection processor 120a (first detection processor) executes first detection process 1A to obtain the number N1A of defective pixels A and position information of the detected defective pixels A.

[0141] In step S503, threshold setting change unit 119a (first threshold setting change unit) determines whether the number of defective pixels N1A detected in step S502 exceeds a predetermined number (e.g., S1). If the number of defective pixels N1A exceeds the predetermined number S1 (S503-NO), the process proceeds to step S504. Here, the predetermined number S1 is the upper limit number of data that can be stored in storage unit 122.

[0142] In step S504, the threshold setting change unit 119a changes the first threshold TH1A by an amount of change ΔT1 A is added to increase the setting of the first threshold TH1A.

[0143] In step S505, the threshold setting change unit 119a increments the number of repetitions R1 by 1 (R1=R1+1). After that, the process returns to step S502, and the detection processing unit 120a executes the first detection process 1A again. Steps S502 to S505 are repeated until the number of defective pixels N1A becomes equal to or smaller than the predetermined number S1 in step S503 (S503-YES).

[0144] In step S503, if the number of defective pixels N1A is equal to or smaller than the predetermined number S1 (S503-YES), the threshold setting change unit 119a advances the process to step S506.

[0145] In step S506, the threshold setting change unit 119b (second threshold setting change unit) changes the amount of change (accumulated change amount: number of repeats R1×ΔT1 B) is added to the second threshold TH1B to change the setting of the second threshold TH1B. By setting the second threshold TH1B using the number of repetitions R1 performed in the first detection process 1A, the number of times the second detection process is performed corresponding to the cumulative change amount of the second threshold TH1B can be reduced in the defective pixel detection process by the detection processing unit 120b. Also, by setting the second threshold TH1B using the number of repetitions R1 performed in the first detection process 1A, the sensitivity for determining a defective pixel can be aligned with the first threshold TH1A.

[0146] Thereafter, in step S507, detection processor 120b (second detection processor) executes second detection process 1B to obtain the number N1B of defective pixels B and position information of the detected defective pixels B.

[0147] In step S508, threshold setting change unit 119b obtains a total number of defective pixels N1 by adding together the number of defective pixels N1B of defective pixels B detected by detection processing unit 120b and the number of defective pixels N1A of defective pixels A detected by detection processing unit 120a. First defective pixel detection unit 117 and second defective pixel detection unit 116 are connected to each other so that they can communicate with each other, and are able to share threshold settings and defective pixel detection results with each other.

[0148] In step S509, threshold setting change unit 119b determines whether total number of defective pixels N1 acquired in step S508 exceeds a predetermined number (e.g., S2). If total number of defective pixels N1 exceeds predetermined number S2 (S509-NO), processing proceeds to step S510. Here, predetermined number S2 may be the same as predetermined number S1 or may be smaller than predetermined number S1, so long as it is equal to or smaller than the upper limit number of data (predetermined number S1) that can be stored in storage unit 122.

[0149] In step S510, the threshold setting change unit 119a changes the first threshold TH1A by an amount (ΔT1 A ) to change the setting of the first threshold TH1A.

[0150] In step S511, detection processing unit 120a executes a first detection process 1A to obtain the number N1A of defective pixels A and position information of the detected defective pixels A.

[0151] In step S512, the threshold setting change unit 119b changes the second threshold TH1B by an amount of change ΔT1 B is added to change the setting of the second threshold TH1B.

[0152] In step S513, detection processor 120b (second detection processor) executes a second detection process to obtain the number N1B of defective pixels B and position information of the detected defective pixels B.

[0153] Steps S508 to S513 are repeated until the total number of defective pixels N1 becomes equal to or less than the predetermined number S2 in step S509.

[0154] In step S509, the threshold setting change unit 119b determines whether the total number of defective pixels N1 acquired in step S508 exceeds a predetermined number S2. If the total number of defective pixels N1 is equal to or less than the predetermined number S2 (S509-YES), the defective pixel detection process in sensitivity mode M1 is terminated, and the process proceeds to step S402 in FIG. 8.

[0155] (Defective pixel detection process in sensitivity mode 2) Step S402 represents defective pixel detection processing in sensitivity mode 2, and the processing of this step corresponds to the processing of steps S601 to S613 in Fig. 10. Details of the processing of steps S601 to S613 in Fig. 10 will be described below.

[0156] In step S601, the number of times R2 to repeat the defective pixel detection process is initialized to zero.

[0157] In step S602, detection processor 120a (first detection processor) executes first detection process 2A to obtain the number N2A of defective pixels A and position information of the detected defective pixels A.

[0158] In step S603, threshold setting change unit 119a (first threshold setting change unit) determines whether the number of defective pixels N2A detected in step S602 exceeds a predetermined number (for example, S1). If the number of defective pixels N2A exceeds the predetermined number S1 (S603-NO), the process proceeds to step S604. Here, the predetermined number S1 is the upper limit number of data that can be stored in storage unit 122.

[0159] In step S604, the threshold setting change unit 119a changes the first threshold TH2A by an amount of change ΔT2 A is added to increase the setting of the first threshold TH2A.

[0160] In step S605, the threshold setting change unit 119a increments the number of repetitions R2 by 1 (R2=R2+1). After that, the process returns to step S602, and the detection processing unit 120a executes the first detection process 2A again. Steps S602 to S605 are repeated until the number of defective pixels N2A becomes equal to or smaller than the predetermined number S1 in step S603 (S603-YES).

[0161] In step S603, if the number of defective pixels N2A is equal to or smaller than the predetermined number S1 (S603-YES), the threshold setting change unit 119a advances the process to step S606.

[0162] In step S506, the threshold setting change unit 119b (second threshold setting change unit) changes the amount of change (accumulated change amount: number of repeats R2 × ΔT2 B ) is added to the second threshold TH2B to change the setting of the second threshold TH2B. By setting the second threshold TH2B using the number of repetitions R2 performed in the first detection process 2A, it is possible to reduce the number of processes of the second detection process 2B, which corresponds to the cumulative change amount of the second threshold TH2B, in the defective pixel detection process by the detection processing unit 120b. Also, by setting the second threshold TH2B using the number of repetitions R2 performed in the first detection process 1A, it is possible to align the sensitivity for determining a defective pixel with the first threshold TH2A.

[0163] Thereafter, in step S607, detection processor 120b (second detection processor) executes second detection process 2B to obtain the number N2B of defective pixels B and position information of the detected defective pixels B.

[0164] In step S508, threshold setting change unit 119b obtains a total number of defective pixels N2 by adding together the number N2B of defective pixels B detected by detection processing unit 120b and the number N2A of defective pixels A detected by detection processing unit 120a. First defective pixel detection unit 117 and second defective pixel detection unit 116 are connected to each other so that they can communicate with each other, and are able to share threshold settings and defective pixel detection results with each other.

[0165] In step S609, threshold setting change unit 119b determines whether total number of defective pixels N2 acquired in step S608 exceeds a predetermined number (e.g., S2). If total number of defective pixels N2 exceeds predetermined number S2 (S609-NO), processing proceeds to step S610. Here, predetermined number S2 may be the same as predetermined number S1 or may be smaller than predetermined number S1, so long as it is equal to or smaller than the upper limit number of data (predetermined number S1) that can be stored in storage unit 122.

[0166] In step S610, the threshold setting change unit 119a changes the first threshold TH2A by an amount (ΔT A ) to change the setting of the first threshold TH2A.

[0167] In step S611, detection processing unit 120a executes first detection processing 2A to obtain the number N2A of defective pixels A and position information of the detected defective pixels A.

[0168] In step S612, the threshold setting change unit 119b changes the second threshold TH2B by an amount of change ΔT2 B is added to change the setting of the second threshold TH2B.

[0169] In step S613, detection processor 120b (second detection processor) executes second detection process 2B to obtain the number N2B of defective pixels B and position information of the detected defective pixels B.

[0170] Steps S608 to S613 are repeated until the total number of defective pixels N2 becomes equal to or less than the predetermined number S2 in step S609.

[0171] In step S609, the threshold setting change unit 119b determines whether the total number of defective pixels N2 acquired in step S608 exceeds a predetermined number S2. If the total number of defective pixels N2 is equal to or less than the predetermined number S2 (S609-YES), the defective pixel detection process in sensitivity mode M2 ​​is terminated, and the process returns to step S403 in FIG. 8.

[0172] In step S403, the defective pixel detection unit 113 acquires the sum of the numbers of defective pixels N1 and N2 in each sensitivity mode as the total number of defective pixels.

[0173] In step S404, the threshold setting change unit 119b determines whether the total number of defective pixels is equal to or less than a predetermined number (for example, S1). If the total number of defective pixels exceeds the predetermined number (S404-NO), the threshold setting change unit 119b advances the process to step S405.

[0174] In step S405, defective pixel detection unit 113 determines whether the sensitivity mode with the largest number of defective pixels detected is sensitivity mode M1 among defective pixel counts N1 and N2. If sensitivity mode M1 is the sensitivity mode with the largest number of defective pixels detected (S405-YES), defective pixel detection unit 113 proceeds to step S406.

[0175] In step S406, the threshold setting change unit 119a changes the first threshold TH1A by an amount (ΔT1 A ) to change the setting of the first threshold TH1A.

[0176] In step S407, detection processing unit 120a executes a first detection process 1A to obtain the number N1A of defective pixels A and position information of the detected defective pixels A.

[0177] In step S408, the threshold setting change unit 119b changes the second threshold TH1B by an amount of change ΔT1 B is added to change the setting of the second threshold TH1B.

[0178] In step S409, detection processor 120b (second detection processor) executes a second detection process, and obtains the number N1B of defective pixels B and position information of the detected defective pixels B.

[0179] In step S410, threshold setting change unit 119b acquires the result of adding together number of defective pixels N1A and number of defective pixels N1B as number of defective pixels N1, and returns the process to step S403.

[0180] On the other hand, in step S405, if the sensitivity mode with the largest number of defective pixels is the sensitivity mode M2 ​​(S405-NO), the threshold setting change unit 119b advances the process to step S411.

[0181] In step S411, the threshold setting change unit 119a changes the first threshold TH2A by an amount (ΔT A ) to change the setting of the first threshold TH2A.

[0182] In step S412, detection processing unit 120a executes first detection processing 2A to obtain the number N2A of defective pixels A and position information of the detected defective pixels A.

[0183] In step S413, the threshold setting change unit 119b changes the second threshold TH2B by an amount of change ΔT2 B is added to change the setting of the second threshold TH2B.

[0184] In step S414, detection processor 120b (second detection processor) executes second detection process 2B to obtain the number N2B of defective pixels B and position information of the detected defective pixels B.

[0185] In step S415, the threshold setting change unit 119b acquires the result of adding the number of detected defective pixels N2A and the number of defective pixels N2B as the number of defective pixels N2, and returns the process to step S403.

[0186] In step S403, the threshold setting change unit 119b again obtains the sum of the numbers of defective pixels N1 and N2 in each sensitivity mode as the total number of defective pixels.

[0187] In step S404, the threshold setting change unit 119b determines whether the total number of defective pixels is equal to or less than a predetermined number (for example, S1). If the total number of defective pixels is equal to or less than the predetermined number (S404-YES), the threshold setting change unit 119b advances the process to step S416.

[0188] In step S416, the detection address generation unit 121 acquires position information of the defective pixel A (M1) and the defective pixel A (M2) in the sensitivity mode M1 detected by the detection processing unit 120a. Here, the defective pixel A (M1) indicates a defective pixel that always exists as a bright spot in the sensitivity mode M1. Also, the defective pixel A (M2) indicates a defective pixel that always exists as a bright spot in the sensitivity mode M2.

[0189] In addition, the detection address generation unit 121 acquires position information of the defective pixels B(M1) and B(M2) detected by the detection processing unit 120b. Here, the defective pixel B(M1) indicates a defective pixel that blinks at any time in the sensitivity mode M1. Also, the defective pixel B(M2) indicates a defective pixel that blinks at any time in the sensitivity mode M2.

[0190] In step S417, the defective pixel correction unit 112 acquires the address information generated in step S416, and identifies the position of the defective pixel by referring to the position information of the defective pixel. The defective pixel correction unit 112 then corrects the output signal of the pixel detected as the defective pixel, and ends the process.

[0191] When correcting the output signal of a defective pixel, the defective pixel correction unit 112 may refer to the identification information to identify the type (A, B) or sensitivity mode (M1, M2) of the defective pixel, and correct the output signal of the defective pixel according to the type and sensitivity mode of the defective pixel.

[0192] In the process flow 3 in Fig. 8, the process flow when a bright spot occurs as a defective pixel is explained, but in the case of a black spot, the process flow is similar to the process flow 3. In that case, the threshold value during the repeat can be changed to a lower threshold value by subtracting a predetermined change amount.

[0193] In the third embodiment, defective pixel detection processing (first detection processing, second detection processing) is performed for each of a plurality of sensitivity modes, and the numbers of defective pixels N1, N2 are obtained for each of the plurality of sensitivity modes. If the total number of defective pixels is equal to or greater than a predetermined number, the sensitivity mode with the largest number of defective pixels is selected, a predetermined change amount is added to the threshold value in the defective pixel detection processing to raise the threshold value, the total number of defective pixels is recalculated, and this is repeatedly performed until the total number of defective pixels becomes equal to or less than the predetermined number.

[0194] According to the third embodiment, even if the total number of defective pixels exceeds a predetermined number, it is no longer necessary to repeat the defective pixel detection process in all sensitivity modes. Therefore, the time required for the defective pixel detection process can be reduced compared to the prior art where there is only one threshold value regardless of the imaging mode.

[0195] Furthermore, compared to the conventional technique in which a single threshold is used to determine whether the total number of defective pixels exceeds a predetermined value, the process of the third embodiment allows the threshold to be set finely, and therefore the number of pixels determined to be defective can be reduced by raising the threshold for only a certain sensitivity mode, making it possible to prevent excessive reduction in the number of defective pixels.

[0196] Furthermore, compared to the conventional method in which there was only one threshold value regardless of the imaging mode, the threshold value can be set more precisely, and therefore it is possible to reduce the number of pixels determined to be defective by raising the threshold value only for a certain imaging mode, thereby preventing excessive reduction in the number of defective pixels.

[0197] The disclosure of this specification includes the following processing device, processing system, processing method, and program. (Item 1) A first detection processing unit that performs a first detection process to detect a first defective pixel by comparing first pixel information obtained from an output signal of each pixel of the image sensor with a first threshold value; a first threshold value changing unit that repeatedly changes the first threshold value by using a first change amount that is set at a predetermined rate with respect to an initial value of the first threshold value, until a first detection number of the first defective pixels becomes a predetermined number or less in the first detection process; a second detection processing unit that performs a second detection process to detect a second defective pixel different from the first defective pixel by comparing second pixel information obtained from the output signal and different from the first pixel information with a second threshold value obtained by using the number of times the first threshold value is repeatedly changed and a second change amount; a second threshold changing unit that changes the second threshold by using a second change amount that is set at a predetermined rate with respect to an initial value of the second threshold until added information of the first detection number of the first defective pixels and the second detection number of the second defective pixels becomes equal to or less than the predetermined number. (Item 2) A smoothing unit that acquires the first pixel information by averaging output signals of the pixels in a plurality of frames output from the image sensor, 2. The processing device according to item 1, wherein the first detection processing unit detects the first defective pixel by comparing the first pixel information obtained by averaging the output signals with the first threshold value. (Item 3) The processing device described in Item 1, wherein the first threshold change unit changes the first threshold by adding the first change amount to the first threshold or subtracting the first change amount from the first threshold until the first detection number of the first defective pixels becomes equal to or less than a predetermined number. (Item 4) The processing device according to item 1, wherein the second detection processing unit detects the second defective pixel by comparing the second threshold value obtained by using the product of the number of times the first threshold value is repeatedly changed and the second amount of change with the second pixel information. (Item 5) The image sensor further includes a summation unit configured to acquire, as the second pixel information, a maximum value of an output signal of each of the pixels in a plurality of frames output from the image sensor, The processing device according to item 1, wherein the second detection processing unit detects the second defective pixel by comparing a maximum value of the output signal acquired as the second pixel information with the second threshold value. the first threshold change unit changes the first threshold using the first change amount; the first detection processing unit performs the first detection process by comparing the changed first threshold value with the first pixel information; the second threshold change unit changes the second threshold using the second change amount; 2. The processing device according to item 1, wherein the second detection processing unit executes the second detection process by comparing the changed second threshold value with the second pixel information. (Item 7) The processing device according to item 6, wherein the second threshold change unit changes the second threshold by adding the second change amount to the second threshold or subtracting the second change amount from the second threshold. (Item 8) The camera head further includes a plurality of image sensors. 2. The processing device according to item 1, wherein the plurality of image sensors output different imaging signals as the output signals. (Item 9) The plurality of image sensors output imaging signals corresponding to a plurality of imaging modes as the output signals, the first threshold changing unit sets a different first threshold for each of the imaging signals; 9. The processing device according to item 8, wherein the second threshold changing unit sets a different second threshold for each of the imaging signals. (Item 10) The first detection processing unit executes the first detection process to detect the first defective pixel for each of the imaging signals, the second detection processing unit executes the second detection process to detect the second defective pixel for each of the imaging signals; The second threshold change unit is 10. The processing device according to item 9, further comprising: a processing device for acquiring a number of defective pixels for each of the imaging signals using the first defective pixels and the second defective pixels. (Item 11) The second threshold change unit is when the sum information of the number of defective pixels for each of the imaging signals exceeds the predetermined number, the imaging signal having the largest number of defective pixels is identified from among the imaging signals; the first threshold changing unit changes the first threshold for the identified imaging signal; Item 11. The processing device according to item 10, wherein the second threshold change unit changes the second threshold for the identified imaging signal using the second change amount. (Item 12) The first detection processing unit repeatedly executes the first detection process on the identified imaging signal until the added information becomes equal to or less than the predetermined number, Item 12. The processing device according to item 11, wherein the second detection processing unit repeatedly executes the second detection process on the identified imaging signal until the added information becomes equal to or less than the predetermined number. (Item 13) The plurality of imaging modes include an imaging mode for observing a subject using RGB imaging signals separated from visible light; 10. The processing device according to item 9, further comprising: an imaging mode for observing fluorescence emitted from a fluorescent substance present in the subject. (Item 14) The camera head further includes a sensitivity mode selection unit that selects a sensitivity mode for changing a gain of an output signal of the image sensor included in the camera head and outputs a selection signal of the sensitivity mode to the camera head, 9. The processing device according to item 8, wherein the plurality of image sensors included in the camera head are set to different sensitivity modes based on the selection signal. (Item 15) The plurality of image sensors output imaging signals based on the sensitivity mode, the first threshold value changing unit sets a different first threshold value for each of the sensitivity modes; Item 15. The processing device according to item 14, wherein the second threshold change unit sets a different second threshold for each of the sensitivity modes. (Item 16) The first detection processing unit executes the first detection process to detect the first defective pixel for each of the sensitivity modes, the second detection processing unit executes the second detection process to detect the second defective pixel for each of the sensitivity modes; The second threshold change unit is Item 16. The processing device according to item 15, wherein the processing device obtains the number of defective pixels for each of the sensitivity modes using the first defective pixel and the second defective pixel. (Item 17) The second threshold change unit is when the sum information of the number of defective pixels for each of the sensitivity modes exceeds the predetermined number, identifying the sensitivity mode having the largest number of defective pixels among the sensitivity modes; the first threshold change unit changes the first threshold for an imaging signal of the identified sensitivity mode; Item 17. The processing device according to item 16, wherein the second threshold change unit changes the second threshold using the second change amount for an imaging signal of the identified sensitivity mode. (Item 18) The first detection processing unit repeatedly executes the first detection process on the imaging signal of the identified sensitivity mode until the added information becomes equal to or less than the predetermined number, Item 18. The processing device according to item 17, wherein the second detection processing unit repeatedly executes the second detection process on the imaging signal of the identified sensitivity mode until the added information becomes equal to or less than the predetermined number. a generating unit that generates address information indicating a position of a defective pixel of the image sensor by using a pixel position of the first defective pixel and a pixel position of the second defective pixel when the sum information is equal to or less than the predetermined number; A storage unit that stores the address information; a correction unit that corrects output signals of the first defective pixel and the second defective pixel included in the image sensor by using the address information stored in the storage unit; 2. The processing device according to item 1, further comprising: (Item 20) The processing device according to Item 19, wherein the predetermined number indicates a number of data items that can store the addition information in the storage unit. (Item 21) A processing device according to any one of items 1 to 20, a display device having a display control unit that performs display control to display an image on a display unit based on a signal output from the processing device; A processing system comprising: (Item 22) A step of performing a first detection process of detecting a first defective pixel by comparing first pixel information obtained from an output signal of each pixel of the image sensor with a first threshold value; changing the first threshold value by using a first change amount set at a predetermined ratio to an initial value of the first threshold value when a first detected number of the first defective pixels exceeds a predetermined number in the first detection process; detecting a second defective pixel different from the first defective pixel by comparing second pixel information different from the first pixel information obtained from the output signal with a second threshold value obtained by using the number of times the first threshold value is repeatedly changed and a second change amount; changing the second threshold by using a second change amount that is set at a predetermined rate with respect to an initial value of the second threshold until added information of the first detection number of the first defective pixels and the second detection number of the second defective pixels becomes equal to or less than the predetermined number; The processing method according to claim 1, (Item 23) A program for causing a computer to execute the processing method described in Item 22.

[0198] [Other embodiments] The disclosed technology can also be realized by supplying a program that realizes one or more functions of the above-mentioned embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0199] The disclosed technology is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to disclose the scope of the invention. [Explanation of symbols]

[0200] 100: camera head, 101: image sensor, 102: image sensor R, 103: image sensor G, 104: image sensor B, 110: camera control unit, 111: signal processing unit, 112: defective pixel correction unit 113: defective pixel detection unit, 114: multi-frame smoothing unit, 115: multi-frame union unit, 117: first defective pixel detection unit, 116: second defective pixel detection unit, 119a, 119b: threshold setting change unit, 120a, 120b: detection processing unit, 121: detection address generation unit, 122: storage unit (RAM), 123: transmission unit, 124: sensitivity mode selection unit, 130: display device

Claims

1. a first detection processing unit that performs a first detection process to detect a first defective pixel by comparing first pixel information obtained from an output signal of each pixel of the image sensor with a first threshold value; a first threshold value changing unit that repeatedly changes the first threshold value by using a first change amount that is set at a predetermined rate with respect to an initial value of the first threshold value, until a first detection number of the first defective pixels becomes a predetermined number or less in the first detection process; a second detection processing unit that performs a second detection process to detect a second defective pixel different from the first defective pixel by comparing second pixel information obtained from the output signal and different from the first pixel information with a second threshold value obtained by using the number of times the first threshold value is repeatedly changed and a second change amount; a second threshold changing unit that changes the second threshold by using a second change amount that is set at a predetermined rate with respect to an initial value of the second threshold until added information of the first detection number of the first defective pixels and the second detection number of the second defective pixels becomes equal to or less than the predetermined number; A processing device is provided.

2. a smoothing unit that acquires the first pixel information by averaging output signals of the pixels in a plurality of frames output from the image sensor, The processing device according to claim 1 , wherein the first detection processing unit detects the first defective pixel by comparing the first pixel information obtained by averaging the output signals with the first threshold value.

3. 2. The processing device according to claim 1, wherein the first threshold changing unit changes the first threshold by adding the first change amount to the first threshold or subtracting the first change amount from the first threshold until the first detection number of the first defective pixels becomes a predetermined number or less.

4. 2 . The processing device according to claim 1 , wherein the second detection processing unit detects the second defective pixel by comparing the second threshold value obtained by multiplying the number of times the first threshold value is repeatedly changed and the second change amount with the second pixel information.

5. a summation unit configured to acquire, as the second pixel information, a maximum value of output signals of the pixels in a plurality of frames output from the image sensor; The processing device according to claim 1 , wherein the second detection processing unit detects the second defective pixel by comparing a maximum value of the output signal acquired as the second pixel information with the second threshold value.

6. Until the added information is equal to or less than the predetermined number. the first threshold change unit changes the first threshold using the first change amount; the first detection processing unit performs the first detection process by comparing the changed first threshold value with the first pixel information; the second threshold change unit changes the second threshold using the second change amount; The processing device according to claim 1 , wherein the second detection processing unit performs the second detection process by comparing the second pixel information with the changed second threshold value.

7. The processing device according to claim 6 , wherein the second threshold change unit changes the second threshold by adding the second change amount to the second threshold or by subtracting the second change amount from the second threshold.

8. A camera head having a plurality of image sensors is further provided, The processing device according to claim 1 , wherein the plurality of image sensors output different imaging signals as the output signals.

9. the plurality of image sensors output, as the output signals, imaging signals corresponding to a plurality of imaging modes; the first threshold value changing unit sets a different first threshold value for each of the imaging signals; The processing device according to claim 8 , wherein the second threshold value changing section sets a different second threshold value for each of the imaging signals.

10. the first detection processing unit executes the first detection process to detect the first defective pixel for each of the imaging signals; the second detection processing unit executes the second detection process to detect the second defective pixel for each of the imaging signals; The second threshold change unit is The processing device according to claim 9 , further comprising: a processing unit configured to obtain a number of defective pixels for each of the imaging signals by using the first defective pixel and the second defective pixel.

11. The second threshold change unit is when the sum information of the number of defective pixels for each of the imaging signals exceeds the predetermined number, the imaging signal having the largest number of defective pixels is identified from among the imaging signals; The first threshold changing unit changes the first threshold for the identified imaging signal, The processing device according to claim 10 , wherein the second threshold change unit changes the second threshold for the identified imaging signal using the second change amount.

12. the first detection processing unit repeatedly performs the first detection process on the identified imaging signal until the added information becomes equal to or less than the predetermined number; The processing device according to claim 11 , wherein the second detection processing unit repeatedly executes the second detection process on the identified imaging signal until the added information becomes equal to or less than the predetermined number.

13. The plurality of imaging modes include: an imaging mode for observing an object using RGB imaging signals separated from visible light; The processing device according to claim 9 , further comprising an imaging mode for observing fluorescence emitted from a fluorescent substance present in the subject.

14. a sensitivity mode selection unit that selects a sensitivity mode for changing a gain of an output signal of the image sensor included in the camera head and outputs a selection signal of the sensitivity mode to the camera head; The processing device according to claim 8 , wherein different sensitivity modes are set for the plurality of image sensors included in the camera head based on the selection signal.

15. the plurality of image sensors output imaging signals based on the sensitivity mode; the first threshold value changing unit sets a different first threshold value for each of the sensitivity modes; The processing device according to claim 14 , wherein the second threshold change unit sets a different second threshold for each of the sensitivity modes.

16. the first detection processing unit executes the first detection process to detect the first defective pixel for each of the sensitivity modes; the second detection processing unit executes the second detection process to detect the second defective pixel for each of the sensitivity modes; The second threshold change unit is The processing device according to claim 15 , wherein the number of defective pixels is obtained for each of the sensitivity modes using the first defective pixels and the second defective pixels.

17. The second threshold change unit is when the sum information of the number of defective pixels for each of the sensitivity modes exceeds the predetermined number, identifying the sensitivity mode having the largest number of defective pixels among the sensitivity modes; the first threshold change unit changes the first threshold for an imaging signal of the identified sensitivity mode; The processing device according to claim 16 , wherein the second threshold change unit changes the second threshold using the second change amount for an imaging signal of the identified sensitivity mode.

18. the first detection processing unit repeatedly executes the first detection process on the imaging signal of the identified sensitivity mode until the added information becomes equal to or less than the predetermined number; The processing device according to claim 17 , wherein the second detection processing unit repeatedly executes the second detection process on the imaging signal of the identified sensitivity mode until the added information becomes equal to or less than the predetermined number.

19. a generating unit that generates address information indicating a position of a defective pixel of the image sensor, using a pixel position of the first defective pixel and a pixel position of the second defective pixel, when the sum information is equal to or smaller than the predetermined number; A storage unit that stores the address information; a correction unit that corrects output signals of the first defective pixel and the second defective pixel included in the image sensor by using the address information stored in the storage unit; The processing apparatus of claim 1 further comprising:

20. The processing device according to claim 19 , wherein the predetermined number indicates a number of data items that can be stored in the storage unit as the addition information.

21. A processing device according to any one of claims 1 to 20; a display device having a display control unit that performs display control to display an image on a display unit based on a signal output from the processing device; A processing system comprising:

22. performing a first detection process of detecting a first defective pixel by comparing first pixel information obtained from an output signal of each pixel of the image sensor with a first threshold; changing the first threshold value by using a first change amount set at a predetermined ratio to an initial value of the first threshold value when a first detected number of the first defective pixels exceeds a predetermined number in the first detection process; detecting a second defective pixel different from the first defective pixel by comparing second pixel information obtained from the output signal, the second pixel information being different from the first pixel information, with a second threshold value obtained by using the number of times the first threshold value is repeatedly changed and a second change amount; changing the second threshold by using a second change amount set at a predetermined rate with respect to an initial value of the second threshold until added information of the first detection number of the first defective pixels and the second detection number of the second defective pixels becomes equal to or less than the predetermined number; The processing method according to claim 1,

23. A program for causing a computer to execute the processing method according to claim 22.

Images