PIXEL CORRECTION METHOD

Abstract

A method for correcting defective pixels (19) of an image sensor (17) having a plurality of pixels (19) for generating respective exposure-dependent pixel signals, the method comprising: - determining and storing in a storage device (23) a defect characteristic assigned to each pixel (19) of the image sensor (17) prior to generating the pixel signals; wherein the defect characteristic includes information at least as to whether the respective pixel (19) is unusable, fully usable, or corresponds to one of several predetermined defect classes by which different degrees of restricted usability are captured; - generating pixel signals by means of the pixels (19) of the image sensor (17); - reading out the defect characteristic from the storage device; and - after generating the pixel signals: - pixel signals of pixels (19),Pixel signals that are unusable according to the defect characteristics of the respective pixel (19) are discarded and replaced by a substitute value; pixel signals of pixels (19) that are fully usable according to the defect characteristics of the respective pixel (19) are not corrected; and for each pixel (19) that does not correspond to an unusable or a fully usable pixel (19) according to the defect characteristics of the respective pixel (19), a supplementary decision process is carried out, by which, depending at least on the assigned defect class of the pixel (19) and on pixel signals of several neighboring pixels (19), it is determined whether the generated pixel signal of the pixel (19) should be corrected, whereby, if applicable, the generated pixel signal of the pixel (19) is replaced by a substitute value, whereby, for determining whether the generated pixel signal of the pixel (19) should be corrected,The following steps are performed: - Determining an interpolation error of pixel (19) as a function of at least the aforementioned pixel signals of the several neighboring pixels (19); - Determining an error comparison value of pixel (19) as a function of at least the assigned defect class of pixel (19); and - Comparing the interpolation error with the error comparison value; whereby it is only stipulated that the generated pixel signal of pixel (19) should be corrected if the interpolation error is less than the error comparison value.

Description

[0001] The invention relates to a method for correcting defective pixels of an image sensor which has a plurality of pixels for generating respective exposure-dependent pixel signals, wherein the image sensor is assigned a defect characteristic for each pixel which includes information at least as to whether the pixel is unusable, fully usable or corresponds to one of several predetermined defect classes.

[0002] Image sensors of this type are used, for example, in digital still cameras and motion picture cameras (film cameras) to capture single images or a sequence of moving images. The image signals of these images consist of pixel signals generated by the pixels of the image sensor, which are typically arranged in rows and columns. Due to variations in the manufacturing of the image sensors, individual pixels can be defective. A defect is defined as any deviation of a pixel from a standard specification that renders the pixel, and especially the pixel signals generated by the pixel, unusable or only partially usable. For example, in a typical image sensor for a motion picture camera with several million pixels, 20 pixels might be unusable and 3000 pixels might be of limited use, while the remaining pixels are fully functional.

[0003] Especially in the field of professional motion picture cameras, which are also used for cinema productions, deviations caused by defective pixels cannot be tolerated, even if the number of defective pixels is relatively small compared to the total number of pixels. This is because, firstly, deviations of even individual pixels have a disruptive effect during the post-processing of the image signals generated from the pixel signals. Secondly, deviations of even a single defective pixel can become visible, particularly when projecting the image signals onto large cinema screens. Therefore, the yield in the production of image sensors for cameras with such high quality requirements is unfavorably low.

[0004] Provided that this appears tolerable in individual cases for a relatively small number of defective pixels, the pixel signals of all pixels identified as defective in an image sensor are discarded, and new pixel signals are estimated for the defective pixels by interpolation from the pixel signals of neighboring pixels. The aim is for each estimated pixel signal to approximate as closely as possible to the pixel signal that a fully usable pixel would have produced at that location, so that after correction, as little distortion as possible is perceptible in the image signal. Since all defective pixels—that is, both unusable and partially usable pixels—are discarded uniformly, the same number of pixel values ​​must always be calculated by interpolation, which results in a high correction effort.Furthermore, discarding the pixel signals of pixels with limited usability may result in the loss of potentially usable image information, thus unnecessarily impairing image quality.

[0005] For more nuanced correction, the pixels of an image sensor can be differentiated according to a defect characteristic, based on whether they are unusable, fully usable, or correspond to one of several predefined defect classes. These defect classes allow for the identification of varying degrees of limited usability for individual pixels. In particular, the unusability and / or the full usability of a pixel can also be described as separate defect classes. This classification into defect classes makes it possible to treat pixels with different degrees of defects differently. Pixels within the same defect class, however, continue to be corrected uniformly.

[0006] However, with this approach, the deviations caused by defective pixels are not necessarily constant in terms of their magnitude and / or perceptibility in the image signal. For example, the deviation of a defective pixel can depend on the intensity of its exposure (bright or dark). How noticeable a deviation in the pixel signal of a defective pixel is, and thus how much it requires correction, can depend in particular on the pixel's surroundings, i.e., the pixel signals of neighboring pixels. Therefore, to improve image quality, it is not necessarily necessary to correct all pixels of a defect class equally. Thus, simply considering a single defect characteristic leaves potential for further reduction of processing effort untapped. Furthermore, any correction is inherently subject to a degree of uncertainty.Therefore, a correction that is solely based on a defect class assigned to the respective pixel can sometimes even worsen the image quality.

[0007] It is an object of the invention to provide a method for correcting defective pixels that is efficient and ensures that the correction of the defective pixels leads to an improvement in image quality.

[0008] The problem is solved by a method with the features of claim 1 and in particular by the fact that, after generating the pixel signals for each pixel, at least when the defect characteristic of the pixel does not correspond to an unusable or a fully usable pixel, it is determined, depending at least on the assigned defect class of the pixel and on pixel signals of several neighboring pixels, whether the generated pixel signal of the pixel should be corrected, wherein, if appropriate, the generated pixel signal of the pixel is replaced by a substitute value.

[0009] In this method, the decision regarding how to handle a particular pixel is not simply predetermined by the pixel's defect characteristics. At least when the pixel has limited usability, meaning it is assigned a corresponding defect class, a supplementary decision-making process precedes the actual pixel correction. Conversely, if the pixel is unusable or fully usable, the decision regarding how to handle it can be predetermined: For example, the pixel signals of unusable pixels can always be discarded and replaced with substitute values; the pixel signals of fully usable pixels preferably require no correction.

[0010] For all other pixels, at least, a preliminary decision-making process is carried out. This process determines whether the pixel signal generated by that pixel should be corrected or not. The decision-making process includes considering at least a two-way dependency: The method depends on two factors: firstly, the assigned defect class of the respective pixel, and secondly, the pixel signals of several pixels neighboring the respective pixel. Advantageously, the method according to the invention can utilize this dual dependency in such a way that only those pixels are corrected where replacing the generated pixel signal with a substitute value leads to an improvement in image quality.

[0011] Because the aforementioned setting depends on the environment of a given pixel, it becomes possible, in particular, for some pixels assigned to a common defect class to be corrected based on a specific environment, while others are not corrected based on a different specific environment. For example, some pixels may be defective in that they exhibit an increased dark current, meaning that they generate a pixel signal that is too strong for the available light when there is little or no illumination. According to the method, such pixels are not simply ignored and replaced, but rather, for example, only in dark image areas where the deviation caused by these pixels is particularly noticeable.Furthermore, pixels whose defect is based on a low maximum output level can only be corrected if the ambient brightness is so high that their lower output level would lead to a deviation perceptible in the image signal.

[0012] Replacing the generated pixel signals of pixels designated for replacement does not necessarily have to occur immediately after the pixel correction has been determined. Preferably, the specified replacements are only carried out after the correction has been finalized for all pixels of the image sensor. In this case, the determination of whether a pixel's generated signal should be corrected always considers originally generated, rather than already corrected, pixel signals.

[0013] The defect class assigned to each pixel represents information about a fundamentally unchanging property of the pixel. While the procedure may include steps to determine the defect characteristics, and these characteristics may, as will be explained later, depend on the operating mode of the image sensor, the defect characteristics and the defect class assigned to each pixel are fixed at least before the aforementioned steps for correcting defective pixels are carried out. In contrast, the pixel signals of neighboring pixels are variable. Therefore, for each pixel with limited usability in each individual image, it must be determined whether a correction should be performed.The additional effort involved in the upstream decision-making process is at least partially offset by the fact that, as a result of this process, not all unusable and limited-use pixels need to be corrected, but rather a significantly smaller number. In particular, for the limited-use pixels that have been determined not to be corrected, the time-consuming process of determining a substitute value is unnecessary, thus improving the overall efficiency of the procedure. Furthermore, it is advantageous that the pixel signals of the limited-use pixels can be used, at least in part. This increases the total amount of information generated by the image sensor that is considered, which can lead to higher image quality.

[0014] The defect characteristics of at least the pixels that are usable to a limited extent include a defect class corresponding to the respective pixel. Information that a pixel is unusable and / or information that a pixel is fully usable need not be provided separately, but can also be included indirectly in the defect characteristics, namely by assigning a corresponding defect class.

[0015] A defect class characterizes the behavior of the pixel depending on further parameters, such as the intensity and duration of the exposure, according to a predetermined classification scheme. The classification into defect classes can be performed using a gradation. A fine gradation allows for detailed consideration of different degrees of usability of a given pixel. At the same time, the discrete values ​​of the gradation, compared to considering continuous values, reduce the required computational effort. In particular, this makes it possible to apply the method according to the invention directly in a camera, which is preferably the case.

[0016] When classifying defective pixels, various types of defects can be distinguished, which can also occur in combination within a single pixel. For example, a pixel can be assigned multiple defect classes, one for each defect type. Alternatively, a pixel can be assigned only one defect class, which corresponds to a combination of different defect types for that pixel. This allows different defect types to be considered independently in the defect characteristics with minimal computational effort.

[0017] Depending on the type of defect, the respective classification scheme can have different levels, with differences in the fineness or type of gradation being conceivable. For example, the defect classes for dark current can be linearly graded, while the defect classes for noise in the dark can be logarithmically graded, for example with an increase of 15% from level to level.

[0018] Preferably, the replacement value is estimated based on at least the pixel signals of a set of neighboring pixels, particularly by interpolation. Thus, not only does the determination of whether the generated pixel signal of a given pixel should be corrected depend on the pixel's environment, but also the replacement value by which this pixel signal is to be replaced. This dual consideration of the pixel's environment enables a particularly individualized correction of defective pixels.

[0019] In one embodiment, the set of neighboring pixels used to estimate the substitute value is identical to the multiple neighboring pixels considered to determine whether a pixel signal should be corrected. However, the set of neighboring pixels and the multiple neighboring pixels need not necessarily be the same. In an alternative embodiment, the multiple neighboring pixels and the set of neighboring pixels are different.

[0020] For example, the neighboring pixels used to determine whether a correction is applied may be selected based on their suitability for determining whether the atypical pixel signal of a defective pixel is significant enough in that environment to warrant correction. Conversely, the set of neighboring pixels used to estimate the replacement value may be selected to ensure that the estimation yields high-quality results.

[0021] Both in determining whether a particular pixel should be corrected and in estimating the respective replacement value, the pixel signals of several neighboring pixels can be the pixel signals actually generated by those neighboring pixels. If there are also defective pixels among the neighboring pixels that have already been corrected, these pixel signals can alternatively be previously corrected pixel signals.

[0022] In another embodiment, the substitute value can be estimated at least partially by modifying the pixel signal generated by the pixel according to the pixel's defect characteristics, i.e., depending on the assigned defect class. The substitute value can thus be based, at least in part, on the originally generated pixel signal of the pixel with limited functionality. In this way, the pixel's original signal is not discarded but contributes to the substitute value by which it is replaced. This is particularly useful when the deviation of the pixel signal generated by the defective pixel compared to a pixel signal that a fully functional pixel would have generated can be substantially compensated for.Compensation can be achieved, for example, multiplicatively using a correction factor or by applying a specially adapted mathematical function, whereby the defect characteristic can be used, in particular, to select the appropriate modification. Preferably, the originally generated pixel signal of a pixel with limited usability is only modified and incorporated into the substitute value if the deviation in the pixel signal can be compensated; otherwise, it is discarded. Whether the deviation in the pixel signal can be compensated can preferably be determined based on the defect characteristic of the pixel.

[0023] Furthermore, it is possible to estimate the replacement value based on both the pixel signals of the aforementioned set of neighboring pixels and the originally generated pixel signal of the defective pixel. This can be achieved, for example, by weighting a value interpolated from the pixel signals of the neighboring pixels and combining it with the pixel signal of the defective pixel, modified by a correction factor.

[0024] In an advantageous embodiment, the following steps are performed to determine whether the generated pixel signal of the pixel should be corrected: determining an interpolation error of the pixel as a function of at least the aforementioned pixel signals of the several neighboring pixels; determining an error comparison value of the pixel as a function of at least the assigned defect class of the pixel; and comparing the interpolation error with the error comparison value. In this embodiment, it is determined that the generated pixel signal of the pixel should be corrected only if the interpolation error is less than, or at least not greater than, the error comparison value.

[0025] In this case, the decision as to whether the generated pixel signal of a pixel should be corrected is based on a comparison of an error present due to the pixel defect, which can be derived as an error comparison value from the assigned defect class of the pixel, with an interpolation error. Preferably, the interpolation error corresponds to the error that would be present after correction of the pixel signal precisely because of the imperfection of this correction, i.e., due to the uncertainty of interpolation. By comparing the error comparison value with the interpolation error, it can thus be determined whether replacing the pixel signal of the respective pixel with the replacement value actually leads to a reduction in the deviation in the pixel signal of that pixel and thus results in an improvement in image quality.Advantageously, the pixel signal is only corrected if this is the case, i.e., if the interpolation error is smaller than the error comparison value.

[0026] In particular, to further increase efficiency, it can be stipulated that the generated pixel signal should only be corrected if the interpolation error is at least a certain comparison threshold lower than the error comparison value. The larger this comparison threshold is chosen, the fewer defective pixels will require correction of their respective generated pixel signals, and the lower the overall correction effort. Conversely, the smaller the comparison threshold, the more comprehensive the correction and therefore the better the image quality. By appropriately selecting the comparison threshold, the respective prioritization of these opposing goals can be adjusted. Preferably, the comparison threshold has the lowest possible value at which the effort of the method still permits its application within a camera.

[0027] The inventive principle for determining whether a correction should be made depends on the pixel signals of several neighboring pixels and essentially incorporates the aforementioned interpolation error into the decision. The interpolation error can be considered a measure of how closely the substitute value likely corresponds to, or is similar to, a value that a fully usable pixel at that location would have generated as a pixel signal. To assess this, the pixel signals of the several neighboring pixels are taken into account when determining the interpolation error; that is, the area surrounding the pixel into which a suitable substitute value should be integrated as unobtrusively as possible.

[0028] As explained, estimating the substitute value can be achieved, in particular, by interpolating pixel signals from a set of neighboring pixels. Determining the interpolation error can preferably be done based on the pixel signals of the same neighboring pixels. However, it is also possible to use the pixel signals of several other neighboring pixels, e.g., only a subset of the aforementioned set and / or entirely different neighboring pixels. For example, by considering fewer neighboring pixels for determining the interpolation error than for estimating the substitute value, the efficiency of deciding whether to correct the generated pixel signal of a pixel can be increased. For the correction, if applicable, a comparatively large number of neighboring pixels can be considered to achieve the highest possible quality in estimating the substitute value.In such an embodiment of the method, the interpolation error does not necessarily reflect the actual error of the substitute value, but it can nevertheless be considered, in comparison with the error comparison value, as an indication of whether the correction of the generated pixel signal of the pixel has a beneficial effect on the image quality.

[0029] Preferably, determining a pixel's interpolation error comprises: determining interpolation noise; determining the noise of several neighboring pixels, particularly as a function of at least the respective assigned defect classes of the several neighboring pixels; and determining the interpolation error as a function of at least the interpolation noise and the noise of the several neighboring pixels. The interpolation error can thus essentially consist of at least two error sources: interpolation noise on the one hand and noise from the neighboring pixels on the other. Interpolation noise here refers to the error inherent in the respective method of estimating the substitute value. This is because any estimation is always subject to a certain degree of uncertainty, the magnitude of which depends in particular on the method used, e.g., a specific interpolation procedure.The interpolation noise, which represents a measure of this uncertainty, can be an absolute or a relative value. For example, the interpolation noise might be 10% of a new value for the pixel signal determined by a specific interpolation method. This new value preferably matches the substitute value, but may also differ from it.

[0030] The interpolation error, however, is not only caused by estimation errors, but also by errors in the pixel signals of neighboring pixels. These errors are collectively referred to here as noise and can advantageously be derived from the defect characteristics assigned to the image sensor for the neighboring pixels. The noise of several neighboring pixels can then be obtained by considering the individual errors in the pixel signals of the neighboring pixels together, for example, by calculating a weighted sum or a function value appropriately adapted to the interpolation method in some other way. Once the interpolation noise and the noise of several neighboring pixels have been determined, the interpolation error can be calculated as a function of these quantities, for example, as their sum or as the square root of the sum of their squares.

[0031] The error comparison value (ECV) is a measure of the deviation in the pixel signal caused by the pixel defect. To increase the significance of the EVC, it can depend on additional factors besides the assigned defect class of the pixel. Specifically, the EVC is determined based on the pixel's generated signal, its exposure time, and / or the pixel signals of several neighboring pixels. Depending on the type of pixel defect, the deviation in the pixel signal of that pixel may not be constant, but can depend on the generated signal and the pixel's exposure time. Furthermore, the perceived impact of a deviation in the pixel signal of a defective pixel depends on the pixel's surroundings.Certain defects are particularly noticeable in bright image areas, while others are especially noticeable in dark areas. Therefore, it is advantageous to consider these dependencies when determining the pixel's error comparison value. This allows for a better assessment, based on a comparison with the interpolation error, of whether a correction is beneficial in terms of a perceptible improvement in image quality.

[0032] To account for several different defect types, which may also occur in combination within a single pixel, the defect characteristics for each pixel can preferably include information on whether the pixel corresponds to one of several predetermined defect classes for at least two different defect types. Various ways are possible in which the different defect types can be incorporated into the determination of whether the generated pixel signal of a given pixel should be corrected:

[0033] In one embodiment, a common error comparison value for the pixel is determined for all different defect types, depending at least on the assigned defect class of the pixel. The generated pixel signal is only to be corrected if the interpolation error is less than, or at least no greater than, this common error comparison value. In this embodiment, a single defect class can be assigned to the pixel, representing the combination of the respective degrees of defects of different types in that pixel. The classification into defect classes can then be performed, for example, according to a multidimensional matrix, where each dimension corresponds to a defect type and where the dimensions can be independently and differently graded (e.g., linearly or logarithmically).Thus, a single error comparison value, taking all defect types into account, can be derived from the defect class assigned to the pixel and then compared with the interpolation error. Consequently, all defect types are equally considered in the decision regarding pixel correction based on this comparison.

[0034] In an alternative embodiment, a corresponding error comparison value for each of the different defect types is determined based on at least the pixel's assigned defect class. The generated pixel signal is only corrected if the interpolation error for at least one of the different defect types is less than, or at least no greater than, the corresponding error comparison value. In this alternative embodiment, a corresponding defect class can be assigned to each pixel for each of the different defect types. Thus, a corresponding error comparison value for each of the different defect types can be derived from the pixel's corresponding defect class.The interpolation error is then compared with all error reference values ​​for the pixel. This comparison can be stopped as soon as the interpolation error is less than one of the error reference values. Therefore, a decision to correct the generated pixel signal of a pixel is only made if the pixel's interpolation error is at least less than the maximum of the pixel's error reference values. In this way, only the dominant defect type for that pixel is decisive for the aforementioned decision regarding pixel correction.

[0035] The various types of defects mentioned can include, in particular, increased noise in the dark, low maximum amplitude, dark current, and / or deviations at short exposure times for the respective pixel. Increased noise in the dark means that the fluctuations in the pixel signal, which are always present despite constant exposure, are significantly increased at low exposure intensities. A low maximum amplitude is defined as a defect where the threshold at which an increase in exposure intensities no longer leads to an increase in the pixel signal is lowered. Dark current refers to a pixel signal that occurs at low exposure levels and erroneously represents a higher exposure than the actual exposure. Another defect occurs when the relationship between the pixel signal and the exposure time deviates from the normal relationship at short exposure times.

[0036] In one embodiment of the method, the multiple neighboring pixels comprise pixels directly adjacent horizontally, directly adjacent vertically, and / or directly adjacent diagonally to the pixel in question, and / or indirectly adjacent pixels. This applies independently both to the multiple neighboring pixels considered when determining the correction of a pixel's signal and to the set of neighboring pixels that can be used to estimate the substitute value. Directly neighboring pixels are those between which no other pixels are located on the image sensor, while indirectly neighboring pixels may have a few other pixels positioned between them.

[0037] If the image sensor comprises at least a plurality of pixels exhibiting a first color and a plurality of pixels exhibiting a second color, an embodiment may be advantageous in which the pixel and its neighboring pixels exhibit the same color. Thus, in this embodiment, for determining whether a pixel's signal should be corrected and / or for estimating the replacement value with which the pixel should be appropriately replaced, only those pixels exhibiting the same color as the pixel are considered as the aforementioned multiple neighboring pixels or set of neighboring pixels. Since pixels of the same color generally exhibit a higher correlation with each other, this correlation can be used to improve the quality of the correction.The image sensor can, in particular, have pixels of three different colors arranged according to the scheme of a color mosaic filter, preferably a Bayer filter. If only those pixels that have the same color as the respective pixel are considered adjacent, then two pixels are considered directly adjacent if no other pixel of the same color is arranged between them, and indirectly adjacent if only a few other pixels of the same color are arranged between them.

[0038] Preferably, the method further comprises determining and storing the defect characteristic assigned to each pixel of the image sensor in a storage device before generating the pixel signals for each pixel, and reading the defect characteristic from the storage device for each pixel before determining whether the generated pixel signal should be corrected. Determining and storing the defect characteristic is not necessarily part of the method for correcting defective pixels. For the method to be carried out, it is generally sufficient if a defect characteristic assigned to the image sensor is specified in some way. However, in order to determine this defect characteristic in the first place, a calibration can precede the actual correction method, whereby the defect characteristic is determined for each pixel and stored in a storage device for later use.The storage device can be contained, in particular, within a camera that includes the image sensor. From there, the defect characteristics can then be read from the storage device to determine whether the generated pixel signal of a pixel should be corrected.

[0039] Determining and storing the defect characteristics does not need to be performed immediately before carrying out the rest of the process for correcting defective pixels. In particular, the determination and storage of the defect characteristics for an image sensor can be performed only once by the manufacturer of the image sensor or a camera incorporating the image sensor after the sensor or camera has been manufactured, whereas the subsequent steps of the process, which include the actual correction, can be performed multiple times. Alternatively or additionally, determining and storing the defect characteristics can also be done repeatedly, for example, regularly as part of maintenance or recalibration. In this way, pixel defects that may only appear over time can be captured in the defect characteristics.

[0040] The defect characteristics can be stored in the memory device in a variety of ways. For example, the defect classes assigned to the respective pixels can be stored as one or more bits, with unusable or fully usable pixels being encoded by the same bits. In the latter case, the information that a pixel is unusable or fully usable ultimately constitutes its own defect class.

[0041] In a further embodiment, the aforementioned determination of the defect characteristic comprises determining and storing calibration data for each pixel in a first memory of the storage device, and determining the defect characteristic assigned to the image sensor for each pixel from the respective calibration data, depending on an operating mode of the image sensor, and storing it in a second memory of the storage device. In this embodiment, a defect characteristic is not initially assigned directly to a particular pixel, but instead a set of respective calibration data. In particular, such calibration data for a particular pixel can be determined with respect to a multitude of different calibration parameters, for example, as part of a calibration after the manufacture of the image sensor or a camera comprising the image sensor.The various calibration parameters can correspond to different types of defects and / or different operating modes. However, multiple calibration parameters can also be considered for a single type of defect and / or a single operating mode.

[0042] The calibration data determined for each pixel is stored in a primary memory of the storage device, from where it can be read out to determine a defect characteristic associated with that pixel. However, determining the defect characteristic also depends on the operating mode of the image sensor. This operating mode could be, for example, an exposure time or the temperature of the image sensor. Other operating modes of the image sensor or of a camera containing the image sensor are also possible, such as a set sensitivity value.

[0043] Because only the calibration data is initially stored, and the operating mode of the image sensor is also taken into account when determining the defect characteristics, the defect characteristics can be different for various operating modes and thus be particularly well-suited to the different requirements of each operating mode. While the defect characteristics must be recalculated from the calibration data when changing operating modes, or at least when setting the operating mode for the first time, this process is quick and easy because it does not involve a full calibration, but rather simply evaluates the calibration data already stored in the initial memory depending on the operating mode.

[0044] The defect characteristic can then be stored in the second memory of the storage device and read from there to determine whether the generated pixel signal of a respective pixel should be corrected, at least as long as the operating mode is maintained. However, the second memory can also store several defect characteristics for different operating modes, allowing a selection to be made between them depending on the operating mode.

[0045] The aforementioned storage device can be designed as a single storage element that includes both the aforementioned first storage for the calibration data and the aforementioned second storage for the defect characteristics; however, it can also have different, in particular separate, storage elements, so that the first and the second storage can be located on different, even spatially separated, storage elements.

[0046] The invention also relates to a digital camera, in particular a film camera, with an image sensor having a plurality of pixels for generating respective exposure-dependent pixel signals, and a correction device suitable for carrying out a method of the type described above for correcting defective pixels of the image sensor. In particular, the aforementioned storage device can also be provided as part of the camera.

[0047] The invention is described below only by way of example with reference to the drawings.

[0048] Fig. Figure 1 shows a digital film camera in which a method according to the invention can be carried out.

[0049] Fig. 2 shows an image sensor that is in Fig. 1 camera shown.

[0050] The in Fig. 1 simplified representation of a camera 11 includes a housing 13 with an opening in which a lens15 is arranged. Light from a scene to be recorded, passing through the lens 15 into the camera 11 When this occurs, it is directed onto an image sensor 17 shown.

[0051] The image sensor 17 is in Fig. 2 is shown schematically and comprises a large number of light-sensitive pixels. 19 , which are arranged in rows and columns on the image sensor 17 are arranged. Overall, the image sensor can 17 several million pixels 19 for example, they might have an aspect ratio of 3:2 or 4:3. For the sake of simplicity, only a few pixels are used. 19 shown. In particular, the image sensor 17 exhibit an unrepresented color mosaic filter, so that each pixel 19 is exposed exclusively to light of a single color from the color mosaic filter, such as red, green or blue.

[0052] The individual pixels 19of the image sensor 17 Depending on the incident light, pixel signals are generated, which together form a single image signal. Provided the camera... 11 In the case of a moving image camera (film camera), a large number of image signals are generated in regular succession, for example for 24, 25 or 30 frames per second or a multiple thereof.

[0053] Due to manufacturing variations, some of the pixels may 19 of the image sensor 17 defective, meaning not fully usable. Defective pixels 19 They may be either completely unusable or at least have limited usability.

[0054] As in Fig. Shown in section 1, the camera includes 11 next to the image sensor 17 a correction device 21 with a storage device 23 for a defect characteristic of the image sensor 17, an optional buffer storage 25 for temporarily storing pixel signals and an image memory 27 for storing image signals. The correction device 21 This is necessary for carrying out the procedure for correcting defective pixels as explained in detail above. 19 of the image sensor 17 trained. The correction facility 21 The relevant pixel signals can, for example, be taken directly from the image sensor. 17 received and corrected, forwarded, or from the image memory 27 read, correct and write back or – as in Fig. 1 shown – from the buffer storage 25 received and corrected in the image memory 27 write.

[0055] The buffer storage 25 This can be used in particular to collect individual pixel signals until a complete image signal with pixel signals from all pixels is generated. 19 of the image sensor 17is present. The buffer storage tank 25 However, it can also collect more than one complete image signal or only parts of a complete image signal. The pixel signals stored in this way can then be sent to the correction device. 21 will be issued.

[0056] If the correction device 21 If it receives pixel signals, it can, in particular, take the following steps to correct the defective pixels. 19 of the image sensor 17 carry out: First, the correction device checks 21 Whether a given pixel is unusable or fully usable. The correction device can provide the necessary information about the pixel. 21 from the storage device 23 read out, with which it is connected and in which one is the image sensor 17 assigned defect characteristics with corresponding information for each pixel 19of the image sensor 17 is stored.

[0057] If the respective pixel 19 is unusable, the correction device 21 directly (e.g. by interpolation) a substitute value for the pixel signal of this pixel 19 determine and use this pixel as a new pixel signal 19 output when the respective pixel 19 It is fully usable, the pixel signal of the pixel can 19 remain unchanged. However, if the respective pixel 19 The correction device determines that the condition is neither unusable nor fully usable. 21 depending on the pixel 19 assigned defect class and pixel signals of several adjacent pixels 19 (and optionally depending on other parameters) determines whether the generated pixel signal of the pixel 19 It should be corrected or not. The aforementioned pixel 19The assigned defect class can be used for the correction device 21 in turn, the one in the storage device 23 extract stored information.

[0058] If the aforementioned determination shows that the pixel signal of the respective pixel 19 If correction is required, then the correction device replaces it. 21 the pixel signal of the pixel 19 by a substitute value, which is determined in particular by the correction device 21 can be determined by the user. Otherwise, it may be provided that the pixel signal of the pixel 19 It is not replaced, but remains unchanged.

[0059] To perform these steps, the correction device can be used 21 for example, a microprocessor, with the steps of the procedure then being programmed instructions for the microprocessor in the correction device. 21 They may be stored.

[0060] Ultimately, the corrected image signals, i.e., the image signals on whose pixel signals the described procedure for correcting defective pixels is applied, can be used 19 of the image sensor 17 has been applied in the image storage 27 be stored, from where they can then be output via an output 29 the camera 11 can be read out. However, an image memory is not absolutely necessary. 27 in the camera 11 This is provided for. Alternatively, the corrected image signals can also be sent directly to the output. 29 the camera 11 will be issued. Reference symbol list 11 Camera 13 cases 15 lenses 17 Image sensor 19 pixels 21 Correction device 23 Storage device 25 buffer storage tanks 27 image storage 29 Exit

Claims

[1] Methods for correcting defective pixels ( 19 ) of an image sensor ( 17 ), which contains a large number of pixels ( 19 ) to generate respective exposure-dependent pixel signals, wherein the image sensor ( 17 ) for each pixel ( 19 ) is assigned a defect characteristic that includes information at least about whether the pixel ( 19 ) is unusable, is fully usable, or corresponds to one of several predetermined defect classes, characterized by that after generating the pixel signals for each pixel ( 19 ) at least when the defect characteristic of the pixel ( 19 ) not an unusable or a fully usable pixel ( 19 ) corresponds, depending at least on the assigned defect class of the pixel ( 19 ) and from pixel signals of several neighboring pixels ( 19 ) is determined whether the generated pixel signal of the pixel ( 19) is to be corrected, where applicable the generated pixel signal of the pixel ( 19 ) is replaced by a substitute value. [2] Method according to claim 1, wherein the replacement value depends on at least pixel signals of a set of neighboring pixels ( 19 ) is estimated. [3] Method according to claim 1 or 2, wherein the substitute value is estimated at least partially by the generated pixel signal of the pixel ( 19 ) according to the assigned defect characteristic of the pixel ( 19 ) is modified. [4] Method according to any one of the preceding claims, where for determining whether the generated pixel signal of the pixel ( 19 To correct this, the following steps should be carried out: – Determining the pixel's interpolation error ( 19 ) depending at least on the aforementioned pixel signals of the several neighboring pixels ( 19 ); – Determining an error comparison value of the pixel ( 19 ) depending at least on the assigned defect class of the pixel ( 19 ); and – Comparing the interpolation error with the error comparison value; where it is only then determined that the generated pixel signal of the pixel ( 19 ) should be corrected if the interpolation error is less than the error comparison value. [5] Method according to claim 4, where determining the interpolation error of the pixel ( 19 ) includes, – that interpolation noise is detected; – that noise of the aforementioned several neighboring pixels ( 19 ), in particular depending at least on the respective assigned defect classes of the aforementioned several neighboring pixels ( 19 ), is determined; and – that the interpolation error depends at least on the interpolation noise and the noise of the aforementioned several neighboring pixels ( 19 ) is determined. [6] Method according to claim 4 or 5, wherein determining the error comparison value of the pixel ( 19 ) additionally depending on the generated pixel signal of the pixel ( 19 ) and / or depending on the exposure time of the pixel ( 19 ) and / or depending on the pixel signals of the aforementioned several neighboring pixels ( 19 ). [7] Method according to any one of claims 4 to 6, where the defect characteristic for each pixel ( 19 ) for at least two different types of defects includes information about whether the pixel ( 19 ) corresponds to one of several predetermined respective defect classes; where a common error comparison value of the pixel is used for all different types of defects (19 ) depending at least on the assigned defect class of the pixel ( 19 ) is determined; and where it is only then determined that the generated pixel signal of the pixel ( 19 ) should be corrected if the interpolation error is less than the common error comparison value. [8] Method according to any one of claims 4 to 6, where the defect characteristic for each pixel ( 19 ) for at least two different types of defects includes information about whether the pixel ( 19 ) corresponds to one of several predetermined respective defect classes; where for each of the different defect types a respective error comparison value of the pixel ( 19 ) depending at least on the respective assigned defect class of the pixel ( 19 ) is determined; and where it is only then determined that the generated pixel signal of the pixel ( 19) should be corrected if, for at least one of the different defect types, the interpolation error is less than the respective error comparison value. [9] Method according to one of claims 7 or 8, wherein the different defect types include at least increased noise in the dark, low maximum modulation, dark current and / or deviation at short exposure time of the respective pixel ( 19 ) include. [10] Method according to any of the preceding claims, wherein the aforementioned multiple adjacent pixels ( 19 ) directly horizontally, directly vertically and / or directly diagonally relative to the pixel ( 19 ) neighboring pixels ( 19 ) and / or indirectly neighboring pixels ( 19 ) include. [11] Method according to any one of the preceding claims, wherein the image sensor ( 17 ) at least a large number of pixels ( 19 ), which have a first color, and a multitude of pixels (19 ), which include a second color; and where the pixel ( 19 ) and the aforementioned multiple adjacent pixels ( 19 ) have the same color. [12] Method according to any one of the preceding claims, wherein the method further comprises that the image sensor ( 17 ) for each pixel ( 19 ) assigned defect characteristic before generating the pixel signals for each pixel ( 19 ) determined and stored in a storage device ( 23 ) is stored, and that for each pixel ( 19 ) before determining whether the generated pixel signal of the pixel ( 19 ) is to be corrected, the defect characteristics from the storage device ( 23 ) is read out. [13] The method of claim 12, wherein said determination of the defect characteristic comprises that for each pixel ( 19 ) respective calibration data determined and stored in a first memory of the storage device (23 ) are stored, and that the image sensor ( 17 ) for each pixel ( 19 ) associated defect characteristic depending on an operating mode of the image sensor ( 17 ) determined from the respective calibration data and stored in a second memory of the storage device ( 23 ) is saved. [14] Digital camera ( 11 ), especially a film camera, with an image sensor ( 17 ), which contains a large number of pixels ( 19 ) for generating respective exposure-dependent pixel signals, and a correction device ( 21 ), which is suitable for a method for correcting defective pixels ( 19 ) of the image sensor ( 17 ) according to one of the preceding claims.

Images