A method for processing sky images obtained from a camera using calibration images

Abstract

A method for processing images of the sky taken by an optical sky image capture device, based on the use of calibration images consisting of dark images and bias images. The method comprises: - the determination of a universal master dark image D; - the determination of a universal master bias image B; - the correction of the sky images using the universal master dark image D and the universal master bias image B in order to obtain a corrected image L' j.

Description

Title of the invention: Method for processing images of the sky obtained by a camera via calibration images

[0001] The present invention relates primarily to a method for processing images of the sky, and secondly to an optical apparatus capable of taking photographs for the acquisition of such images, essentially comprising an optical sighting device and an image acquisition device, for example, a telescope combined with a camera. For long exposure times, inherent in particular to night photography, this assembly can preferably be fitted with drive means that compensate for the Earth's rotation, and consequently maintain relative immobility between the image-taking device and the target image.

[0002] Among the challenges of obtaining high-quality images of the sky, some relate to the imaging device itself, generally a camera: it is necessary to consider the defects introduced by the camera itself, the main ones being thermal noise, bad pixels, and read noise. Thermal noise is, in practice, image noise related to the heating of the optical sensor during capture. This noise increases with long exposure times and high temperatures. Bad pixels result from the individual elements of the optical sensor, which may have a physical defect resulting in an inadequate response. Finally, read noise consists of the noise introduced by the camera's electronic circuits during the reading of these individual sensor elements.In this particular field of sky photography, exposure time and gain are the main camera configuration parameters.

[0003] According to the state of the art, to remedy these problems, calibration images acquired with the same parameters as the sky images are used to correct camera defects appearing in the actual data image. In particular, a so-called master dark image is employed, a solution which, it should be noted, is already known and applied for sky image acquisition. This master dark image is used, in particular, to correct the actual data image constituting the image capture. It is constructed from a plurality of individual dark images captured at the time of the image capture: either just before for live use, or just after or during for delayed use. More precisely, according to the state of the art, the conventional construction of the master dark image obeys the following equation:

[0004] D(g,e,t) = 1 i

[0005] With: - di (s'e,t> : an individual dark image taken at time i, with a gain g, an exposure time e and a temperature t, _ p (g,e,t) . master darkness image, considered with the gain g, the exposure time e and the temperature t,

[0006] The corrected image is then obtained in the following manner: £ _ j\g,e,t) _ j-^g,e,t)

[0007] With: - Lj (g'e,t> : a real image taken at time j, with the gain g, the exposure time e and the temperature t, _ ]j.(g,e,t). a jmagC corrected at time j, with the gain g, the exposure time e and the temperature t.

[0008] The validity of this triplet (g,e,t) allows us to obtain the desired optimal image quality. In practice, the validity of this triplet is difficult to guarantee for numerous technical reasons. For example, the temperature ta is very likely to change during observation, which introduces a divergence over time.

[0009] To address this problem, solutions have been proposed. For example, US patent 2004 / 051797 A1 describes a solution based on finding a better fit between the master dark image and the real data images by modeling the behavior of each pixel as a function of temperature and exposure time. The evolution of this behavior is reduced to a modifying factor to be applied to each pixel of the master dark image. The parameters of the function to be applied are determined empirically during a calibration step, based on measurements taken for varying temperatures and exposure times.

[0010] US patent document 6101287 also proposes a solution for adjusting the master dark image to make it more effective in correcting real data images. This solution relies on masked pixels that are present in all images captured by a sensor (but not in the final camera images). In the method described in this document, these pixels are considered to reflect the drift of the dark image and can therefore be used to define a coefficient that is then applied to the master dark image to match it more precisely to each real data image.

[0011] The present invention aims to address the same problem, but with a very different approach, since the solution adopted does not seek to model the camera's behavior, nor to adjust a master dark image "close" to the actual data images. The approach of the invention is actually based on the use of two types of calibration images (dark images and bias images) to account for the defects introduced by the camera itself.

[0012] According to the invention, the method for processing images of the sky, taken by an optical device for capturing such images, is based on the use of calibration images consisting of dark images and slant images, and it is characterized by:

[0013] - the determination of a universal master dark image D;

[0014] - the determination of a universal master bias image B;

[0015] - the correction of sky images by means of the master darkness image universal D and the universal master bias image B with a view to obtaining a corrected image L'j.

[0016] In fact, the universal master dark image D is, according to the invention, a so-called "worst-case" master dark image, not necessarily representative of a real-world use case but rather of use under worse conditions than said real-world case. This approach is particularly advantageous in that the universal master dark image eliminates the need to take dark images with the same parameters as the sky image during all observation sessions, which represents a saving of time and effort since only one universal master dark image is required. It should also be noted that it can be obtained at the factory, before delivery to the end customer, thus simplifying its use by the customer.Furthermore, the absence of pixel behavior modeling eliminates all mathematical errors related to the model itself and the methods used to solve it, thus preventing the introduction of errors into the images.

[0017] Bias images are images acquired by a camera in complete darkness, configured with a very short exposure time. These images therefore reflect the camera's read noise very directly. The universal master bias image, like the universal dark image, is a so-called "worst-case" image, obtained by again choosing the most unfavorable possible configuration in terms of gain, temperature, etc., i.e., exceeding actual operating conditions.

[0018] In practice, according to the invention, the universal master dark image is obtained by combining n individual dark images di acquired at times i by an optical sensor placed in the dark, with a gain g, an exposure time e, and at a temperature t. An individual dark image is an image acquired by a B \, the function g being a mean or median, the The camera is placed in the darkest possible environment. The dark image actually depends heavily on the camera's configuration (exposure time, gain, temperature, etc.), as it essentially results from the behavior of the camera's electronic components, each of which behaves differently from the others.

[0019] Similarly, the universal master bias image is obtained by combining n individual bias images bi acquired at times i by an image acquisition device placed in complete darkness, with a gain g, an exposure time e and at a temperature t.

[0020] More precisely, the calculation of the universal master bias image is obtained by the formula g = g fa j, the function g being a mean or a median, the images 2=1 bi being acquired with the maximum gain of the image acquisition device, with the minimum exposure time permitted by the acquisition device, at room temperature. This is a very short exposure time, such as less than 5 ms. Similarly, the calculation of the universal master dark image is obtained using the formula Σ) = g (d 2=1 These images are acquired with the maximum gain of the image acquisition device, using an exposure time longer than the maximum exposure time used for sky image acquisition at room temperature. This maximum exposure time is known to professionals; for example, it is greater than 30 seconds. The exposure times of these slanted images are considerably shorter than those of dark images, which corresponds well to their function of translating camera read noise.

[0021] By way of example, for calculating the images of universal master darkness and universal master bias respectively, it is possible to choose: • 50 individual "worst-case" dark images, captured with a gain of 30dB, an exposure time of 60s, at room temperature; • 50 individual "worst case" bias images, captured with a gain of 30dB, an exposure time of 1ms, at room temperature.

[0022] The objective of the invention is therefore to use these universal calibration images D and B described above to correct real data images Lj captured by the camera in normal use. To this end, as soon as a new real data image Lj is available, the invention must produce a corrected image L'j. To do this, the method of the invention aims to determine, for each image j, two coefficients c and c' such that XD - XB U JJ J

[0023] More specifically, for each acquisition of a real data image L, the calculation of a corrected image L'j is carried out according to the following steps:

[0024] - fractionation of the image of real data Ljen m zones of N pixels, N being less than or equal to the number 5 pixels constituting the image;

[0025] - for each zone ZJk, k belonging to the interval [l..m], determination of a couple (c jk d, c jkh ) such that the following function is minimal: £ jk ( CSjk' cjk ) — N ^x,y ( ^jJik,y " c^k X Dx,y - c^k X ^.7 ^jk ) with = 77 ^xy^jkky ' c <jk x " cîjk x B*,y

[0026] - calculation of two coefficients / j A db Wa function nh being a \Cj' Cjj = ^Cjk' cjk) average or median;

[0027] - determination of the corrected image L'j for any pixel t with coordinates (x, y) of the image of real data Lj: . . , . T ,x,y T x,yj T^k,yn rJLY V(x, y)Lj = Lj - cyxD - cjxB

[0028] In the method of the invention, the aim is to adapt the universal master image for each real data image Lj by minimizing a local noise function. Finding a local optimum for each real image Lj based on the image itself ensures that the applied master dark image produces an optimal final result, unaffected by any external error whatsoever.

[0029] According to one possibility, determining a pair (c Jk d, c jkh ) for the function to be minimal can be carried out iteratively in two steps:

[0030] - by traversing, according to a predetermined step, a search range for c Jk d and a search range for c Jk h, in order to find the minimum value defjk;

[0031] - by retaining the minimum value of a parabola passing through the point c jk d (respectively c jk h) identified in the previous step.

[0032] In a particular embodiment, the values ​​c jk b can be provided fixed and equal to 1. Alternatively, the values ​​c jk h can also be provided fixed and equal to 0. In general, it is noted that, given that the objective here is to find the pairs (c jkd, c jkh ) minimizing / ^, any suitable optimization method can be used.

[0033] In summary, the method of the invention makes it possible to always apply a darkness image adapted to the real situation, calculated from the universal master images, not only of each image during a given observation, but also of each observation during the year.

[0034] To further optimize the quality of the corrected images, according to the invention, the method of the invention may include the implementation of a detection and identification of bad pixels p Mx,y related to the image acquisition device, generally a camera. In this case, in practice, an additional correction is applied to each corrected image L j by filtering and processing the detected bad pixels p Mx,y. These bad pixels p Mx,y can then be placed in a list H, in order to be readily available for the additional filtering processing applied to each image that has undergone an initial modification based on the universal calibration images specific to the present invention.

[0035] More precisely, for each individual dark image or image fraction dt containing s pixels of coordinates (x,y) and value p, x', the determination of the bad pixels p M x,y results in:

[0036] A. Calculation of two types of indicators:

[0037] the mean and standard deviation of the values ​​of each pixel for the given image dt: Pieta 100381 / jf

[0039] the variation over time of each pixel px,y, by calculating: • the standard deviation of the values ​​for a given pixel px,y (a position in the sensor) over the set of n captured images: (jx,y - -1V n^l • the mean and standard deviation of the^r^ for all positions: Hvet(Jv / \2

[0040] B. of the evaluation of each pixel p,x,y with respect to predetermined conditions, a pixel p,x,y being considered a bad pixel pMx,y if it meets one of the following four conditions:

[0041] 3 ig [Lu] p*? > ji.+k^

[0042] 3 î G [ Lu]P x,y < 11.- k b (J) i

[0043] 0^ > p v + k^Œy

[0044] ax ^ < p v - k d .0 v

[0045] ka, kh, kc and kd, being within the interval [0.10], the coefficients ka, kh, kc and kd being chosen according to the expected detection results, and according to the quality of the sensor used

[0046] According to the invention, to perfect the processing of the captured images, all the bad pixels p Mx,y are processed, for example by selecting one of the following two operations: replace them with a combination of neighboring pixels p, x, y; exclude them from all additional image correction processes

[0047] In the first case, that is to say for the first operation mentioned above, the treatment of bad pixels may consist of replacing a pixel with its neighbors by performing one of the following treatments: calculate the median of 4 neighboring pixels of the same color; calculate the median of 8 neighboring pixels of the same color; calculate the average of 4 neighboring pixels of the same color; Calculate the average of 8 neighboring pixels of the same color.

[0048] The additional processing consisting of modifying or eliminating bad pixels effectively contributes to optimizing the quality of images already processed by the system via the universal calibration images described above, and is therefore preferably implemented in all processing of real images acquired by the shooting device whose configuration is further developed in the following.

[0049] The invention thus also relates to an optical device for capturing images of the sky for implementing the image processing method as described above, which is provided as follows: an optical sighting device of the tube type equipped with at least one optical lens; a real data image acquisition device L j of the camera type coupled to said optical sighting device; and drive means for the assembly comprising the optical sighting device and the image acquisition device, said drive means being motorized in an automated manner in order to compensate for the rotation of the earth; a unit for processing optical signals from captured images of the sky.

[0050] In summary, the invention relies on an optical system that allows: to aim at a celestial object using a focusing optical tube with lenses; to capture images of the target object using a standard acquisition device such as a camera; - to track this object by compensating for the Earth's rotation to allow for long-duration data acquisition while maintaining relative immobility between the optical device and the target; and - to process the images of the sky thus captured.

[0051] Other objects and advantages of the present invention will become apparent in the following description, relating to an essential aspect of the process which is given, however, only by way of illustrative and non-limiting example. Thus, in particular, the understanding of this description will be generally facilitated by referring to the figure attached in the appendix, in which:

[0052] [Fig.1] shows a schematic diagram of the general processing of optical signals specific to each image captured by the optical device of the invention.

[0053] With reference to the single figure, and as already mentioned, as soon as a new image of real data L is available, the invention must produce a corrected image L'j. Generally, the method aims to determine, for each image, two coefficients cd and ch such that L', the image modified according to the present invention, is equal to L - Cd XD - Ch X B. For an image j, the two coefficients will be denoted cjd and cjh, for a general formula which is then written L'j = Lj - Cd XD - X BJ JJJ

[0054] In practice, as shown in the figure, for each real data image L j, the image is divided into m zones of N pixels, N being less than or equal to the size s of L, i.e. the number s of pixels of the image Lj.

[0055] For each zone Zjk, k in [l..m], the objective is to obtain the pair (c jk d, c jkh ) such that the following function is minimal: cjk^ = ~N j,k^,y " cjk X Dx,y " ^jk X Bx,y "

[0057] With ^jk — N ^x^ jkyy " C <jk x dx,y - c bx,y

[0058] As mentioned previously, any optimization method for finding the pairs (cjkd, cjkh) minimizing θ can be used, in particular by implementing an iterative search for the optimum, in this case in two steps. The idea is first to sweep, with a predetermined step size, a search range for cjkd and a search range for cjkh, with the objective of finding the minimum value of θjk. In a second step, to find said optimal parameters, these values ​​are used to retain the minimum value of a parabola passing through the point cjkd (respectively cJkh).

[0059] For a real data image L j, the pair (c jb, c jd ) that we are looking for, considered as giving the optimal parameters ch, cd mentioned in the figure, is then obtained by combining (c jkb, c jkd) via the formula where the function h is preferably a mean function or a median function.

[0060] The corrected image j finally results from this pair (cj.b, c jd), according to the formula the above data applied to image j, a formula which is in practice applied to each pixel with coordinates (x, y), i.e.

[0061] According to a further processing mentioned above, the corrected real data image L' 7 is further reworked as part of the signal processing carried out by the process of the invention in order to optimize it, by means of a further filtering carried out on the basis of the list of bad pixels mentioned above, a list which is stored by the system after the operations which aim to identify them, as detailed above.

[0062] The configuration and processing examples mentioned above are obviously not exhaustive of the invention, which includes, for example, optical devices using other sighting devices than telescopes, other methods of detecting bad pixels, other optimization methods for finding pairs (c jkd, c jkh ) minimizing / ^, etc.< / jk>

Claims

Demands

1. A method for processing images of the sky taken by an optical sky image capture device, based on the use of calibration images consisting of dark images and bias images, characterized by: - ​​the determination of a universal master dark image D; - the determination of a universal master bias image B; - the correction of the sky images by means of the universal master dark image D and the universal master bias image B in order to obtain a corrected image L'j.

2. A method for processing images of the sky according to any one of the preceding claims, characterized in that the universal master dark image is obtained by combining n individual dark images di acquired at times i by an optical sensor placed in the dark, with a gain g, an exposure time e and at a temperature t.

3. A method for processing images of the sky according to any one of the preceding claims, characterized in that the universal master bias image is obtained by combining n individual bias images bi acquired at times i by an image acquisition device placed in complete darkness, with a gain g, an exposure time e and at a temperature t.

4. A method for processing images of the sky according to the preceding claim, characterized in that the calculation of the universal master bias image is obtained by the formula = ( ]jj, the function g being a mean or a median, the bi images being acquired with the maximum gain of the image acquisition device, with the minimum exposure time permitted by the acquisition device, at room temperature.

5. A method for processing sky images according to the preceding claim, characterized in that the calculation of the universal master darkness image is obtained by the formula Σ) = g Σ=l function g being a mean or a median, the images dt being acquired with the maximum gain of the image acquisition device, with an exposure time greater than the maximum exposure time used for acquiring images of the sky, at room temperature.

6. A method for processing images of the sky according to the preceding claim, characterized in that, for each acquisition of a real data image, the calculation of a corrected image L'j is carried out according to the following steps: - fractionation of the real data image Lj into m zones of N pixels, N being less than or equal to the number 5 of pixels constituting the image; - for each zone ZJk, k belonging to the interval [l..m], determination of a pair (c jkd, c jkh ) such that the following function is minimal: fjk^Cjk' cjk) = "N ^xyi^j-kjy ' cjk with ^jk = ' cjk X ®xy ■ X calculation of two coefficients function h being a mean or a median; determination of the corrected image L'j for each pixel pt of coordinates (x, y) of the image of real data L j:

7. V(x, y) L y = LY - x D x y - cx A method for processing images of the sky according to the preceding claim, characterized in that the determination of a pair (c Jk d, c jkh ) for the function to be minimal is carried out in two steps iteratively: by traversing, according to a predetermined step, a search range for c Jk d and a search range for c Jk h, in order to find the minimum value defjk; by retaining the minimum value of a parabola passing through the point c jk d (respectively c,kh) identified in the previous step.

8. A method for processing images of the sky according to claim 6, characterized in that the values ​​c jk h are provided to be fixed and equal to 1.

9. A method for processing images of the sky according to claim 6, characterized in that the values ​​c Jk ^ are provided to be fixed and equal to 0.

10. A method for processing images of the sky according to any one of the preceding claims, characterized by the implementation of a detection and identification of bad pixelsp M related to the image acquisition device.

11. A method for processing images of the sky according to the preceding claim, characterized by the additional correction of each corrected image L j by filtering and processing the bad pixels p M detected.

12. A method for processing images of the sky according to the preceding claim, characterized in that the bad pixels p M are placed in a list H.

13. A method for processing sky images according to any one of claims 10 to 12, characterized in that, for each individual dark image or image fraction dt containing s pixels of coordinates (x,y) and value pt, the determination of the bad pixels pM results from: a. the calculation of two types of indicators: • the mean and standard deviation of the values ​​of each pixel for the given image di: 2 - fi) * the variation over time of each pixel p by calculating: • the standard deviation of the values ​​for a given pixel p (a position in the sensor) over the set of n captured images: ~ ,y) • the mean and standard deviation of the for all positions: pv6t(Jv \2 Pv = - Pv) b. of the evaluation of each pixelp t with respect to predetermined conditions, a pixel p , * ' being considered a bad pixel p M if it meets one of the following four conditions: 3 i € [ Ln ] > ji.+k^ Bie['Ln]pxi'y<]ii-kir(ji ax'y>pv+kc. <jv vxy<liv-kd.(jv k a, h, cetkd, étant compris dans l’intervalle [0..104n], les coefficients a,k h,k c et d choisis en fonction des résultats de détection attendus, la qualité du capteur utilisé.

14. A method for processing sky images according to any one of claims 10 to 13, characterized in that all bad pixels p Mx,y are processed by selecting one of the following two operations: - replacing them with a combination of neighboring pixels p tx,y; - excluding them from all additional image correction processing L'j.

15. A method for processing images of the sky according to the preceding claim, characterized in that the processing of bad pixels consists of replacing a pixel with its neighbors by performing one of the following processing methods: - calculating the median of 4 neighboring pixels of the same color; - calculating the median of 8 neighboring pixels of the same color; - calculating the average of 4 neighboring pixels of the same color; - calculating the average of 8 neighboring pixels of the same color.

16. An optical apparatus for capturing images of the sky for implementing the image processing method according to the preceding claims, characterized in that it comprises: an optical sighting device of the tube type equipped with at least one optical lens; a real data image acquisition device L; of the camera type coupled to said optical sighting device; and drive means for the assembly comprising the optical sighting device and the image acquisition device, said drive means being motorized in an automated manner in order to compensate for the rotation of the earth; a unit for processing optical signals from captured images of the sky.< / jv>