Method for processing a raw image, and associated computer program product and processing device

Abstract

The invention relates to a method (20) for processing a raw image acquired by a camera module having an optical transfer function, the method comprising the steps of: - receiving (20) an enlargement instruction indicative of a portion of the raw image to be enlarged forming an input image; - initially calculating (24) an estimated image, by deconvoluting the optical transfer function of the input image; - iteratively implementing a processing loop (26) comprising: - calculating a current simulated image from the current estimated image and the optical transfer function; - determining a cost function; - if a predetermined stop criterion is reached, removing from the processing loop, and otherwise updating the estimated image from the cost function determined; - providing (28), at the output, a rendered image equal to the current estimated image.

Description

Title: Method for processing a raw image, computer program product and associated processing device DESCRIPTION Field of invention

[0001] The present invention relates to a method for processing a raw image from a camera module.

[0002] The invention also relates to a computer program, a processing device configured to implement such a method, and an electronic apparatus comprising such a processing device.

[0003] The invention applies to the field of digital image sensors, in particular digital cameras and digital cameras, for example for telephones, tablets or laptops. State of the art

[0004] It is known to design so-called color filter array image sensors. More precisely, in such a sensor, a mosaic of color filters is arranged on a matrix of photosites forming a detector, each color filter being placed opposite a respective photosite.

[0005] For example, the mosaic of color filters is a regular mosaic whose elementary pattern forms a Bayer matrix. Such a Bayer matrix consists of four filters arranged in two rows of two filters: two green filters (i.e., rejecting photons outside the green range) form the diagonal of the matrix, while the other two locations are respectively occupied by a blue filter (rejecting photons outside the blue range) and a red filter (rejecting photons outside the red range).

[0006] Such sensors have the advantage of detecting multiple color bands through a single optical system that projects the observed scene onto a single image plane, namely the detector plane. Therefore, due to their optical simplicity, such sensors are commonly used in the camera modules of smartphones.

[0007] Due to the presence of the mosaic of colored filters, each color is detected only at a fraction of the photosites (namely those associated with the corresponding filter). The raw image thus obtained is called "mosaiced", and can be decomposed into three images R (red), G (green), B (blue), aligned with each other to the first order, corresponding to the nested lattices of colored filters.

[0008] Then, for each pixel in the raw image, the value of the missing channels (red and green, green and blue, or red and blue) is obtained by interpolation, called "demosaicing", from the value of said channels in the neighboring pixels. Other image processing algorithms, such as noise reduction (for dark areas of the image) or enhancement of high spatial frequencies (to compensate for losses due to noise reduction), are also likely to be implemented.

[0009] In this way, a final image is obtained, in which each pixel is associated with a photosite and includes a value for each red, green and blue channel.

[0010] However, such a process is not entirely satisfactory.

[0011] Indeed, when performing a digital zoom on the final image thus obtained, it is common to observe artifacts, such as colored fringes on either side of an object in the imaged scene, for example red on one side, blue on the other. In addition, after such a digital zoom, the sharpness is often insufficient, and even below the pixelation step of the image.

[0012] This results, in part, from chromatic aberrations of the optical system, which lead to shifts in the focal points of the three R, G, B components of light coming from the scene, introducing errors during the demosaicing operation.

[0013] Furthermore, the focal point of the optical system is not always smaller than or equal to the size of the detector pixels (or the size of a Bayer array). Therefore, the zoomed image is likely to contain areas with insufficient sharpness.

[0014] An aim of the present invention is to remedy at least one of the drawbacks of the state of the art.

[0015] Another object of the invention is to provide a method which, when applying a digital zoom, provides an output image of satisfactory quality.

[0016] In particular, an aim of the invention is to propose a method which does not lead, when applying the digital zoom, to the appearance of artifacts, and / or for which the image provided as output presents fewer sharpness problems than in the method of the state of the art.

[0017] Another aim of the invention is to propose a method which has these advantages while being fast and requiring little computing resources. Statement of the invention

[0018] To this end, the invention relates to a method of the aforementioned type, in which the raw image is representative of a scene acquired by a sensor of the camera module through an optical system of the camera module, the camera module being associated with an optical transfer function between the scene and the sensor, the method being implemented by computer and comprising the steps: - receiving an enlargement instruction indicative of a part of the raw image, strictly smaller than the raw image and forming an input image, to be enlarged; - initial calculation of an estimated image, by deconvolution of the optical transfer function of the input image; - iterative implementation of a processing loop comprising: • calculation of a current simulated image from the current estimated image and the optical transfer function; • determination of a cost function from at least one of the current simulated image and the current estimated image; • if a predetermined stopping criterion is reached, exit from the processing loop, otherwise, update of the estimated image from the determined cost function; - providing, at output, a restored image equal to the current estimated image.

[0019] Indeed, thanks to such a process, the use of the optical transfer function makes it possible to overcome, to the first order, the defects introduced by the optical system of the camera module. Then, thanks to the iterative implementation of the calculation loop, a convergence towards an image which is the most faithful to the captured scene is acquired.

[0020] Furthermore, taking into account the enlargement instruction leads to implementing the calculation loop only on a part of the image, which reduces the number of calculations to be carried out, for example for deconvolution or updating the estimated image.

[0021] Finally, by taking into account the enlargement instruction, it is more likely that the part of the image considered contains an object located at a single distance from the sensor, compared to the situation where the entire captured scene is considered. In this case, it is likely that the optical transfer function is uniform over the entire part considered, or at least over a non-negligible fraction of it. As a result, the need for computing resources is further reduced.

[0022] Advantageously, the method according to the invention has one or more of the following characteristics, taken in isolation or in any technically possible combination(s):

[0023] the enlargement instruction is also indicative of a desired size of the rendered image;

[0024] the determination of the cost function includes: - the calculation of a difference term from a comparison between the current simulated image and the input image; and / or - the calculation of a penalty term from characteristics of the current estimated image;

[0025] the difference term is a function of differences between the pixel values ​​of the current simulated image and the input image;

[0026] the penalty term includes at least one of: - an intensity component which is an increasing function of the intensity variations of at least one channel between neighboring pixels of the estimated image; and / or - a color component which is an increasing function of the variations in hue and / or saturation and / or brightness between neighboring pixels of the estimated image;

[0027] updating the estimated image includes a minimization of the determined cost function;

[0028] the initial calculation step of the estimated image includes: - determination of a Fourier transform of the optical transfer function; - determination of an inverse optical frequency transfer function by inversion of the coefficients of the determined Fourier transform; - implementation: • a first calculation comprising: o determination of an intermediate frequency image, equal to the result of a multiplication of a Fourier transform of the input image by the inverse optical frequency transfer function; and o calculation of an inverse Fourier transform of the intermediate frequency image to obtain the estimated image, or • a second calculation including: o determination of an inverse optical transfer function equal to an inverse Fourier transform of the frequency inverse optical transfer function; and o convolution of the input image with the determined inverse optical transfer function to obtain the estimated image;

[0029] the method further comprises receiving distance information representative of distances between objects in the scene and the sensor, the optical transfer function implemented in the calculation of the current simulated image depending on the distance information;

[0030] the sensor comprises an array of photosites associated with an array of color filters, each color filter masking an input surface of a respective photosite.

[0031] According to another aspect of the invention, there is provided a computer program comprising executable instructions which, when executed by computer, implement the steps of the method as defined above.

[0032] The computer program can be in any computer language, such as machine language, C, C++, JAVA, Python, etc.

[0033] According to another aspect of the invention, there is proposed a device for processing a raw image from a camera module, the raw image being representative of a scene acquired by a sensor of the camera module through an optical system of the camera module, the camera module being associated with an optical transfer function between the scene and the sensor, the processing device being configured to: - receive an enlargement instruction indicative of a part of the raw image, strictly smaller than the raw image and forming an input image, to be enlarged; - carry out an initial calculation of an estimated image, by deconvolution of the optical transfer function of the input image; - iteratively implement a processing loop comprising: • calculation of a current simulated image from the current estimated image and the optical transfer function; • determination of a cost function from at least one of the current simulated image and the current estimated image; • if a predetermined stopping criterion is reached, exit from the processing loop, otherwise, update of the estimated image from the determined cost function; - provide, as output, a restored image equal to the current estimated image.

[0034] The device according to the invention can be any type of device such as a server, a computer, a tablet, a calculator, a processor, a computer chip, programmed to implement the method according to the invention, for example by executing the computer program according to the invention.

[0035] According to another aspect of the invention, there is provided an electronic apparatus comprising a processing device as defined above, a user interface and a camera module configured to acquire at least one raw image, the processing device being connected to the camera module to receive, as input, all or part of the raw images acquired by the camera module, and being configured to process each raw image received from an enlargement instruction received from the user interface.

[0036] Preferably, the electronic device is a smart mobile phone (or "smartphone" in English), also called a ordiphone or multifunction mobile phone).

[0037] In this case, the invention also relates to a use of a processing device according to the invention and / or a processing method according to the invention, within the smart mobile telephone, for processing at least one raw image acquired by the camera module of the smart mobile telephone.

[0038] Alternatively, the electronic device is a touchscreen tablet.

[0039] In this case, the invention also relates to a use of a treatment device according to the invention and / or a treatment method. according to the invention, within the touchscreen tablet, to process at least one raw image acquired by the camera module of the touchscreen tablet. Brief description of the figures

[0040] The invention will be better understood on reading the description which follows, given solely as a non-limiting example and made with reference to the appended drawings in which:

[0041] Figure 1 is a schematic representation of an image processing system comprising an enlarging device according to the invention; and

[0042] Figure 2 is a flowchart of an enlargement method according to the invention, implemented by the device of Figure 1.

[0043] It is understood that the embodiments which will be described below are in no way limiting. In particular, it is possible to imagine variants of the invention comprising only a selection of characteristics described below isolated from the other characteristics described, if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection includes at least one preferably functional characteristic without structural details, or with only part of the structural details if it is this part which is only sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art.

[0044] In particular, all the variants and embodiments described can be combined with each other if there is no technical obstacle to this combination.

[0045] In the figures and in the rest of the description, the elements common to several figures retain the same reference. Detailed description

[0046] An image processing device 2 (or “processing device”) according to the invention is illustrated in Figure 1.

[0047] The processing device 2 is configured to calculate a restored image, determined from a raw image representative of a scene 3 and acquired by a camera module 4, and from an enlargement instruction relating to a part of the raw image to be enlarged. Such a restored image advantageously has better sharpness and / or better resolution and / or less noise than said part of the raw image simply enlarged.

[0048] The processing device 2 and the camera module 4 are, for example, both integrated into the same electronic device 6. In this case, the processing device 2 is connected to the camera module 4 to receive, as input, each raw image acquired by the camera module 4.

[0049] The electronic device is, for example, a smartphone, a tablet, or a digital camera. Alternatively, the electronic device 6 is a video surveillance device, a vehicle, a drone, a satellite, a microscopy device, or a medical imaging device (such as a tomography imaging device).

[0050] Preferably, the electronic apparatus 6 also comprises a user interface 8 for input, by a user, of the aforementioned enlargement instruction. Such an instruction is representative of the portion of the raw image that the user wishes to have enlarged.

[0051] Subsequently, the part of the raw image that the user wishes to enlarge will be referred to as the "input image".

[0052] Optionally, the electronic device 6 also comprises a distance measurement module 10, configured to deliver distance information representative of distances between all or part of the objects of the scene 3 and a sensor (described later) of the camera module 4. The measuring module 10 is, in particular, a rangefinder, for example an optical rangefinder. Camera module

[0053] The camera module 4 comprises at least one assembly comprising an optical system 12 associated with a sensor 14.

[0054] In particular, the sensor 14 comprises an array of photosites associated with an array of color filters. In this case, each color filter is arranged to mask an input surface of a respective photosite.

[0055] The camera module 4 is associated with an optical transfer function between the scene 3 and the sensor 14. Such an optical transfer function establishes a relationship between the characteristics (spatial and spectral, in particular) of the light coming from the scene 3 and the characteristics of the light reaching the sensor 14, in particular due to the influence of the optical system 12.

[0056] For example, the optical transfer function of the camera module 4 is described by means of a point spread function of said camera module 4.

[0057] The optical transfer function is, for example, a function of at least one of: - a distance between the sensor 14 and an object of the scene 3 imaged by the sensor 14; - a distance between the optical system 12 and the sensor 14; - a distance between the optical system 12 and an object of the scene 3 imaged by the sensor 12; - a state of the optical system 12, such as a zoom, focal length and / or numerical aperture setting; - the considered photosite of the sensor; and - one or more angle(s) between the sensor 14 and the optical system 12.

[0058] The processing device 2 is configured to implement a processing method 20 (FIG. 2) to calculate the restored image from the raw image and the enlargement instruction.

[0059] Such a processing method 20 comprises a reception step 22, an initialization step 24, an iterative processing loop 26 and a restitution step 28.

[0060] The processing device 2 may be in a hardware form, such as a computer, a server, a processor, an electronic chip, etc. Alternatively, or additionally, the processing device 2 may be in a software form such as a computer program, or an application, for example an application for a user device such as a tablet or smartphone. Reception stage

[0061] More specifically, the processing device 2 is configured to receive, during the reception step 22, the raw image from the camera module 4. Alternatively, the raw image is already stored in a memory of the processing device 2. Alternatively, the raw image has been acquired by another electronic device, then transmitted to the processing device 2 for processing.

[0062] Furthermore, the processing device 2 is configured to receive, during the reception step 22, the enlargement instruction entered through the user interface 8.

[0063] As stated earlier, the enlargement instruction is indicative of the input image concerned by the processing, i.e. the part of the raw image that the user wishes to enlarge. Such an input image is strictly smaller than the raw image from which it is extracted.

[0064] Advantageously, the enlargement instruction is also indicative of a desired size of the rendered image. This is, for example, the size of a window where the rendered image will be displayed, or the size of a file in which the rendered image will be stored. This is advantageous, insofar as such an indication conditions the number of pixels of the rendered image, for example to correspond to the number of pixels of a screen on which the rendered image will have to be displayed. Initialization step

[0065] The processing device 2 is also configured to implement, during the initialization step 24, an initial calculation of an estimated image. Said initially calculated estimated image is intended to be subsequently updated iteratively by the implementation of the iterative processing loop 26 (described later).

[0066] More specifically, the processing device 2 is configured to calculate the initial estimated image from the input image, by deconvolution of the optical transfer function of the input image.

[0067] Advantageously, during such a calculation, the distance information from the measurement module 8 is taken into account in the choice of the optical transfer function.

[0068] In particular, for each photosite of the sensor 14, the processing device 2 is configured to perform a deconvolution of the optical transfer function for the distance of the point of the scene imaged by said photosite, obtained through the distance information from the measurement module 10. For example, in the example of FIG. 1, the value of the optical transfer function for the photosites which image the tree 30 in the background is likely to differ from its value for the photosites which image the rodent 32 in the foreground.

[0069] This is advantageous since implementing, for each element of scene 3, an optical transfer function representative of its distance to the sensor leads to an estimated image having improved sharpness on several view planes in the same image. This is particularly advantageous given that elements outside the focal plane are generally blurred, especially when the depth of field is reduced.

[0070] Preferably, the processing device 2 is configured to perform a deconvolution of the optical transfer function corresponding to the current state of the optical system 12 (optical zoom, numerical aperture, etc.).

[0071] Preferably, to calculate the initial estimated image, the processing device 2 is configured to determine a Fourier transform of the optical transfer function.

[0072] In the case of the present application, the Fourier transforms, as well as the corresponding inverse Fourier transforms, are understood in the sense of spatial frequencies, in particular in the plane of the photosite matrix.

[0073] The processing device 2 is further configured to then determine a frequency inverse optical transfer function.

[0074] More precisely, to determine the inverse frequency optical transfer function, the processing device 2 is configured to invert each of the coefficients of the previously determined Fourier transform.

[0075] Preferably, the processing device 2 is also configured to modify the value of at least a portion of the inverses obtained. In particular, the processing device 2 is configured to reduce the value of the coefficients corresponding to the highest frequencies, for example by implementing a Wiener filter.

[0076] The processing device 2 is configured to implement either a first calculation or a second calculation.

[0077] More specifically, the processing device 2 is configured to, when implementing the first calculation: - determine an intermediate frequency image, equal to the result of a multiplication of a Fourier transform of the input image by the inverse frequency optical transfer function obtained; and - calculate an inverse Fourier transform of the intermediate frequency image: the result of such a calculation forms the initial estimated image.

[0078] Furthermore, the processing device 2 is configured to, when implementing the second calculation: - determine an inverse optical transfer function, equal to an inverse Fourier transform of the frequency inverse optical transfer function; and - convolve the input image with the determined inverse optical transfer function: the result of such a calculation forms the initial estimated image.

[0079] Such a second calculation, although conceptually simpler, is generally more resource-intensive than the first calculation, especially if the optical transfer function is uniform.

[0080] Advantageously, to take into account the spatial spread of the optical point (i.e. the focal point), the input image is increased by a band of pixels neighboring the raw image, during the first or second calculation.

[0081] According to another advantageous aspect, the estimated image size is a function of the desired size included in the enlargement instruction.

[0082] As previously indicated, at the end of the initialization step 24, the initial estimated image is obtained. Iterative processing loop

[0083] Furthermore, the processing device 2 is configured to implement, during step 26, an iterative processing loop.

[0084] The iterative processing loop includes a phase of calculating a current simulated image, a subsequent phase of determining a cost function, and a phase of determining whether or not a predetermined stopping criterion has been reached.

[0085] More specifically, during a given iteration of the processing loop, the processing device 2 is configured to first calculate the current simulated image.

[0086] More specifically, the processing device 2 is configured to calculate the current simulated image from the current estimated image and the optical transfer function. In particular, at the first iteration of the iterative processing loop, the current estimated image is the initial estimated image obtained at the end of the initialization step 24.

[0087] For example, the current simulated image is equal to a result of the convolution of the current estimated image and the optical transfer function.

[0088] Furthermore, the processing device 2 is configured to then determine the cost function E from at least one of the current simulated image and the current estimated image.

[0089] Preferably, to determine the cost function E, the processing device 2 is configured to: - calculate a difference term D, from a comparison between the current simulated image and the input image; and / or - calculate a penalty term P from characteristics of the current estimated image.

[0090] Preferably, the difference term D depends on differences between the pixel values ​​of the current simulated image and the input image.

[0091] By "difference between the values ​​of the pixels of the current simulated image and the input image" is meant the difference, for at least one pixel of the input image, between the value of said pixel and the value of the corresponding pixel of the current simulated image.

[0092] For example, the difference term D includes a component equal to the sum of the squares of the pixel-by-pixel deviations between the input image and the current simulated image.

[0093] Alternatively, or additionally, the difference term D includes a component equal to the sum of the absolute values ​​of the differences, pixel by pixel, between the input image and the current simulated image.

[0094] Alternatively, or additionally, other components are also possible, provided that their value decreases when the difference, in absolute value, between the value of a pixel of the current simulated image and the value of the corresponding pixel of the input image decreases.

[0095] For the purposes of the present invention, the term "pixel value" means the values ​​R, G, B, or brightness, hue, saturation. Conventionally, brightness, hue and saturation are obtained by a linear combination of the components R, G, B.

[0096] Preferably, the penalty term P comprises an intensity component which is an increasing function of the intensity variations of at least one channel between neighboring pixels of the estimated image. In this way, for each channel among the red, green and blue channels of the estimated image, the penalty term is all the greater as the value of the intensity jumps between a pixel and one of the neighboring pixels is large.

[0097] Alternatively, or in a complementary manner, the penalty term P includes a color component which is an increasing function of the variations in hue and / or saturation and / or brightness between neighboring pixels of the estimated image. In this way, the penalty term is all the greater as the value of the jumps in hue and / or saturation and / or brightness between a pixel and one of the neighboring pixels is large.

[0098] Alternatively, or in a complementary manner, the penalty term P includes a regularity component whose value is lower the more regular the contours of the objects present in the image are, i.e. there are few changes of direction. This is advantageous in the case where a pixel density required for the rendered image is higher than that of the sensor. Consequently, a minimization of the regularity component results in contours of the objects, on the rendered image, which are more regular than the slots corresponding to the pixels of the sensor which would have been obtained at equal density between the rendered image and the input image.

[0099] Finally, the processing device 2 is configured to then determine whether the predetermined stopping criterion is reached.

[0100] Such a predetermined criterion is, for example, the achievement of a predetermined number of iterations of the iterative processing loop, or a variation (absolute or relative) of the cost function, compared to the previous iteration, below a predetermined threshold.

[0101] Furthermore, the processing device 2 is configured to, when the loop exit criterion is not met, update the estimated image from the determined cost function E.

[0102] Advantageously, the processing device 2 is configured to update the estimated image so as to minimize the cost function E. In this way, the estimated image converges towards the image which would have been acquired with an optical system 12 allowing focusing of the light coming from each point of the scene 3 on an area of ​​the order of the dimensions of a photosite of the sensor 14, or even smaller.

[0103] Also, in particular, due to the consideration of distance in the optical transfer function, sharpness is progressively achieved both for near objects (e.g., rodent 32) and far objects (e.g., tree 30), a situation in which the cost function E is minimal.

[0104] For example, the processing device 2 is configured to calculate a gradient of the cost function E as a function of parameters of the estimated image. Such parameters are, in particular, the parameters on which the difference term D and / or the penalty term P depend.

[0105] Alternatively, or additionally, the processing device 2 is configured to perform such calculations on the basis of objects represented on the estimated image, and not on the basis of the pixels of the estimated image. In this case, the processing device 2 is configured to identify such objects on the basis, for example, of the value of certain properties R, G, B and / or of variations, in the image, of these properties (for example spatial gradient of the values ​​R, G, B), and / or on the basis of the position and / or the shape of said objects. In this case, the processing device 2 is, for example, configured to calculate the gradient of the cost function E with respect to the parameters of these objects, rather than directly in relation to the pixels they contain. This potentially has certain advantages, such as faster convergence, or even better definition of the contours of the resulting objects. As an illustrative example, if the shape to be rendered is a circle, it may be faster and less computationally intensive to calculate the color value of the circle, and to move it and change its diameter. In addition, the contours of the objects obtained in the image may be more regular than their representation by pixels

[0106] In both cases, the processing device 2 is configured to modify the estimated image with respect to the calculated gradient, in particular in a direction opposite to the calculated gradient.

[0107] Alternatively, the processing device 2 is configured to update the estimated image by a Newton gradient method or an annealing method, based on the calculated cost function E. Restitution stage

[0108] Finally, the processing device 2 is configured to, when the stopping criterion is reached, provide, as output, a restored image equal to the current estimated image, i.e. the estimated image calculated during the last implementation of the processing loop.

[0109] For example, the rendered image is displayed on a screen, or stored in memory, for display or subsequent processing or use. Functioning

[0110] The operation of the processing device 2 will now be described with reference to Figure 2.

[0111] During the reception step 22, the processing device 2 receives the raw image previously acquired by the camera module 4, as well as the enlargement instruction entered through the user interface 8.

[0112] Then, during the initialization step 24, the processing device 2 implements the initial calculation of the estimated image, by deconvolving the optical transfer function of the input image.

[0113] Then, during step 26, the processing device 2 implements the iterative processing loop.

[0114] More precisely, at each iteration of the processing loop, the processing device 2: - calculates the current simulated image; - determines the cost function E; and - updates the estimated image, if the predetermined stopping criterion is not reached.

[0115] Finally, when the stopping criterion is reached, the processing device 2 provides, as output, the restored image, equal to the estimated image calculated during the last implementation of the processing loop 26.

Claims

CLAIMS 1. Method (20) for processing a raw image from a camera module (4), the raw image being representative of a scene (3) acquired by a sensor (14) of the camera module (4) through an optical system (12) of the camera module (4), the camera module (4) being associated with an optical transfer function between the scene (3) and the sensor (14), the method being implemented by computer and comprising the steps: - reception (20) of an enlargement instruction indicative of a part of the raw image, strictly smaller than the raw image and forming an input image, to be enlarged; - initial calculation (24) of an estimated image, by deconvolution of the optical transfer function of the input image; - iterative implementation of a processing loop (26) comprising: • calculation of a current simulated image from the current estimated image and the optical transfer function; • determination of a cost function from at least one of the current simulated image and the current estimated image; • if a predetermined stopping criterion is reached, exit from the processing loop, otherwise, update of the estimated image from the determined cost function; - supply (28), at output, of a restored image equal to the current estimated image.

2. The method of claim 1, wherein the enlargement instruction is also indicative of a desired size of the rendered image.

3. The method of claim 1 or 2, wherein determining the cost function comprises: - the calculation of a difference term from a comparison between the current simulated image and the input image; and / or the calculation of a penalty term from characteristics of the current estimated image.

4. The method of claim 3, wherein the difference term is a function of differences between the values ​​of the pixels of the current simulated image and the input image.

5. A method according to claim 3 or 4, wherein the penalty term comprises at least one of: - an intensity component which is an increasing function of the intensity variations of at least one channel between neighboring pixels of the estimated image; and / or - a color component which is an increasing function of the variations in hue and / or saturation and / or brightness between neighboring pixels of the estimated image.

6. Method according to any one of claims 1 to 5, wherein the updating of the estimated image comprises a minimization of the determined cost function.

7. Method according to any one of claims 1 to 6, in which the step of initially calculating the estimated image comprises: - determination of a Fourier transform of the optical transfer function; - determination of an inverse optical frequency transfer function by inversion of the coefficients of the determined Fourier transform; - implementation: • a first calculation comprising: o determination of an intermediate frequency image, equal to the result of a multiplication of a Fourier transform of the input image by the inverse optical frequency transfer function; and o calculation of an inverse Fourier transform of the intermediate frequency image to obtain the estimated image, or • a second calculation comprising: o determination of an inverse optical transfer function equal to an inverse Fourier transform of the inverse frequency optical transfer function; and o convolution of the input image with the determined inverse optical transfer function to obtain the estimated image.

8. Method according to any one of claims 1 to 7, further comprising receiving distance information representative of distances between objects (30, 32) of the scene and the sensor (14), the optical transfer function implemented in the calculation of the current simulated image depending on the distance information.

9. Method according to any one of claims 1 to 8, in which the sensor (14) comprises an array of photosites associated with an array of color filters, each color filter masking an input surface of a respective photosite.

10. A computer program comprising executable instructions which, when executed by a computer, implement the steps of the method according to any one of 1 to 9.

11. Device (2) for processing a raw image from a camera module (4), the raw image being representative of a scene acquired by a sensor (14) of the camera module (4) through an optical system (12) of the camera module (4), the camera module (4) being associated with an optical transfer function between the scene (3) and the sensor (14), the processing device (2) being configured to: - receive an enlargement instruction indicative of a part of the raw image, strictly smaller than the raw image and forming an input image, to be enlarged; - carry out an initial calculation of an estimated image, by deconvolution of the optical transfer function of the input image; - iteratively implement a processing loop comprising: • calculation of a current simulated image from the current estimated image and the optical transfer function; • determination of a cost function from at least one of the current simulated image and the current estimated image; • if a predetermined stopping criterion is reached, exit from the processing loop, otherwise, update of the estimated image from the determined cost function; - provide, as output, a restored image equal to the current estimated image.

12. Electronic apparatus (6) comprising a processing device (2) according to claim 11, a user interface (8) and a camera module (4) configured to acquire at least one raw image, the processing device (2) being connected to the camera module (4) to receive, as input, all or part of the raw images acquired by the camera module (4), and being configured to process each raw image received from an enlargement instruction received from the user interface (8).

13. Electronic device (6) according to claim 12, the electronic device (6) being a smart mobile phone or a touchscreen tablet.

14. Use of a processing method according to any one of claims 1 to 9, or of a processing device (2) according to claim 11, in an electronic device (6) according to claim 12 or 13, for processing at least one raw image acquired by the camera module (4) of the electronic device (6).

Images