Method for obtaining a higher-resolution raw image of a scene, and computer program, device and system implementing such a method

Abstract

The invention relates to a method for obtaining a higher-resolution raw image (IMBC), referred to as the target raw image, of a scene, from: - one lower-resolution raw image (IMBP), referred to as the primary raw image, and - at least one other lower-resolution raw image (IMBS1-IMBSl), referred to as the secondary raw image, of the scene, the method (200) comprising the following steps: - initializing (204) a matrix, referred to as the target matrix, of value patterns to represent the target raw image (IMBC); - copying, into the target matrix, value patterns from the primary raw image (IMBP); - enriching the target matrix by adding, to the target matrix, value patterns originating from the secondary images (IMBS1-IMBSl). The invention also relates to a computer program, to a device, to a system and to a vehicle implementing such a method.

Description

DESCRIPTION Title: Method for obtaining a higher-resolution raw image of a scene, computer program, device and apparatus implementing such a method

[0001] The present invention relates to a method for obtaining a higher-resolution raw image of a scene from several lower-resolution raw images of said scene. It also relates to a computer program, a device, and an apparatus implementing such a method.

[0002] The field of the invention is the field of signal processing captured by an image sensor of a camera module in order to obtain an image. State of the art

[0003] Demosaicing, also called debayering, is a process for processing the raw signal, also called the raw image, from the sensor of a digital camera. Demosaicing can, for example, involve interpolating the data provided by each of the monochrome photosites—for example, red, green, and blue—that make up the image sensor to obtain a tri-color value for each pixel of the image, in the case of an RGB image, for example.

[0004] The image sensor, or photographic sensor, of a camera module typically comprises red, blue, or green photosites. Each photosite reacts to the light it receives and converts it into an electrical charge, similar to a photoelectric cell. The most common arrangement of photosites in an image sensor is called the Bayer matrix. This matrix is ​​formed by a multitude of photosite patterns, called Bayer patterns, each pattern generally being square and comprising two green photosites arranged diagonally, and one red and one blue photosite arranged on the opposite diagonal. The signal received from such an image sensor is demosaiced, and there are widely known demosaicing techniques.

[0005] In an image scene, a fine pattern repeating an alternation of brightness, such as a bird's feather, and whose spatial periodicity closely matches or is specifically related to that of the color detection alternation pattern, generally within the Bayer matrix, can also produce periodic over- and underexposures of certain colored pixels within the pattern relative to others. A gray feather can thus appear with false iridescence reminiscent of the colors of the rainbow, resulting in an inaccurate representation of the real scene.

[0006] These problems can be partially solved by applying demosaicing rules involving neighboring pixels, which minimize the effects described above. However, existing solutions are generally not completely satisfactory at the edges of objects, or when the effective size of the object projected onto the detection matrix is ​​of a comparable order of magnitude to that of the detection matrix. Similarly, current demosaicing solutions implicitly favor linear patterns in certain directions, which can lead to the appearance of imaginary lines, distorted object outlines, or even make them difficult to observe due to these artifacts, and so on.

[0007] One objective of the present invention is to remedy at least one of the aforementioned drawbacks.

[0008] Another aim of the invention is to provide a solution for obtaining a raw image allowing a more precise and less erroneous, and more faithful, reproduction of the real scene. Description of the invention

[0009] The invention proposes to achieve at least one of the aforementioned goals by a method of obtaining a higher-resolution raw image, referred to as the target raw image, of a scene from: - of a raw image, called the main raw image, and - of at least one other raw image, called secondary raw image; of said scene, less resolved than said target raw image; each of the said primary and secondary raw images being acquired by at least one camera module comprising an image sensor having a pattern matrix (Bi,j) of monochrome photosites; each of said primary and secondary raw images being represented by a matrix of value patterns, each value pattern being provided by a pattern of photosites of said image sensor and comprising a value provided by each monochrome photosite of said photosite pattern; said method comprising the following steps: - initialization of a matrix, called the target matrix, of patterns of values ​​to represent said raw target image; - copying into said target matrix, patterns of values ​​from the main raw image; -enrichment of said target matrix by adding, in said target matrix, at least a part of at least one pattern of values ​​from a secondary image.

[0010] Thus, the invention proposes to obtain a raw image, that is, an image not yet demosaiced, with a resolution higher than the maximum possible resolution of the image sensor used. To achieve this, the invention proposes combining a primary raw image, that is, an undemosaiced image, with part or all of at least one secondary raw image, that is, an undemosaiced image, to obtain the higher-resolution target raw image.

[0011] It should be noted that the enrichment step proposed by the invention is carried out before demosaicing, if applicable, of the target raw image.

[0012] Thus, the higher-resolution target raw image provides a more accurate and richer representation of the scene compared to each of the primary and secondary raw images, and allows for a more accurate and richer dematrixed image of the scene, and therefore one that is more faithful to the real scene.

[0013] By "raw image," we mean a set of values ​​provided by an image sensor in an imaging module, such as a CMOS sensor, a CCD sensor, or any other type of sensor. Specifically, the raw image is an image formed by a multitude of value patterns, each value pattern comprising the monochrome component values ​​provided by the monochrome photosites making up a photosite pattern of the image sensor.

[0014] In some embodiments, the image sensor may include a Bayer matrix, said Bayer matrix comprising B Bayer patterns, each Bayer pattern Bi comprising P monochrome photosites. In this case, the raw image IMB may comprise a matrix of B value patterns, each value pattern Bi being provided by a Bayer pattern Bi of the image sensor. Each value pattern Bi comprises P values, each value being provided by a monochrome photosite of the Bayer pattern Bi providing said value pattern.

[0015] In some embodiments, a Bayer pattern may comprise P=4 photosites, for example, 2 green photosites, one red photosite, and one blue photosite. In other embodiments, the Bayer pattern may be square, with the two green photosites arranged on one diagonal of the square and the red and blue photosites arranged on the other diagonal.

[0016] The demosaiced image can be represented by N sets of values, each set corresponding to a pixel in the demosaiced image. Each set of values ​​in the demosaiced image can contain C values. For example, in the case of an RGB image, each set of values ​​can contain three values: R, G, and B, one for each of the colors red, green, and blue.

[0017] Following some implementation examples, for a raw image containing B patterns, demosaicing can provide a demosaiced image containing N pixels such that N = B: in this case, each Bayer pattern contributes one pixel to the demosaiced image. Following other implementation examples, for a raw image containing B patterns of varying values, demosaicing can provide a demosaiced image containing N pixels such that N = P*B: in this case, each Bayer pattern contributes P pixels to the demosaiced image.

[0018] Depending on embodiments, for at least one pattern of values ​​from a secondary image, the matrix enrichment step target can perform the addition of only a part of said pattern of values ​​into said target matrix.

[0019] In this case, only a portion of the values ​​forming the pattern of values ​​are copied into the target matrix.

[0020] For example, a value pattern provided by a Bayer pattern comprises four values ​​(V1, R, V2, B): such a pattern therefore corresponds to a square matrix of four values. According to some embodiments, only certain values ​​can be copied: for example, only the values ​​denoted "VI" and "V2" from the monochrome green photosites, or only the value denoted "R" provided by the red photosite, or only the value denoted "B" provided by the blue photosite, etc.

[0021] According to embodiments, for at least one pattern of values ​​from a secondary image, the target matrix enrichment step can perform the addition of the entire pattern of values ​​into the target matrix.

[0022] In this case, all the values ​​forming the pattern of values ​​are copied into the target matrix.

[0023] For example, a value pattern provided by a Bayer pattern comprises four values ​​(V1, R, V2, B): such a pattern therefore corresponds to a square matrix of four values. According to some embodiments, all the values ​​of such a value pattern can be copied into the target matrix when copying said value pattern.

[0024] The two embodiments described above concerning the addition, in the target matrix, of value patterns from a secondary image, namely partial addition or total addition, can be combined.

[0025] For example, a first pattern of values ​​from a secondary image can be added totally into the target matrix, and a second pattern of values ​​from said secondary image can be added partially into the target matrix.

[0026] For example, the value patterns from a first area of ​​a secondary image can be fully added to the target matrix. and the value patterns from a second area of ​​said secondary image can be partially added into the target matrix.

[0027] For example, value patterns from a first secondary image can be added totally to the target matrix, and value patterns from a second secondary image can be added partially to the target matrix.

[0028] Of course, other combinations are possible and the examples given above are by no means limiting or exhaustive.

[0029] When a value pattern from a secondary image is added, partially or totally, to the target matrix, this pattern is not added at a random position.

[0030] The pattern is added to a position in the target matrix corresponding to the point in the scene that this pattern represents in the secondary image. In other words, if the pattern values ​​correspond to the point (x,y) in the scene, then it is added to the target matrix at a position (u,v) corresponding to that point (x,y) in the scene.

[0031] For a pattern of values ​​from a secondary image, the position (u,v) in the target matrix to which said pattern is added can be determined in different possible ways.

[0032] According to some embodiments, at least one pattern of values ​​from a secondary raw image can be added to the target raw image at a position determined by: - the position of said motif in said secondary image, and - a spatial offset between the primary raw image and said secondary raw image.

[0033] In other words, for a pattern from a secondary raw image located at a position (ul,vl) in said secondary raw image, i.e. in the matrix representing said secondary raw image, this pattern is added to a position (u2,v2) in the target matrix, this position (u2,v2) being calculated as follows: u2=ul+du v2=vl+dv with : - the offset between the main image and the secondary image, and more specifically the offset of the secondary image relative to the main image, in the direction u, and - dv the offset between the main image and the secondary image, and more particularly the offset of the secondary image relative to the main image, in the direction v.

[0034] According to some embodiments, for at least one secondary image, the spatial offset between the primary raw image and the secondary raw image, and in particular the offset of the secondary image relative to the primary image, can be known in advance.

[0035] For example, the secondary raw image can be acquired with a known spatial offset. In other words, in this case, a spatial offset is applied before the secondary raw image is acquired. This offset can be applied and determined in various ways, which are described later.

[0036] According to embodiments, for at least one secondary raw image, the spatial offset between the primary raw image and the secondary raw image, and in particular the offset of the secondary image relative to the primary image, can be measured with at least one sensor, such as for example an inertial measurement unit.

[0037] For example, the device used to capture the primary raw image and the secondary raw image may be equipped with a displacement sensor configured to measure the displacement of said device between the acquisition position of the primary raw image and the acquisition position of the secondary raw image.

[0038] According to embodiments, for at least one secondary raw image, the spatial offset between the main image and the secondary image can be determined by calculation / analysis of said main and secondary raw images, for example during a registration step.

[0039] According to embodiments, for at least one secondary raw image, the spatial offset between said secondary raw image and the primary raw image may be less than the spatial repetition step of the photosite patterns in the image sensor, such that at least one pattern of the secondary raw image is inserted in a position between two patterns of the primary raw image, and in particular in the pattern of the primary raw image corresponding to the same part of the scene, when the coordinates of the offset are positive.

[0040] This embodiment allows obtaining a target matrix with more value patterns, which allows obtaining a more resolved target raw image.

[0041] In other words, if, in the direction u of the image sensor, the spatial repetition step of the image sensor patterns is PASu, then the offset between the secondary raw image and the primary raw image in said direction u can preferably be: of <PASu (Cl) In particular, 0 < of. Alternatively, or in addition, if, in the direction v of the image sensor, perpendicular to the direction u, the spatial repetition step of the image sensor patterns is PASv, then the offset dv between the secondary raw image and the primary raw image in said direction v can preferably be: dv <PASv (C2) In particular, 0 < dv.

[0042] Depending on the embodiment, it is possible to use secondary raw images respecting only one of the Cl and C2 relationships above.

[0043] In some embodiments, it is possible to use secondary raw images of which at least one respects relation Cl, and at least one respects relation C2. In some embodiments, it is possible to use secondary raw images of which at least one respects relation Cl, at least one respects relation C2 and at least one respects both relations Cl and C2, which in the latter case ensures that the pattern of the secondary raw image can be inserted at the position of the pattern of the main raw image corresponding to the same part of the scene.

[0044] As mentioned above, for at least one secondary raw image, the spatial offset between said secondary raw image and the primary raw image can be determined by analysis of said raw images.

[0045] In this case, the process according to the invention may further include, for at least one secondary raw image, a step of registering said secondary raw image with the primary raw image before the enrichment step.

[0046] This registration step allows us to determine the spatial offset, and therefore the position of the patterns, in the target raw image.

[0047] Spatial registration can be achieved using any suitable technique.

[0048] Depending on the embodiment, for at least one secondary raw image, spatial registration can be performed by calculating the correlation between the secondary raw image and the primary raw image.

[0049] The correlation calculation can be performed on all of the said raw images.

[0050] Alternatively, when adding a given pattern from the secondary raw image to the target matrix, the correlation calculation can be performed on a limited area selected from that pattern. This could be an area centered on the pattern, or an area that begins or ends with it, or more generally, an area encompassing the pattern. The limited area can have a predetermined size smaller than the total size of the raw image, for example, a size between 10x10 and 50x50 patterns, and in particular, 30x30 patterns. The size of the limited area can be fixed or variable, determined according to the expected displacement between the raw images.

[0051] Following an example embodiment, the secondary raw image, or the limited area of ​​the secondary raw image, can be spatially offset relative to the primary raw image, and a correlation score can be calculated for each spatial offset. This operation is repeated iteratively until a maximum value for the correlation score is obtained, or a value exceeding a predetermined correlation threshold. The offset used to obtain said correlation score corresponds to the spatial offset between the primary raw image and the secondary raw image.

[0052] Following another example of implementation, the offset / re-alignment of the secondary raw image relative to the primary raw image can also be determined by the Lucas-Kanade method, or any other suitable method.

[0053] Depending on the embodiment, the correlation score can be determined individually for at least one, and in particular each, monochrome component in the secondary and primary raw images. In this case, the spatial registration is calculated for at least one, and in particular each, monochrome component individually.

[0054] Depending on the embodiment, the correlation score can be determined based on a parameter derived from at least two, in particular all, monochrome components. In this case, the registration is calculated in a common manner for said at least two monochrome components.

[0055] Depending on the embodiment, the enrichment step can be carried out individually for at least one, and in particular each, monochrome component.

[0056] Depending on the embodiment, the enrichment step can be performed for only one monochrome component.

[0057] Depending on the embodiment, the enrichment step can be performed individually for several monochrome components.

[0058] When performed individually for several monochrome components, the enrichment step can be performed in turn, or in parallel, for at least two of said, and in particular all, monochrome components.

[0059] Depending on the embodiment, the enrichment step can be carried out individually for at least one, and in particular each, secondary raw image.

[0060] Depending on the embodiment, the enrichment step can be performed only for a single secondary raw image.

[0061] Depending on the embodiment, the enrichment step can be performed individually for several secondary raw images.

[0062] When performed individually for several secondary raw images, the enrichment step can be carried out in turn, or in parallel, for at least two of said, and in particular all of the secondary raw images.

[0063] Depending on the embodiment, the, or at least one, secondary raw image can be captured by another camera module.

[0064] In other words, the aforementioned secondary raw image is captured by a different camera module: - of the camera module used to capture the main raw image, and - optionally, of the camera module(s) used to capture the other secondary raw image(s).

[0065] According to some embodiments, the secondary raw image, or at least one of them, can be captured by the same camera module as that used to capture the primary raw image.

[0066] In other words, the same camera module is used to capture: - the main raw image, and - at least one, and in particular each, secondary raw image.

[0067] Depending on embodiments, for at least a part of the scene, at least one, in particular each, secondary raw image may be spatially offset relative to the primary raw image.

[0068] In other words, for at least this part of the scene, there is a non-zero spatial offset between the primary raw image and the, or each, secondary raw image.

[0069] Spatial offset can affect only a portion of the scene. Alternatively, spatial offset can exist for the entire scene.

[0070] According to some embodiments, a spatial offset between a secondary raw image and the primary raw image, for at least a part of the scene, may be due to a moving object located in said part of the scene on at least one of said raw images.

[0071] In this case, the acquisition position of said raw images may be the same or different.

[0072] Depending on embodiments, a spatial offset between a secondary raw image and the primary raw image, for at least part of the scene, may be due to movement of the imaging device, or to the use of imaging modules positioned in different locations.

[0073] For example, two acquisition devices positioned in different locations can be used.

[0074] For example, two imaging modules equipping the same acquisition device and positioned in different locations within said acquisition device can be used, as is generally the case in a Smartphone with multiple optical lenses.

[0075] In general, for at least one secondary raw image, spatial offset can be achieved by using different viewing angles, imaging positions, etc. for the acquisition of the primary and secondary raw images.

[0076] According to advantageous embodiments, for at least one secondary raw image, the spatial shift can be obtained by controlling an image stabilizer equipping the imaging device to introduce said spatial shift.

[0077] In this case, the primary and secondary raw images are acquired from the same position for the imaging module. The image stabilizer on the imaging device is controlled to introduce the spatial offset. In other words, the stabilizer is diverted from its original function to introduce a spatial offset between the primary and secondary raw images. This allows the same device, and the same camera module, in the same position for the acquisition of the main raw image and the secondary raw image(s).

[0078] This embodiment allows for the rapid and simple, even completely transparent acquisition of the primary and secondary raw images, and therefore a more ergonomic implementation of the present invention.

[0079] For example, during the acquisition of the primary and secondary raw images, an offset in the stabilizer's positioning can be added to the stabilization command applied to the image stabilizer, to obtain an offset between the primary and secondary raw images, or between each secondary raw image. Furthermore, this offset can optionally be varied over time along at least one, and preferably both, axis of repetition of the image sensor's photosite patterns, synchronously with the image acquisitions, so as to obtain an offset in the pattern matrix from one image to the next. Optionally, but preferably, the offset can be equal to a fraction of the pattern pitch in at least one pattern repetition direction.

[0080] According to embodiments, when several secondary IMBSI-IMBSK images are acquired with different spatial offsets di-dx, preferably, said raw secondary IMBSI-IMBSK images may be acquired in an order other than the ascending order, or the descending order, of spatial offsets, to avoid any synchronization of said spatial offsets with a movement of an object in the scene.

[0081] For example, secondary raw IMBSI-IMBSK images can be acquired in a random order.

[0082] Following another example, the raw secondary IMBSI-IMBSK images can be acquired in a predetermined order different from ascending or descending order, and more generally different from a rectilinear trajectory, or even more generally according to a spatial offset different from a displacement of constant amplitude and / or sense and / or direction between each successive acquisition of the raw secondary images.

[0083] For example, assuming the use of ten secondary IMBSi-IMBSio images, each with a spatial offset from the image main, respectively di-dio, with di <d2, dg<dio, alors il est préférable de prendre les images brutes secondaires IMBS1-IMBS10 dans un ordre différent de l'ordre allant de 1 à 10 ou de 10 à 1. Par exemple, il est possible de prendre lesdites images dans un ordre aléatoire, dans l'ordre suivant : IMBSi, IMBSio, IMBS2, IMBS9, IMBS3, IMBSs, IMBS4, IMBSe et IMBSs. Bien entendu, cet ordre est donné à titre d'exemple non limitatif. De plus, ce qui vient d'être décrit en ce qui concerne l'ordre de prise des images peut être appliqué dans la direction u et / ou dans la direction v, et / ou dans les deux directions à la fois.

[0084] As mentioned above, the enrichment step to obtain the higher-resolution target raw image is carried out before demosaicing of said target raw image, if applicable.

[0085] In other words, the higher-resolution target raw image obtained after the enrichment step can then be demosaiced if needed.

[0086] According to another aspect of the same invention, an image demosaicing process is proposed comprising the following steps: - obtaining a target raw image by the method according to the invention; and - demosaicing of said raw target image to obtain a demosaicated image, noted IMD in the following without loss of generality.

[0087] The dematting stage can be carried out using any known and suitable technique.

[0088] Depending on embodiments, the demosaicing step can be carried out so as to provide one image pixel for each pattern of values ​​in the target raw image.

[0089] In this case, each pattern of values ​​in the target raw image provides the values ​​of a single pixel in the demosaiced IMD image. For example, when the image is an RGB image and each pattern is a Bayer pattern, each pattern of values ​​in the target raw image provides a set of three values. (VR,VG,VB), respectively for the monochrome red, green and blue components.

[0090] In this case, if the target matrix representing the target raw image includes NBCxMBC patterns of values, the demosaiced IMD obtained after demosaicing the target raw image will have a definition of NBC x MBC, i.e., an image of NBCxMBC pixels.

[0091] Depending on the embodiment, the demosaicing step can be carried out so as to provide an image pixel for each monochrome component of each value pattern.

[0092] In this case, each value pattern provides the values ​​for as many pixels as there are photosites, or monochrome components, in a pattern. For example, when the image is an RGB image and each pattern is a Bayer pattern comprising 4 photosites, each value pattern provides: - four sets of three values ​​(VR, VG, VB), and therefore four image pixels, respectively for the monochrome red, green, and blue components; or -three sets of three values ​​(VR,VG,VB), and therefore three image pixels, respectively for the monochrome red, green and blue components;

[0093] In this case, according to a general definition, if the target matrix representing the raw target image comprises an NBC x MBC pattern matrix, the demosaiced image (IMD) obtained after demosaicing the raw target image will have a resolution of 4xNBCxMBC, i.e., an image of 4xNBCxMBC pixels, or 3xNBCxMBC, i.e., an image of 3xNBCxMBC pixels.

[0094] In this case, for each monochrome component of each value pattern, we obtain a pixel for which: - The value of said monochrome component corresponds to the value of said monochrome component in said value pattern; - The value of each other monochrome component is estimated / interpolated from the value of said other component monochrome in said pattern and / or in neighboring or surrounding patterns.

[0095] The demosaiced IMD image provided by the demosaicing step can be used directly, for example to be stored or displayed on a display screen.

[0096] Alternatively, the demosaiced IMD image provided by the demosaicing step can undergo image processing before being used.

[0097] According to another aspect of the same invention, an image processing method is proposed comprising the following steps: - obtaining a target raw image by the process according to the invention, or a demosaiced image by the process according to the invention; and - digital processing of said target raw image, or of said demosaiced image.

[0098] The digital processing stage can perform any type of processing of the target raw image, or of the demosaiced IMD image, such as, for example, correction of an optical aberration, correction of a geometric aberration, modification of sharpness, modification of brightness, enhancement of at least one color in the image, etc.

[0099] The digital processing stage can be carried out by the same device that provides the target raw image, or the demosaicing, or by another device.

[0100] According to another aspect of the same invention, a method is proposed for imaging a scene comprising the following steps: - Acquisition of a raw image, known as the main raw image, by a camera module, -processing of said primary raw image by the method according to the invention, and in particular by: ■ the method for obtaining a target raw image according to the invention; or ■ the image demosaicing process according to the invention; or ■ the image processing method according to the invention.

[0101] The processing stage can be carried out, at least in part, in the same device that carried out the acquisition stage of the main raw image.

[0102] Alternatively, or in addition, the processing step can be carried out, at least in part, in a device other than the one that carried out the primary raw image acquisition step.

[0103] The image acquisition method according to the invention may further include, prior to the processing step, a step of acquiring at least one secondary raw image used during the processing of said primary raw image, in particular during the step of enriching the primary raw image to obtain a higher resolution target raw image.

[0104] At least one secondary raw image can be captured by the same camera module, or by another camera module, as the primary raw image.

[0105] At least one secondary raw image can be captured by the same camera module, or by another camera module, as another secondary raw image.

[0106] At least one secondary raw image can be obtained using any one of the non-limiting options described above.

[0107] According to another aspect of the same invention, a computer program is proposed comprising executable instructions which, when executed by at least one computing device, implement all the steps of the process according to the invention, and in particular: - the method for obtaining a target raw image according to the invention; or - the image demosaicing method according to the invention; or - the image processing method according to the invention; or - the method for imaging a scene according to the invention.

[0108] The computer program can be in any computer language, such as for example machine language, C, C++, JAVA, Python, etc.

[0109] Such a computer program can be presented as a standalone application. Alternatively, such a computer program can be integrated into a photo or video application, or even into an image or video playback application.

[0110] The computer program can be stored in a non-transient, or non-volatile, manner in a storage medium. [YES] According to another aspect of the invention, a device is proposed comprising means configured to implement all the steps of the process according to the invention, and in particular: - the method for obtaining a target raw image according to the invention; or - the image demosaicing method according to the invention; or - the image processing method according to the invention; or - the method for imaging a scene according to the invention.

[0112] The device according to the invention can be a computer, a processor, a computer chip, etc. programmed to implement the method according to the invention, for example by executing the computer program according to the invention.

[0113] The device according to the invention can be integrated into any type of device such as a smartphone, a tablet, a computer, a calculator, a processor, a computer chip, a medical imaging device, etc.

[0114] The device according to the invention may include, in terms of technical means, at least one, or any combination of at least two, of the characteristics described above with reference to the method according to the invention, and which are not repeated here exhaustively for the sake of brevity.

[0115] According to another aspect of the invention, a device, and in particular a user device, is proposed, comprising: - at least one camera module to acquire at least one raw image of a scene, and - a unit of calculation; configured to implement all the steps of the process according to the invention, and in particular: - the method for obtaining a target raw image according to the invention; or - the image demosaicing method according to the invention; or - the image processing method according to the invention; or - the method for imaging a scene according to the invention

[0116] The device according to the invention may comprise a single camera module, or several camera modules.

[0117] In particular, the device according to the invention can be a user device such as a smartphone, tablet, etc.

[0118] The user device according to the invention may further include a display screen.

[0119] In particular, the device according to the invention can be a computer-type user device.

[0120] The computer-type user device according to the invention may further include a display screen.

[0121] In particular, the device according to the invention can be a television.

[0122] The television according to the invention may further include a display screen.

[0123] In particular, the device according to the invention can be: - a virtual reality headset or glasses; or - an augmented reality headset, or glasses.

[0124] The helmet, or glasses respectively, according to the invention may further comprise at least one display screen.

[0125] In particular, the device according to the invention can be a medical imaging device.

[0126] In particular, the medical imaging device according to the invention can be an endoscope, an ultrasound device, etc.

[0127] The medical imaging device according to the invention may further include at least one display screen.

[0128] Of course, the device according to the invention is not limited to the examples of devices that have just been given as examples.

[0129] According to another aspect of the present invention, a vehicle comprising is proposed: - at least one camera module to acquire at least one raw image of a scene, and - a unit of calculation; configured to implement all the steps of the process according to the invention, and in particular: - the method for obtaining a target raw image according to the invention; or - the image demosaicing method according to the invention; or - the image processing method according to the invention; or - the method for imaging a scene according to the invention

[0130] The vehicle according to the invention may further include a display screen, for example arranged in a passenger compartment of the vehicle, or a projector for projecting at least one image onto a display surface generally known as a "head-up display".

[0131] The vehicle according to the invention may comprise a single camera module, or several camera modules.

[0132] According to embodiments, the vehicle according to the invention can be a land vehicle, such as a car, autonomous, semi-autonomous or non-autonomous.

[0133] According to embodiments, the vehicle according to the invention can be a flying vehicle, such as a drone, an airplane, a helicopter, autonomous, semi-autonomous or non-autonomous.

[0134] According to embodiments, the vehicle according to the invention can be a maritime vehicle, such as a boat or a submarine, autonomous, semi-autonomous or non-autonomous. Description of the figures and methods of implementation

[0135] Other advantages and features will become apparent upon examination of the detailed description of non-limiting embodiments and the accompanying drawings, in which: - FIGURE 1 is a schematic representation of a non-limiting example of an image sensor implementation; - FIGURE 2 is a schematic representation of a non-limiting example of a method according to the invention for obtaining a higher resolution target raw image of a scene; - FIGURE 3 is a schematic representation of a non-limiting example of an embodiment of a process according to the invention for demosaicing a raw image; - FIGURE 4 is a schematic representation of a non-limiting example of an embodiment of a method according to the invention for processing an image of a scene; - FIGURE 5 is a schematic representation of a non-limiting example of an embodiment of a method according to the invention for imaging a scene; - FIGURE 6 is a schematic representation of a non-limiting example of an embodiment of a device according to the invention; - Figures 7a-7c are schematic representations of non-limiting examples of embodiments of devices according to the invention; and - FIGURE 8 is a schematic representation of a non-limiting example embodiment of a vehicle according to the invention.

[0136] It is understood that the embodiments described below are by no means exhaustive. In particular, variants of the invention may be conceived comprising only a selection of features described below, isolated from the other features described, if this selection of features is sufficient to confer a technical advantage or to differentiate the invention from the prior art. This selection includes at least one feature of functional preference without structural details, or with only part of the structural details if that part is sufficient to confer a technical advantage or to differentiate the invention from the prior art.

[0137] In particular, all the variants and embodiments described can be combined with each other if there are no technical obstacles to this combination.

[0138] In the figures and in the rest of the description, elements common to several figures retain the same reference.

[0139] FIGURE 1 is a schematic representation of a non-limiting example of an image sensor that can be used for the acquisition of a raw image.

[0140] The image sensor 100 in Figure 1 comprises a Bayer pattern array. In particular, the sensor has N rows of Bayer patterns extending in a direction u, and M columns of Bayer patterns extending in a direction v perpendicular to the direction u. Thus, the sensor 100 comprises a total of B = N x M Bayer patterns. Of course, "N" and "M" are integers and may be different or equal. Furthermore, the invention is not limited to a Bayer pattern array, and the image sensor may include an array of other types of photosite patterns.

[0141] Each Bayer motif, noted Bi,j with the l <i<N et l<j<M, comprend un nombre P de photosites monochromes, choisis parmi des photosites rouge, vert ou bleu, avec P> 2. In the example shown, in no way limiting, each Bayer Bi,j pattern is square in shape and comprises four monochrome photosites, namely two green photosites V arranged on one diagonal of the Bayer Bij pattern, and one red photosite R and one blue photosite B arranged on the other diagonal of the Bayer Bi pattern.

[0142] In the example described, and in no way limitingly, the photosite patterns Bi are arranged at a step size, denoted PASu, in the u direction, and at a step size, denoted PASv, in the v direction. Of course, PASu and PASv can be different. In what follows, to avoid making the description too cumbersome, examples of implementation, but without loss of generality, we consider PASu=PASv=PAS.

[0143] When acquiring an image of a scene, the 100 sensor provides a raw image represented by a matrix of B=NM value patterns. Each value pattern is provided by a Bayer pattern Bi,j and comprises four values, each provided by a monochrome photosite of said pattern Bi,j. Thus, in the example shown, each value pattern comprises two values ​​VI and V2 for the color green, one value B for the color blue, and one value R for the color red.

[0144] Therefore, a raw image, denoted IMB, provided by the 100 sensor is represented by a matrix of NxM patterns of values. This allows obtaining an image comprising at most NxM pixels, after demosaicing the said raw IMB image.

[0145] The invention proposes a solution for increasing the resolution of a raw image acquired by the sensor 100.

[0146] FIGURE 2 is a schematic representation of a non-limiting example of a method according to the present invention for obtaining a higher-resolution raw image.

[0147] The method 200 of FIGURE 2 allows obtaining a higher-resolution target raw image, denoted IMBC hereafter, of a scene, from a primary raw image, denoted IMBP, of said scene and K secondary raw images, denoted IMBSI-IMBSK, with K>1, of said scene. In what follows, IMBSi denotes any one of the secondary raw images IMBSI-IMBSK with 1 = 1, ..., k.

[0148] Each raw image is represented by a matrix of pattern values, as shown above with reference to FIGURE 1 for example. In the following, the matrix representing the target raw image is called the target matrix, the matrix representing the primary image is called the primary matrix, and the matrix representing the secondary raw image I is called the secondary matrix I.

[0149] Each of the raw IMBP and IMBSi images can be captured by an image sensor comprising a photosite array. For example, each raw image can be captured by sensor 100 in FIGURE 1. Thus, each The raw IMBP image, IMBSi is represented by a matrix of NxM patterns of values, each pattern of values ​​comprising as many values ​​as there are photosites in each pattern of photosites of the image sensor, for example four in the example of sensor 100.

[0150] At least one of the secondary raw images IMBSI-IMBSK is spatially offset from the primary raw image IMBP for at least a portion of the scene. Preferably, each of the raw images IMBP, IMBSI-IMBSK is spatially offset from each of the other raw images IMBP, IMBSI-IMBSK for at least a portion of the scene.

[0151] Spatial offset can be achieved in different ways.

[0152] For example, if the scene includes moving objects, the spatial shift is produced by the movement of said moving objects in said scene.

[0153] Alternatively, or in addition, the spatial offset can be achieved by introducing an offset in an image stabilizer fitted to the imaging device used to capture the raw images.

[0154] Alternatively, or in addition, spatial offset can be achieved by using multiple camera modules positioned in different locations. Alternatively, or in addition, spatial offset can be achieved by capturing secondary raw images with offset acquisition directions and / or different viewing angles, etc.

[0155] Preferably, the spatial offset between the primary raw IMBP image and at least one, and in particular each, secondary raw IMBSi images is a fraction of the spatial repetition pitch of the image sensor photosite patterns.

[0156] Following a non-limiting example, it is possible to use Kl secondary raw images, IMBSI-IMBSKI, spatially offset relative to the main raw image IMBP, along the direction u. In this case, these secondary raw images IMBSI-IMBSKI can be spatially offset by an offset step of d u in said direction u. The offset step d u can be constant: in this case, according to a non-limiting example of implementation, d u = NOT U / (K1 + 1), so that the secondary raw image IMBSi, with 1 < 1 < Kl, is spatially shifted relative to the IMBP principal image by a distance du,i = ld u in the direction u. Alternatively, the offset step d u may vary from one secondary image to another.

[0157] Following a non-limiting example, it is possible to use K2 secondary raw images, IMBSI-IMBSKZ, spatially offset relative to the primary raw image, along the direction v. In this case, these secondary raw images can be spatially offset by an offset step d v in said direction v. The offset step d v can be constant: in this case, according to a non-limiting example of implementation, d v = PASv / (K2+l), so that the secondary raw image IMBSi, with 1 <I<K2, est décalée spatialement par rapport à l'image brute principale IMBP d'une distance dv,i= l.d v in the direction v. Alternatively, the offset step d v may vary from one secondary raw image to another.

[0158] K1 and K2 can be different. Alternatively, K1 = K2.

[0159] Following another embodiment, it is possible to use (Kl + 1) x (K2+1) - 1 secondary images, with Kl different spatial shifts in the u direction, K2 different spatial shifts in the v direction, and a set of secondary images shifted in the u direction and in the v direction, to cover at least some of the combinations of shifts according to the (Kl + 1) positions in the u direction and the (K2+1) positions in the v direction. Thus, the space between four Bi,j patterns of adjacent photosites is gridded according to a grid of size (Kl + l)x(K2+l): - a pattern from the main raw IMBP image corresponding to the corners of said grid; and - a repetition of a pattern from each of the secondary raw IMBSi images corresponding to at least some of each of the other positions of said grid. It is not mandatory to cover all positions of said grid, even though all positions of said grid can be covered according to certain embodiments. The number of positions covered in said grid depends on the number of secondary raw images.

[0160] In the following, without loss of generality, it is considered that the process uses K secondary raw images IMBSI-IMBSK, each of the said images having a spatial offset, respectively di-dx, relative to the main raw IMBP image. Each spatial offset di can be written di=(d u ,i,dv,i) with du,i the spatial shift in the u direction, dv,i the spatial shift in the v direction.

[0161] According to some embodiments, for at least one secondary raw image IMBSi, the spatial offset between said secondary raw image IMBSi and the primary raw image IMBP can be known in advance. This is the case when said spatial offset is introduced in a controlled manner during the acquisition of said secondary image: - for example, through an image stabilization mechanism fitted to the imaging device used; and / or - through the use of several camera modules whose relative positions are known; and / or - by using several known imaging positions; and / or - by using several known viewing angles; and / or - by using several known image capture directions; as described above.

[0162] According to other embodiments, even if the spatial offset between the secondary raw image IMBSi and the primary raw image IMBP is known in advance, it may be desirable to check said spatial offset.

[0163] According to yet other embodiments, for at least one secondary raw image IMBSi, the spatial offset between said secondary raw image IMBSi and the primary raw image IMBP may not be known in advance.

[0164] In all embodiments, the process 200 may include an optional step 202, called the registration step, to determine the spatial offset between the primary raw IMBP image and at least one secondary raw IMBSi image. The registration step 202 may be performed for at least one, several, or all of the secondary raw IMBSI-IMBSK images, simultaneously or sequentially.

[0165] The realignment can be carried out using any suitable technique.

[0166] For example, for at least one secondary image, registration can be performed by calculating the correlation between said raw secondary image and the primary raw image. Correlation can be calculated on the entire image or by image area. Correlation can be calculated for one or more color components individually, or by taking into account all color components.

[0167] Thus, optional step 202 can provide, for at least one, and in particular each, secondary raw image IMBSi, a spatial offset relative to the primary raw image IMBP, and in particular a value or vector representing the spatial offset in the (U,V) plane of the image sensor, between the primary image IMBP and said secondary raw image IMBSi.

[0168] In step 204, the target matrix is ​​initialized. This matrix has a larger size than each of the matrices representing a raw image. Following an example implementation, in the case where (Kl + l)x(K2+l)-l secondary images are used as described above, the target matrix can potentially have (Kl + l)xN rows and (K2+l)xM columns. In the case where K1 = K2 = K, then the target matrix has a size of (K+l) 2 xNxM.

[0169] The target matrix can be initialized with a predetermined value, for example 0, or any other value.

[0170] The dimension of the target matrix can be fixed.

[0171] Alternatively, but preferably, in step 204, the target matrix can be sized by taking into account the number of secondary raw images with a spatial offset from the primary IMBP raw image, the spatial offsets of the secondary images being either known in advance or determined in the optional step 202.

[0172] In step 206, the values ​​of the primary matrix representing the primary raw IMBP image are copied into the target matrix at locations spaced apart from each other according to the number of spatially offset secondary raw images gridding the space between four adjacent photosite patterns.

[0173] More specifically, the value patterns are copied to locations spaced a value of K1 apart in the rows of the raw matrix and a value of K2 apart in the columns. For example: - the pattern of values ​​Bi,i of the main matrix is ​​copied to the location (1,1) of the target matrix; - the BI,2 value pattern of the main matrix is ​​copied to the location (1,2+K1) of the target matrix; - the pattern of values ​​B2 from the main matrix is ​​copied to the location (2+K2, 1) of the target matrix; - the pattern of values ​​62,2 from the main matrix is ​​copied to the location (2+K2,2 + Kl) of the target matrix; - etc. More generally, the pattern of values ​​Bi,j of the main matrix is ​​copied to the location (1 +( i-1) *(1 + K2), 1 + (j-1) *(1 + K1)) of the target matrix;

[0174] Process 200 then includes a step 208 of enriching the target matrix.

[0175] In step 208, the target matrix is ​​completed with patterns of values ​​from at least one, and in particular from each, of the IMBSI-IMBSK secondary images.

[0176] A pattern from a secondary image originating from an IMBSi secondary image is copied into the target matrix at the location determined by the offset along the u-axis and the offset along the v-axis. For example: - a Bi,j pattern originating from a secondary image IMBSi shifted by a distance Jxd u in the direction u, is copied into the target matrix at the location (1+ (il)*(l + K2), l+J+(jl)*(H-Kl)); - a Bi,j pattern originating from a secondary image IMBSi shifted by a distance lxd v in the direction v, is copied into the target matrix at the location (l+H-(il)*(H-K2), 1 +(j- 1 )*( 1 + Kl )); - a Bi,j pattern originating from a secondary image IMBSi shifted by a distance Jxd u in the direction u and at a distance Ixd v in the direction v, is copied into the target matrix at the location (l+I+(i- 1)*(1 + K2), l+J + (jl)*(l + Kl)).

[0177] Thus, the target matrix is ​​completed with the value patterns from the secondary images. Potentially, it is possible to have a target matrix comprising (K1 + l)xN rows and (K2+l)xM columns of value patterns corresponding to all the value patterns of the raw image main IMBP and secondary raw images IMBSI-IMBSK with K=(Kl+l)x(K2+l)-l.

[0178] This target matrix represents the IMBC target image. It contains many more patterns of values ​​compared to the main matrix and each secondary matrix I: therefore, the raw IMBC image is more resolved than the raw main IMBP image and each raw secondary image IMBSi.

[0179] Of course, depending on the number of secondary images, or the offset between the secondary images and the main raw image, it is possible that some positions in the target matrix will remain "empty," that is, not populated by value patterns from the secondary images. In this case, during an optional step 210, at least one of these positions can be filled with a value pattern obtained by calculation, for example, by interpolating adjacent or surrounding value patterns within the target matrix.

[0180] The 200 process thus provides a target matrix representing a target raw IMBC image that is more resolved, and potentially (K+l) times more resolved, than the primary raw image and each of the secondary raw images.

[0181] In the example described, method 200 uses IMBSI-IMBSK secondary raw images, some of which are shifted along the u direction, others along the v direction, and still others along both u and v directions, to grid the space between four adjacent photosite patterns. Of course, the invention is not limited to this example. It is possible to use one or more secondary raw images shifted in the u direction only, or one or more secondary raw images shifted in the v direction only, or one or more secondary raw images shifted in both the u and v directions.

[0182] The number of secondary raw images is not limiting and the invention relates to the use of a number of secondary raw images greater than or equal to 1.

[0183] The example of copying value patterns from the main raw image into the target matrix is ​​not limiting. In the given example, the copying is performed by matching the first value pattern Bi,i of the main matrix to the first pattern Bi,i of the target matrix. Of course, it is possible to match the first value pattern Bi,i of the main matrix to another pattern Bi,j of the target matrix and adjust the position of the other patterns accordingly.

[0184] In the example described, copying a value pattern from a secondary image involves all values ​​of that value pattern. Of course, this example is by no means exhaustive. For a pattern from a secondary image, copying that pattern into the target matrix can involve the value of a single monochrome component or only certain monochrome components. In general terms, for a pattern from a secondary image, copying that pattern can involve at least one value of that value pattern.

[0185] FIGURE 3 is a schematic representation of a non-limiting example embodiment of an image demosaicing process according to the present invention.

[0186] The 300 method of FIGURE 3 allows obtaining a dematrixed image, noted IMD, without loss of generality, with higher resolution from a primary raw image IMBP and at least one secondary raw image IMBSi-IMBSk with lower resolution.

[0187] The process 300 includes a step 302 for obtaining a higher-resolution target raw IMBC image from a primary raw IMBP image and at least one lower-resolution secondary raw IMBSi-IMBSk image. In particular, step 302 can implement process 200 of FIGURE 2, so that process 300 includes all the steps of process 200 of FIGURE 1.

[0188] Next, process 300 includes a step 304 of demosaicing, of the target raw image, according to any known technique.

[0189] The demosaicing of the target IMBC raw image can be performed using any known technique to obtain a demosaiced IMD image composed of pixels, and in particular color pixels, each pixel being represented by a set of values. If the image is an RGB image, each pixel can be represented by a set of at least three values ​​(R, G, B): one for the color red, one for the color green, and one for the color blue.

[0190] When the raw image contains [(Kl + l)xN]x[(K2+l)xM] positions, it is possible that a pattern has been copied to all of these positions. It is also possible that there are positions where a pattern has not been copied. In this case, it is possible to interpolate the missing values ​​at these positions from the values ​​of adjacent, or neighboring, patterns, in order to have a pattern at each of the [(Kl + l)xN]x[(K2+l)xM] positions in step 304. Several interpolation methods are possible, such as writing an average of neighboring pixels to the missing values, or writing values ​​obtained by a least-squares function that approximates known neighboring values ​​to the missing values.

[0191] In step 304, the IMBC target raw image can be demosaiced to obtain one image pixel for each pattern in the IMBC target raw image. In this case, when, for example, the IMBC target raw image comprises [(Kl + l)xN]x[(K2+l)xM] patterns, the resulting IMD demosaiced image comprises [(Kl + l)xN]x[(K2+l)xM] pixels. Therefore, for a pixel corresponding to a pattern of values ​​in the target matrix representing the IMBC target raw image: - the value of the color red corresponds to the value of the red monochrome component of said pattern, or to the average of the red monochrome components of said pattern if said pattern includes several red components; - the value of the blue color corresponds to the value of the blue monochrome component of said pattern, or to the average of the blue monochrome components of said pattern if said pattern includes several blue components; - the value of the green color corresponds to the value of the green monochrome component of said pattern, or to the average of the green monochrome components of said pattern if said pattern includes several green components.

[0192] Alternatively, in step 304, the demosaicing of the target IMBC raw image can be performed to obtain one image pixel for each monochrome component of each pattern in the target raw IMC image. In this case, for each monochrome component of a pattern in the target raw image, we obtain an image pixel for which: - the value of said monochrome component is assigned to the corresponding color; - the value of each of the other colours is obtained by interpolation, and in particular by averaging, the values ​​of said colour of neighbouring or surrounding patterns, within the target matrix. For example, for the red monochrome component of a pattern in the target raw IMBC image, we obtain a pixel for which: - the value of the color red corresponds to the value of said red monochrome component, or to the average of the red monochrome components of said pattern if said pattern includes several red components; - the value of the blue color is deduced, by interpolation, from the values ​​of the blue component of the neighboring, or surrounding, patterns within the target matrix; - the value of the green color is deduced, by interpolation, from the values ​​of the green component of the neighboring, or surrounding, patterns within the target matrix. The same logic applies respectively to the green monochrome component, and to the blue monochrome component.

[0193] Of course, other embodiments are possible for the demosaicing step and the examples given above with reference to FIGURE 3 are not limiting.

[0194] FIGURE 4 is a schematic representation of a non-limiting example embodiment of a method according to the present invention for processing an image according to the invention.

[0195] The 400 process allows obtaining a processed image, IMT, of a scene.

[0196] The process 400 includes a step 402 to obtain a higher-resolution target raw image, IMBC, or a higher-resolution demosaiced image, IMD, from a lower-resolution primary raw image, IMBP, and at least one lower-resolution secondary raw image, IMBSi-IMBSk. Specifically, step 402 can implement process 200 of Figure 2, or process 300 of Figure 3, respectively.

[0197] The process 400 further includes a step 404 for processing the target raw IMBC image, or the demosaiced IMD image. This processing step 404 can perform any type of digital processing on the target raw IMBC image, or the demosaiced IMD image.

[0198] For example, step 404 can perform the correction of an optical aberration, such as a chromatic aberration, in particular displacement aberration, a geomatic aberration, etc.

[0199] For example, step 404 can achieve image sharpness improvement, color enhancement, brightness modification, etc.

[0200] Step 404 provides the processed IMT image.

[0201] FIGURE 5 is a schematic representation of a non-limiting example of an embodiment of a method according to the present invention for imaging a scene.

[0202] The method 500 of FIGURE 5 includes a step 502 for acquiring a raw image, referred to as the master raw image, and denoted IMBP, of a scene with an imaging device comprising at least one camera module. The master raw image is acquired with a camera module comprising an optical lens associated with an image sensor, such as, for example, the image sensor 100 of FIGURE 1. Optionally, the camera module (or imaging device) may include, be equipped with, or be associated with, an image stabilizer. The function of such a stabilizer is to eliminate the effects of slight movements of the camera module, or the imaging device, used during the acquisition of the master raw image IMBP.

[0203] The 500 process of FIGURE 5 includes a step 504 which acquires one or more secondary raw IMBSI-IMBSK images of the scene.

[0204] At least one IMBSi secondary raw image can be acquired with a camera module comprising an optical lens coupled to an image sensor, such as, for example, the 100 image sensor in Figure 1. Optionally, the camera module (or imaging device) may include, be equipped with, or be coupled with an image stabilizer. Such a stabilizer is designed to eliminate the effects of slight movements of the camera module, or imaging device, used during the acquisition of the IMBSi secondary raw image.

[0205] At least one, in particular each, secondary raw IMBSi image can be acquired with the same camera module as that used for the acquisition of the primary raw IMBP image.

[0206] At least one, in particular each, secondary raw IMBSi image may be acquired with a camera module other than the one used for the acquisition of the primary raw IMBP image. This other camera module may be the same as, or different from, the one used for the acquisition of the primary raw IMBP image.

[0207] Preferably, at least one, in particular each, secondary raw IMBP image can be acquired with a spatial offset from the primary raw IMBP image.

[0208] Following an example of an embodiment, at least one, in particular each, secondary raw IMBSi image can be acquired from a different acquisition position, and / or a different viewing angle, etc., so as to obtain a spatial offset relative to the main raw image.

[0209] Alternatively, or in addition, at least one, specifically each, secondary raw IMBSi image can be acquired by introducing a spatial offset through the image stabilizer fitted to the imaging device. For example, when each of the secondary raw IMBSI-IMBSK images is acquired with a spatial offset, respectively di-dK, then it is possible to introduce the corresponding spatial offset into the stabilizer control. image. In this case, the stabilizer is controlled to eliminate all movement, except for the desired spatial shift.

[0210] When multiple secondary images are acquired with different spatial offsets di-dm, preferably, said raw secondary IMBSI-IMBSK images can be acquired in an order other than ascending or descending spatial offsets to avoid synchronizing said spatial offsets di-dx with the movement of an object in the scene. For example, the raw secondary IMBSI-IMBSK images can be acquired in a random order. As another example, the raw secondary IMBSI-IMBSK images can be acquired in a predetermined order other than ascending or descending order, and more generally, other than a rectilinear order or trajectory. For example, assuming the use of ten secondary IMBSI-IMBSio images, each with a spatial offset di-dio, respectively, with di <d2, ..., dg <dio, alors il est préférable de prendre les images brutes secondaires IMBSi-IMBSio dans un ordre différent de l'ordre allant de 1 à 10 ou de 10 à 1. Par exemple, il est possible de prendre lesdites images dans un ordre aléatoire, ou dans l'ordre suivant : IMBSi, IMBSio, IMBS2, IMBS9, IMBS3, IMBSs, IMBS4, IMBSe et IMBSs. Bien entendu, cet ordre est donné à titre d'exemple non limitatif.

[0211] After the acquisition of the raw images, process 500 includes a processing step 506. Step 506 can implement any one of the processes 200, 300 or 400 of FIGURES 2 to 4 to obtain, respectively, a target raw IMBC image, a dematrix IMD image or a processed IMT image.

[0212] FIGURE 6 is a schematic representation of a non-limiting example embodiment of a device according to the present invention.

[0213] The 600 device in FIGURE 6 can be used to implement: - a method according to the invention for obtaining a higher-resolution target raw image, and in particular method 200 of FIGURE 2; or - a method according to the invention for demosaicing a target raw image to obtain a demosaiced image, and in particular method 300 of FIGURE 3; or - an image processing method according to the invention, and in particular method 400 of FIGURE 4; or - a method according to the invention for imaging a scene, and in particular method 500 of FIGURE 5.

[0214] The device 600 in FIGURE 6 may include an optional module 602 for registering at least one, and in particular each, secondary raw image with respect to the primary raw image. This optional module 602 may, in particular, be configured to perform the optional step 202 of the process 200.

[0215] Device 600 includes a module 604 for initializing a target matrix representing the target raw IMBC image. This module 604 can, in particular, be configured to perform step 204 of process 200.

[0216] Device 600 includes a module 606 for copying the value patterns from the master matrix representing the primary raw image IMBP into the target matrix representing the target raw image IMBC. This module 606 can, in particular, be configured to perform step 206 of process 200.

[0217] Device 600 includes a module 608 for enriching the target matrix. This module 608 can, in particular, be configured to perform step 208 of process 200.

[0218] Device 600 may include an optional module 610 for calculating, by interpolation, at least one pattern of values ​​in the target matrix. This optional module 610 can, in particular, be configured to perform the optional step 210 of process 200.

[0219] When the device 600 is used to implement a demosaicing process according to the invention, and in particular the demosaicing process 300 of FIGURE 3, said device 600 further includes a module 612 for demosaicing the target raw IMBC image. This module 612 can, in particular, be configured to perform the demosaicing step 304 of process 300.

[0220] When device 600 is used to implement an image processing method according to the invention, and in particular method 400 of FIGURE 4, said device 600 further comprises a module 614 for applying processing, in particular digital processing, to the target raw IMBC image. This module 614 can, in particular, be configured to perform the digital processing step 404 of method 400.

[0221] When device 600 is used to implement a method according to the invention for imaging a scene, and in particular method 500 of FIGURE 4, said device 600 further comprises a module 616 for controlling one or more camera modules for acquiring a primary raw image IMBP and at least one secondary raw image IM BSI-IM BSK. This module 616 can, in particular, be configured to perform, or trigger the execution of, steps 502 and 504 of method 500.

[0222] Optionally, the 600 device can also include an image stabilizer 618. This image stabilizer can be controlled by the 616 module to introduce a spatial offset during the acquisition of at least one secondary raw IMBSi image. The 618 stabilizer is optional because it may already exist in the imaging device used to image the scene.

[0223] Optionally, the 600 device may also include at least one 620 camera module for acquiring the primary raw IMBP image and / or at least one secondary raw IM BSI-IM BSK image. The at least one 620 camera module can be controlled by the 616 module for acquiring the raw image(s). The at least one 620 camera module is optional because it may already be present in the imaging device used to image the scene.

[0224] At least one of the 602-616 modules can be a module independent of the other 602-616 modules.

[0225] At least two of the 602-616 modules can be integrated within a single module. In particular, the 602-616 modules can be integrated within a 622 computing unit.

[0226] At least one of the 602-616 modules can be a hardware module, such as a processor, an electronic chip, etc.

[0227] At least one of the 602-616 modules can be a software module, such as a computer program.

[0228] At least one of the 602-616 modules can be a combination of at least one software module and at least one hardware module.

[0229] In particular, at least one of the 602-616 modules can be integrated into an electronic chip, or into an application installed in a user device such as, for example, a photo application or a video application.

[0230] In particular, the 622 computing unit can be, or can be integrated, into an electronic chip, or into an application installed in a user device.

[0231] FIGURE 7a is a schematic representation of a non-limiting example embodiment of a device according to the present invention.

[0232] The apparatus 700 of FIGURE 7a includes means configured to implement the invention, and in particular any one of the methods 200, 300, 400 or 500.

[0233] The apparatus 700 of FIGURE 7a may include a device according to the invention, and in particular the device 600 of FIGURE 6.

[0234] In the example shown in FIGURE 7a, device 700 is a smartphone, or a tablet, comprising device 600 from FIGURE 6.

[0235] Optionally, the device 700 can also include a display screen 702, optionally equipped with a sensing surface 704, for example capacitive.

[0236] Of course, the 700 device may include other components than those indicated above.

[0237] FIGURE 7b is a schematic representation of another non-limiting embodiment of a device according to the present invention.

[0238] The apparatus 710 of FIGURE 7b includes means configured to implement the invention, and in particular any one of the methods 200, 300, 400 or 500.

[0239] The apparatus 710 of FIGURE 7b may include a device according to the invention, and in particular the device 600 of FIGURE 6.

[0240] In the example shown in FIGURE 7b, device 710 is: - a virtual reality (VR) headset or glasses, or - an augmented reality headset or glasses; including device 600 of FIGURE 6.

[0241] Optionally, the 710 device may also include a 712 display screen, for example in / on / under a visor of said 710 helmet.

[0242] Of course, the 710 helmet may include other components than those listed above.

[0243] FIGURE 7c is a schematic representation of a non-limiting example embodiment of a device according to the present invention.

[0244] The apparatus 720 of FIGURE 7c includes means configured to implement the invention, and in particular any one of the methods 200, 300, 400 or 500.

[0245] The apparatus 720 of FIGURE 7c may include a device according to the invention, and in particular the device 600 of FIGURE 6.

[0246] In the example shown in FIGURE 7c, device 720 is a medical imaging device, such as an endoscope, an ultrasound machine, etc.

[0247] Optionally, the device 720 can also include a display screen 722, optionally equipped with a sensing surface 724, for example capacitive.

[0248] Of course, the 720 medical imaging device may include other organs than those indicated above.

[0249] FIGURE 8 is a schematic representation of a non-limiting example embodiment of a vehicle according to the present invention.

[0250] The vehicle 800 of FIGURE 8 includes means configured to implement the invention, and in particular any one of the methods 200, 300, 400 or 500.

[0251] The vehicle 800 of FIGURE 8 may include a device according to the invention, and in particular the device 600 of FIGURE 6.

[0252] In the example shown in FIGURE 8, vehicle 800 is a land vehicle, in particular a car, comprising device 600 of FIGURE 6.

[0253] Optionally, the vehicle 800 may also include a display screen 802, optionally equipped with a sensing surface 804, for example capacitive, arranged in the passenger compartment of the vehicle 800.

[0254] Of course, the 800 vehicle may include other components than those indicated above.

[0255] Of course, the invention is not limited to the examples that have just been described.

Claims

DEMANDS 1. Method (200) for obtaining a raw image (IMBC), called the target raw image, of a scene, with higher resolution from: - of a raw image (IMBP), called the primary raw image, and - of at least one other raw image (IMBSi-IMBSi), called secondary raw image; of said scene, less resolved than said target raw image (IMBC); each of said primary (IMBP) and secondary (IMBSi-IMBSi) raw images being acquired by at least one camera module comprising an image sensor (100) having a pattern matrix (Bi,j) of monochrome photosites; each of said primary (IMBP) and secondary (IMBSi-IMBSi) raw images being represented by a matrix of value patterns, each value pattern being provided by a photosite pattern of said image sensor (100) and comprising a value provided by each monochrome photosite of said photosite pattern; said process (200) comprising the following steps: - initialization (204) of a matrix, called the target matrix, of value patterns to represent said target raw image (IMBC); - copying into said target matrix, the value patterns of the main raw image (IMBP); -enrichment of said target matrix by adding, in said target matrix, at least a part of at least one pattern of values ​​from a secondary image (IMBSi-IMBSi).

2. A method (200) according to the preceding claim, characterized in that at least one pattern of values ​​from a secondary raw image (IMBSi-IMBSi) is added to the target matrix at a position determined as a function of: - the position of said motif in said secondary raw image (IMBSi-IMBSi), and - a spatial offset between the main raw image (IMBP) and said secondary raw image (IMBSi-IMBSi), and more particularly the offset of the secondary image (IMBSi-IMBSi) relative to the main image (IMBP).

3. Method (200) according to the preceding claim, characterized in that, for at least one secondary raw image (IMBSi-IMBSi), the spatial offset between said secondary image (IMBSi-IMBSi) and the primary raw image (IMBP) is less than the spatial repetition step of the patterns in the image sensor (100), so that at least one pattern of the secondary raw image (IMBSi-IMBSi) is inserted in a position located between two patterns of the primary raw image (IMBP).

4. Method (200) according to any one of the preceding claims, characterized in that it further comprises, for at least one secondary raw image (IMBSi-IMBSi), a step (202) of registration of said secondary raw image (IMBSi-IMBSi) with respect to the primary raw image (IMBP) before the enrichment step (208).

5. Method (200) according to any one of the preceding claims, characterized in that the enrichment step (208) is carried out individually for at least one, and in particular each, monochrome component of the value pattern, or of the photosite pattern.

6. Method (200) according to any one of the preceding claims, characterized in that the enrichment step (208) is carried out individually for at least one, and in particular each, secondary raw image (IMBSi-IMBSi).

7. Method (200) according to any one of the preceding claims, characterized in that the, or at least one, secondary raw image (IMBSi-IMBSi) is captured by another camera module.

8. Method (200) according to any one of the preceding claims, characterized in that the, or at least one, secondary raw image (IMBSi-IMBSi) is captured by the same camera module as that used to capture the primary raw image (IMBP).

9. Method (200) according to any one of the preceding claims, characterized in that, for at least a part of the scene, at least one, in particular each, secondary raw image (IMBSi-IMBSi) is spatially offset with respect to the primary raw image (IMBP).

10. Method (200) according to the preceding claim, characterized in that, for at least one secondary image (IMBSi-IMBSi), the spatial shift is obtained by controlling an image stabilizer equipping the imaging device to introduce said spatial shift.

11. A method (300) for image demosaicing comprising the following steps: - obtaining (302) a target raw image (IMBC) by the process (200) according to any one of the preceding claims; and - demosaicing (304) said target raw image (IMBC) to obtain a demosaiced image (IMD).

12. Method (300) according to the preceding claim, characterized in that the demosaicing step (304) provides an image pixel for each pattern of values ​​of the target raw image (IMBC).

13. Method (300) according to claim 11, characterized in that the demosaicing step (304) provides an image pixel for each monochrome component of each pattern of values ​​of the target raw image (IMBC).

14. Image processing method (400) comprising the following steps: - obtaining (402) a target raw image (IMBC) by method (200) according to any one of claims 1 to 10, or a demosaiced image (IMD) by method (400) according to any one of claims 11 to 13; and -digital processing (404) of said target raw image (IMBC), or of said demosaiced image (IMD).

15. Method (500) for imaging a scene comprising the following steps: - acquisition (502) of a raw image (IMBP), called the main raw image, by a camera module, -processing (504) of said primary raw image (IMBP) by the process (200;300;400) according to any one of claims 1 to 14.

16. Computer program comprising executable instructions which, when executed by at least one computing device, implement all the steps of the process (200;300;400) according to any one of the preceding claims.

17. Device (600) comprising means configured to carry out all the steps of the process (200;300;400) according to any one of claims 1 to 15.

18. Device (700;710;720) comprising: - at least one camera module (620) to acquire at least one raw image of a scene; and - a calculation unit (622); configured to implement all steps of the process (200;300;400;500) according to any one of claims 1 to 15.

19. Vehicle (800) comprising: - at least one camera module (620) to acquire at least one raw image of a scene; and - a calculation unit (622); configured to implement all steps of the process (200;300;400;500) according to any one of claims 1 to 15.

Images