Image construction process, associated electronic device and computer program

Abstract

An image construction process comprises the following steps: - for each pixel of the image, determination (E4, E6, E8) of modified coordinates (X3, Y3) based on the (X, Y) coordinates of the pixel in the image; - construction (E10) of the image by assigning to each pixel of the image the value associated with a photosensitive element defined by the modified coordinates determined for that pixel of the image. The modified coordinates are determined for each pixel based on the coordinates of that pixel by successively applying a plurality of transformations comprising: - a first transformation (E4) increasing the distances from a central point; - a second transformation (E6) resulting in a reduction of the distances by applying a reduction factor; - a third transformation (E8) having the effect of a distortion caused by an optical element in the plane of the photosensitive elements.The increase in distances during the first transformation depends on the reduction factor. Figure for the abbreviation: Figure 2.

Description

Title of the invention: Method for constructing an image, associated electronic device and computer program Technical field of the invention

[0001] The present invention relates to the technical field of image acquisition using a sensor.

[0002] It relates in particular to a method for constructing an image, as well as an associated electronic device and computer program. State of the art

[0003] It is known to use electronic devices (for example, video camera type) using a sensor comprising photosensitive elements to acquire an image.

[0004] An optical element, such as a lens, is generally placed in the path of the light flux incident on the sensor so as to image the environment to be observed by means of the sensor in the plane of the photosensitive elements.

[0005] The optical element may, however, create a distortion of the light flux incident on the sensor, in particular when the optical element has a reduced focal length so as to allow observation with a large field angle.

[0006] Techniques for compensating this distortion have been proposed, but the resulting image lacks realism due to a distortion of the objects present in the observed scene. Presentation of the invention

[0007] In this context, the present invention proposes a method for constructing an image based on a set of values ​​respectively associated with photosensitive elements of a sensor receiving a light flux through an optical element causing a distortion of the light flux, comprising the following steps:

[0008] - for each pixel of the image, determination of modified coordinates on the basis coordinates of this pixel in the image;

[0009] - constructing the image by assigning, to each pixel of the image, the associated value to the photosensitive element defined by the modified coordinates determined for that pixel of the image,

[0010] characterized in that the modified coordinates are determined for each pixel of the image on the basis of the coordinates of that pixel in the image by successive application of a plurality of transformations comprising:

[0011] - a first transformation transforming an input point into an output point and such that the entry point, the exit point and a point associated with the optical axis of the optical element are aligned and that a first distance between the exit point and said associated point is (strictly) greater than a second distance between the entry point and said associated point;

[0012] - a second transformation resulting in a reduction of distances by application of a reduction factor;

[0013] - a third transformation having the effect of said distortion in the plane of photosensitive elements of the sensor,

[0014] in which the ratio between the first distance and the second distance depends on the reduction factor.

[0015] The distortion caused by the optical element is corrected by the third transformation. However, the first transformation corrects the constructed image to avoid distortion of the objects it contains, modulating this effect according to the reduction performed by the second transformation. Indeed, the distortion of objects caused by the third transformation occurs mainly at low image magnification, that is, during a small reduction by the second transformation.

[0016] The first distance can be determined by applying to the second distance an increasing function fk depending on the reduction factor.

[0017] The increasing function fk is for example defined on an interval [0 ; Ak [ ; the values ​​fk(d) of the increasing function fk can in this case tend towards infinity when d tends towards Ak.

[0018] The reduction factor used can be the inverse of a desired magnification factor (introduced for example by the user via a user interface).

[0019] The upper bound Ak of ​​said interval can then be increasing as a function of the desired magnification factor. Thus, the effect of the function fk (and therefore of the first transformation) will be greater for low magnification factors.

[0020] The second transformation can be carried out by means of a homothety with a ratio equal to the reduction factor.

[0021] The third transformation can for example transform an input point into an output point so that the input point, the output point and said associated point are aligned and that a third distance between the output point and said associated point is less than (strictly) a fourth distance between the input point and said associated point.

[0022] The method may also include a step of displaying the image on a display device.

[0023] The invention also proposes an electronic device designed to construct an image based on a set of values ​​respectively associated with photosensitive elements of a sensor receiving a light flux through an optical element causing a distortion of the light flux, comprising:

[0024] - a memory storing, for each pixel of the image, the pixel coordinates in the image;

[0025] - a processor configured to determine, for each pixel of the image, modified coordinates based on the coordinates stored in memory for that pixel, and to construct the image by assigning, to each pixel of the image, the value associated with the photosensitive element defined by the modified coordinates determined for that pixel of the image,

[0026] characterized in that the processor is configured to determine the modified coordinates for each pixel of the image based on the coordinates stored in memory for that pixel by successively applying a plurality of transformations comprising:

[0027] - a first transformation transforming an input point into an output point and such that the entry point, the exit point and a point associated with the optical axis of the optical element are aligned and that a first distance between the exit point and said associated point is (strictly) greater than a second distance between the entry point and said associated point;

[0028] - a second transformation resulting in a reduction of distances by application of a reduction factor;

[0029] - a third transformation having the effect of said distortion in the plane of photosensitive elements of the sensor,

[0030] in which the ratio between the first distance and the second distance depends on the reduction factor.

[0031] This electronic device may include said sensor (as well as optionally the optical element).

[0032] The invention finally proposes a computer program comprising instructions executable by a processor and designed to implement a process as defined above when these instructions are executed by the processor.

[0033] The optional characteristics presented above in terms of method can also be applied to this electronic device.

[0034] Of course, the various features, variants, and embodiments of the invention can be combined in various ways, provided they are not incompatible or mutually exclusive. Detailed description of the invention

[0035] In addition, various other features of the invention become apparent from the attached description made with reference to the drawings which illustrate non-limiting embodiments of the invention and where:

[0036] [Fig. 1] represents the main elements of an example of an electronic device according to the invention;

[0037] [Fig.2] is a logic diagram showing an example of a method for constructing an image according to the invention;

[0038] [Fig.3] represents functions used in the process of [Fig.2]; and

[0039] [Fig.4] represents the distortion caused by an optical element of the device electronics of the [Fig.1].

[0040] Fig. 1 represents the main elements of an example of an electronic device according to the invention.

[0041] Such an electronic device 2 (for example a video camera) comprises an optical element 4, a sensor 6, a processor 8 and a memory 10. Such an electronic device is for example mounted in a vehicle (such as a motor vehicle) in order to acquire an image (or a plurality of images so as to form a video sequence) of the interior (for example of the passenger compartment) of the vehicle or of an environment outside the vehicle.

[0042] The sensor 6 comprises an array of photosensitive elements 12.

[0043] The optical element 4 (for example, a lens, here a wide-angle and / or short focal length lens, or "fisheye" lens) images the environment observed by the camera in the plane of the photosensitive elements 12 of the sensor 6. The focal length of the lens 4 is, for example, between 2 mm and 3 mm, and is 2.5 mm in the example described. The angle of view is, for example, 160° in the horizontal plane and 90° in the vertical plane.

[0044] The photosensitive elements 12 of the sensor thus receive a luminous flux through the optical element 4 which causes (due to the reduced focal length) a distortion of this luminous flux, in particular in the plane of the photosensitive elements 12.

[0045] Each photosensitive element 12 produces a value representative of the intensity of the light flux incident on the photosensitive element 12 concerned.

[0046] These values ​​respectively produced by the photosensitive elements 12 of the sensor 6 are processed by the processor 8 to construct (according to a process described below) an image to be displayed on a display device (not shown).

[0047] The processor 8 (for example a microprocessor) thus implements a process for processing the values ​​produced by the photosensitive elements 12 and for constructing the aforementioned image, as described below with reference to [Fig.2], due to the execution by this processor 8 of computer program instructions (stored for example in memory 10).

[0048] Fig. 2 is a flowchart showing an example of a method for constructing an image according to the invention.

[0049] As indicated above, this process is implemented here by the processor 8 due to the execution by this processor 8 of computer program instructions stored in memory 10.

[0050] It is assumed here that the user wants a magnification of a factor k of the image (the desired magnification factor being entered for example by the user by means of a user interface not shown).

[0051] This process begins with a step E2 during which the processor 8 defines a template including, for each pixel of the image to be constructed (or image to be displayed), X, Y coordinates of the pixel in the image and stores these X, Y coordinates in memory 10.

[0052] The X, Y coordinates of a pixel in the image are, for example, two integers that define respectively the horizontal and vertical position of the pixel in the image (this image being raster). Other coordinate systems may, however, be used as an alternative.

[0053] The X, Y coordinates thus stored (initial coordinates) are then processed (by processor 8) by three successive transformations described below in steps E4 to E8 in order to obtain, for each pixel of the image to be constructed, modified coordinates X3, Y3. In other words, for each pixel of the image to be constructed, the X, Y coordinates associated with this pixel in step E2 undergo successively the three transformations described in steps E4 to E8 in order to obtain modified coordinates X3, Y3 associated with this pixel.

[0054] In step E4, the processor 8 applies, for each pixel of the image to be constructed, a first transformation of the X, Y coordinates relative to that pixel. This first transformation is a transformation of the plane (i.e., a transformation defined in the plane) which transforms input coordinates X, Y (defining an entry point) into output coordinates XI, Y1 (defining an exit point).

[0055] This first transformation transforms the input point (with coordinates X, Y) into an output point (with coordinates XI, Yl) defined as follows:

[0056] - the entry point, the exit point and a point associated with the optical axis of the element optics 4 (referred to as the central point hereafter) are aligned;

[0057] - the distance D between the exit point and the central point is determined by application of a function fk to the distance d between the entry point and the central point.

[0058] In the example described here, it is assumed that the image to be constructed is centered on the sensor 6 and that the sensor 6 and the optical element 4 are aligned, so that the point associated with the optical axis of the optical element 4 (central point) is located at the center of the image to be constructed.

[0059] The function fk used during the current implementation of the first transformation depends on the desired magnification factor k.

[0060] We have represented on [Fig.3] three functions fkmin, fk, fkmax, where kmin is the minimum usable magnification factor and kmax is the maximum usable magnification factor; we therefore have kmin < k < kmax.

[0061] As can be seen in [Fig.3], all the fk functions used have the following properties:

[0062] - fk is increasing on its domain of definition;

[0063] - for all x in its domain of definition, fk(x) > x (therefore, the curve representative of any function fk is located above the line with equation (D=d) as shown in [Fig.3]).

[0064] Thus, the distance D between the exit point and the central point is greater (strictly) than the distance d between the entry point and the central point.

[0065] Furthermore, at the point with abscissa d=0, the graph of any function fk is tangent to the line with equation (D=d), as also shown in [Fig.3]. (In other words, the derivative of any function fk at the point with abscissa d=0 is 1.)

[0066] Thus, the first transformation does little to modify the entry points associated with a small distance d, that is to say located close to the central point.

[0067] Moreover, as can be seen in [Fig.3], the functions fk are distinct from one another so that, for each possible value of the distance d between the entry point and the central point, the ratio between the distance D = fk(d) and the distance d depends on the factor k.

[0068] Specifically, the ratio between the distance D (distance between the exit point and the central point) and the distance d (distance between the entry point and the central point) decreases when the desired magnification factor k increases.

[0069] In other words, the first transformation causes a significant dilation around the central point when the desired magnification factor is low, but a less significant dilation around the central point when the desired magnification factor is high.

[0070] Moreover, as also shown in [Fig. 3], each function fk is defined on an interval [0; Ak[ and the value fk(x) tends to infinity as x tends to Ak. In other words, for each function fk, the (vertical) line with equation (x = Ak) is an asymptote to the graph of that function fk.

[0071] For example, the following values ​​are used:

[0072] - Ak min = r / 2, where r is the radius of the largest circle centered on the central point and inscribed in the image to be constructed (in the coordinate system used in step E2);

[0073] - Ak = Ak min.k / krnin for other values ​​of k.

[0074] The upper bound Ak of ​​the interval of definition [0 ; Ak [ of the function fk is therefore increasing here as a function of the desired magnification factor k.

[0075] For example, we can use the family of functions fk defined by:

[0076] fk(d) = (2.Ak / jt). tan(0.5.ir.d / Ak)

[0077] with here as already indicated Ak = (rk) / (2.kmin) and tan the trigonometric function "tangent".

[0078] In practice, the application of the function fk is, for example, carried out by reading from a look-up table. For this purpose, memory 10 stores, for example, for each of a plurality of ranges of values ​​of k (these ranges of values ​​covering the allowed values ​​of the magnification coefficient k), a look-up table associating with distance values ​​d respectively the distance values ​​fk(d).

[0079] In this case, the application of the function fk includes the selection of a lookup table associated with the coefficient k and the reading of the distance D associated with the distance d in the selected lookup table.

[0080] The application of the first transformation makes it possible to pre-compensate for the adverse effects of the distortion correction carried out by means of the third transformation (step E8).

[0081] Indeed, the first transformation tends to move away from the central point the points where the values ​​of the photosensitive elements of the sensor will be read and therefore, conversely, to bring back towards the central point the positioning of these photosensitive elements in the image.

[0082] The use of the value Ak min = r / 2 allows in this respect that with the minimum magnification coefficient, even distant points (in particular due to the distortion correction in step E8) are brought to the edge of the constructed image.

[0083] In step E6, the processor 8 applies a second transformation of the coordinates (here XI, Y1) relative to each pixel of the image to be constructed. This second transformation transforms the input coordinates (here the coordinates XI, Y1 produced at the output of step E4) into the output coordinates X2, Y2.

[0084] The second transformation results in a reduction of distances by application of the reduction factor 1 / k equal to the inverse of the magnification factor k.

[0085] Indeed, as explained below, it is the modified coordinates (produced at the output of steps E4 to E8) which indicate which photosensitive element must be read to construct (i.e. form) the image and a reduction of the distances in step E6 therefore makes it possible to read closer photosensitive elements on the sensor, i.e. the desired magnification.

[0086] The second transformation is for example carried out in practice by applying a homothety with a ratio of 1 / k centered on the central point defined above.

[0087] In step E8, the processor 8 applies, for each pixel of the image to be constructed, a third transformation of the coordinates (here X2, Y2) relative to that pixel. This third transformation transforms the input coordinates (here the X2, Y2 coordinates produced at the output of step E6) into the output coordinates X3, Y3.

[0088] The third transformation has the same effect as the distortion caused by the optical element 4 in the plane of the photosensitive elements 12 of the sensor 6.

[0089] As seen in [Fig.4], this distortion has the effect of sending onto the sensor 6, at a distance d' from the optical axis of the optical element 4, a light ray which should have reached the sensor 6 at a distance D' from the optical axis of the optical element 4, with D' > d'.

[0090] The third transformation therefore causes a reduction of the distances around the central point (associated with the optical axis of the optical element 4 as already indicated), this reduction of distances corresponding to the effect of distortion in the plane of the photosensitive elements 12.

[0091] The third transformation is a transformation of the plane (i.e., defined in the plane) and transforms the input point (with coordinates X2, Y2) into an output point (with coordinates X3, Y3) defined as follows:

[0092] - the entry point, the exit point and the central point are aligned;

[0093] - the distance between the exit point and the central point is determined by application of a function g to the distance D' between the entry point and the central point, the function g representing the effect of distortion in the plane of the photosensitive elements 12 and being such that d' = g(D') < D'.

[0094] The function g is independent of the desired magnification factor k and the third transformation performed is therefore the same regardless of the desired magnification factor k.

[0095] The application of the function g is for example carried out by reading from a lookup table (stored here in memory 10). This lookup table stores a plurality of values ​​g(D') respectively associated with a corresponding plurality of values ​​D' and determined for example by means of prior measurements carried out on the optical element 4 concerned (or an optical element of the same type).

[0096] The processor 8 can then construct the image (step E10) by assigning, to each pixel (with coordinates X, Y) of the image, the value produced by the photosensitive element defined by the modified coordinates X3, Y3 determined for that pixel of the image.

[0097] Thanks to the third transformation performed in step E8, the processor 8 reads the light beam associated with a pixel at the point where this light beam is incident (on sensor 6) and the distortion is thus compensated. The first transformation carried out in step E4 however makes it possible to partially compensate for this correction, according to the desired magnification factor (magnification carried out by means of the second transformation of step E6), in order to avoid too much distortion of objects in the image, visible in particular in the absence of magnification or in the case of low magnification.

[0098] The method can then include a step E12 of displaying the image constructed on a display device (not shown).

[0099] The embodiment described above is only one possible example of implementing the invention. It is also possible, in particular, to carry out the transformations in a different order than that presented above, for example, to carry them out in the following order: second transformation, first transformation, third transformation.

[0100] Furthermore, in the case where the electronic device is a video camera, the processing described above for constructing an image is applied to each of the images of the video sequence taken by the video camera.

Claims

Demands

1. Method of constructing an image on the basis of a set of values ​​respectively associated with photosensitive elements (12) of a sensor (6) receiving a light flux through an optical element (4) causing a distortion of the light flux, comprising the following steps: - for each pixel of the image, determination (E4, E6, E8) of modified coordinates (X3, Y3) on the basis of coordinates (X, Y) of the pixel in the image;- construction (E10) of the image by assigning, to each pixel of the image, the value associated with the photosensitive element (12) defined by the modified coordinates (X3, Y3) determined for that pixel of the image, characterized in that the modified coordinates (X3, Y3) are determined for each pixel of the image on the basis of the coordinates (X, Y) of that pixel in the image by successive application of a plurality of transformations comprising: - a first transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element (4) are aligned and that a first distance (D) between the output point and said associated point is greater than a second distance (d) between the input point and said associated point; - a second transformation resulting in a reduction of the distances by application of a reduction factor;- a third transformation having the effect of said distortion in the plane of the photosensitive elements (12) of the sensor (6), in which the ratio between the first distance (D) and the second distance (d) depends on the reduction factor.;

2. Method according to claim 1, wherein the first distance (D) is determined by applying to the second distance (d) an increasing function fk depending on the reduction factor.

3. Method according to claim 2, wherein the increasing function fk is defined on an interval [0 ; Ak [ and the values ​​fk(d) of the increasing function fk tend towards infinity as d tends towards Ak.

4. A method according to claim 3, wherein the reduction factor is the inverse of a desired magnification factor (k) and wherein

5.

6.

7.

8. the upper bound (Ak) of said interval [0 ; Ak [ is increasing as a function of the desired magnification factor (k). A method according to any one of claims 1 to 4, wherein the second transformation is carried out by means of a homothety with a ratio equal to the reduction factor. A method according to any one of claims 1 to 5, wherein the third transformation transforms an input point into an output point such that the input point, the output point and said associated point are aligned and a third distance (d') between the output point and said associated point is less than a fourth distance (D') between the input point and said associated point. A method according to any one of claims 1 to 6, comprising a step (E12) of displaying the image on a display device. An electronic device (2) designed to construct an image based on a set of values ​​respectively associated with photosensitive elements (12) of a sensor (6) receiving a light flux through an optical element (4) causing a distortion of the light flux, comprising: - a memory (10) storing, for each pixel of the image, coordinates (X, Y) of the pixel in the image; - a processor (8) configured to determine, for each pixel of the image, modified coordinates (X3, Y3) based on the coordinates (X, Y) stored in memory for that pixel, and to construct the image by assigning to each pixel of the image the value associated with the photosensitive element (12) defined by the modified coordinates determined for that pixel of the image, characterized in that the processor (8) is configured to determine the modified coordinates (X3, Y3) for each pixel of the image based on the coordinates (X, Y) stored in memory (10) for that pixel by successively applying a plurality of transformations comprising: - a first transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element (4) are aligned and that a first distance (D) between the output point and said associated point is greater than a second distance (d) between the input point and said associated point; - a second transformation resulting in a reduction of distances by application of a reduction factor; - a third transformation having the effect of said distortion in the plane of the photosensitive elements (12) of the sensor (6), in which the ratio between the first distance (D) and the second distance (d) depends on the reduction factor.

9. Electronic device according to claim 8, comprising said sensor (6).

10. Computer program comprising instructions executable by a processor (8) and designed to implement a method according to any one of claims 1 to 7 when these instructions are executed by the processor (8).

Images