Image construction process, associated electronic device and computer program

Abstract

An image construction process includes the following steps: - for each photosensitive element of a sensor, determination (E4, E6, E8) of modified coordinates (X3, Y3) on the basis of coordinates (X, Y) of the photosensitive element concerned; - construction (E10) of the image by assigning the value associated with a given photosensitive element to a pixel of the image defined by the modified coordinates determined for that given photosensitive element.The modified coordinates are determined for each photosensitive element based on its (X, Y) coordinates by successively applying a plurality of transformations comprising: - a first transformation (E4) compensating for the effect of distortion caused by an optical element in the plane of the photosensitive elements; - a second transformation (E6) resulting in an increase in distances by applying a magnification factor; - a third transformation (E8) reducing distances relative to a central point. The reduction in distances during the third transformation depends on the magnification factor. See Figure 2 for the abbreviated version.

Description

Title of the invention: Method for constructing an image, associated electronic device and computer program Technical field of the invention

[0001] The present invention relates to the technical field of image acquisition using a sensor.

[0002] It relates in particular to a method for constructing an image, as well as an associated electronic device and computer program. State of the art

[0003] It is known to use electronic devices (for example, video camera type) using a sensor comprising photosensitive elements to acquire an image.

[0004] An optical element, such as a lens, is generally placed in the path of the light flux incident on the sensor so as to image the environment to be observed by means of the sensor in the plane of the photosensitive elements.

[0005] The optical element may, however, create a distortion of the light flux incident on the sensor, in particular when the optical element has a reduced focal length so as to allow observation with a large field angle.

[0006] Techniques for compensating this distortion have been proposed, but the resulting image lacks realism due to a distortion of the objects present in the observed scene. Presentation of the invention

[0007] In this context, the present invention proposes a method for constructing an image based on a set of values ​​respectively associated with photosensitive elements of a sensor receiving a light flux through an optical element causing a distortion of the light flux, comprising the following steps:

[0008] - for each photosensitive element, determination of modified coordinates on the coordinate base of the photosensitive element concerned within the sensor;

[0009] - image construction by assigning (for at least some of the elements photosensitive, that is to say for example for photosensitive elements for which the modified coordinates are located in the image) the value associated with a given photosensitive element at a pixel of the image defined by the modified coordinates determined for that given photosensitive element,

[0010] characterized in that the modified coordinates are determined for each photosensitive element on the basis of the coordinates of that photosensitive element within the sensor by successive application of a plurality of transformations including:

[0011] - a first transformation compensating for the effect of said distortion (i.e. having the opposite effect of said distortion) in the plane of the photosensitive elements of the sensor;

[0012] - a second transformation resulting in an increase in distances by application of a magnification factor;

[0013] - a third transformation transforming an input point into an output point and such that the entry point, the exit point and a point associated with the optical axis of the optical element are aligned and that a first distance between the exit point and said associated point is (strictly) less than a second distance between the entry point and said associated point,

[0014] in which the ratio between the first distance and the second distance depends on the magnification factor.

[0015] The distortion caused by the optical element is corrected by the first transformation. The third transformation, however, corrects the constructed image to avoid distortion of the objects it contains, modulating this effect according to the magnification performed by the second transformation. Indeed, the distortion of objects caused by the first transformation occurs mainly at low image magnification.

[0016] The first distance is for example determined by applying to the second distance an increasing function fk depending on the reduction factor.

[0017] According to one possible embodiment, the increasing function fk can have as its image an interval [0 ; Ak [.

[0018] The values ​​fk(D) of the increasing function fk can then, for example, tend towards Ak when D tends towards infinity.

[0019] The upper bound of said interval [0; Ak[ may also be increasing as a function of the magnification factor.

[0020] The second transformation can be carried out by means of a homothety with a ratio equal to the magnification factor.

[0021] The first transformation can for example transform an input point into an output point such that the input point, the output point and said associated point are aligned and that a third distance between the output point and said associated point is greater than a fourth distance between the input point and said associated point.

[0022] The method may further include a step of displaying the image on a display device.

[0023] The invention also proposes an electronic device designed to construct an image based on a set of values ​​respectively associated with photosensitive elements of a sensor receiving a light flux through an optical element causing a distortion of the light flux, comprising:

[0024] - a memory storing, for each photosensitive element, coordinates of the photosensitive element within the sensor;

[0025] - a processor configured to determine, for each photosensitive element, modified coordinates based on the coordinates stored in memory for this photosensitive element, and to construct the image by assigning the value associated with a given photosensitive element to a pixel of the image defined by the modified coordinates determined for that given photosensitive element,

[0026] characterized in that the processor is configured to determine the modified coordinates for each photosensitive element based on the coordinates stored in memory for that photosensitive element by successively applying a plurality of transformations comprising:

[0027] - a first transformation compensating for the effect of said distortion in the plane of photosensitive elements of the sensor;

[0028] - a second transformation resulting in an increase in distances by application of a magnification factor;

[0029] - a third transformation transforming an input point into an output point and such that the entry point, the exit point and a point associated with the optical axis of the optical element are aligned and that a first distance between the exit point and said associated point is less than a second distance between the entry point and said associated point,

[0030] in which the ratio between the first distance and the second distance depends on the magnification factor.

[0031] This electronic device may include said sensor (as well as optionally the optical element).

[0032] The invention finally proposes a computer program comprising instructions executable by a processor and designed to implement a process as defined above when these instructions are executed by the processor.

[0033] The optional characteristics presented above in terms of method can also be applied to this electronic device.

[0034] Of course, the various features, variants, and embodiments of the invention can be combined in various ways, provided they are not incompatible or mutually exclusive. Detailed description of the invention

[0035] In addition, various other features of the invention become apparent from the attached description made with reference to the drawings which illustrate non-limiting embodiments of the invention and where:

[0036] [Fig. 1] represents the main elements of an example of an electronic device according to the invention;

[0037] [Fig.2] is a logic diagram showing an example of a method for constructing an image according to the invention;

[0038] [Fig.3] represents the distortion caused by an optical element of the electronic device of [Fig.1]; and

[0039] [Fig.4] represents functions used in the process of [Fig.2].

[0040] Fig. 1 represents the main elements of an example of an electronic device according to the invention.

[0041] Such an electronic device 2 (for example a video camera) comprises an optical element 4, a sensor 6, a processor 8 and a memory 10. Such an electronic device is for example mounted in a vehicle (such as a motor vehicle) in order to acquire an image (or a plurality of images so as to form a video sequence) of the interior (for example of the passenger compartment) of the vehicle or of an environment outside the vehicle.

[0042] The sensor 6 comprises an array of photosensitive elements 12.

[0043] The optical element 4 (for example, a lens, here a wide-angle and / or short focal length lens, or "fisheye" lens) images the environment observed by the camera in the plane of the photosensitive elements 12 of the sensor 6. The focal length of the lens 4 is, for example, between 2 mm and 3 mm, and is 2.5 mm in the example described. The angle of view is, for example, 160° in the horizontal plane and 90° in the vertical plane.

[0044] The photosensitive elements 12 of the sensor thus receive a luminous flux through the optical element 4 which causes (due to the reduced focal length) a distortion of this luminous flux, in particular in the plane of the photosensitive elements 12.

[0045] Each photosensitive element 12 produces a value representative of the intensity of the light flux incident on the photosensitive element 12 concerned.

[0046] These values ​​respectively produced by the photosensitive elements 12 of the sensor 6 are processed by the processor 8 to construct (according to a process described below) an image to be displayed on a display device (not shown).

[0047] The processor 8 (for example a microprocessor) thus implements a process for processing the values ​​produced by the photosensitive elements 12 and for constructing the aforementioned image, as described below with reference to [Fig.2], due to the execution by this processor 8 of computer program instructions (stored for example in memory 10).

[0048] Figure 2 is a flowchart showing an example of a construction process of an image conforming to the invention.

[0049] As indicated above, this process is implemented here by the processor 8 due to the execution by this processor 8 of computer program instructions stored in memory 10.

[0050] It is assumed here that the user wants a magnification of a factor k of the image (the desired magnification factor being entered for example by the user by means of a user interface not shown).

[0051] This process begins with a step E2 during which the processor 8 defines a template comprising, for each photosensitive element 12 of the sensor 6, X, Y coordinates of this photosensitive element 12 within the sensor 6 and stores these X, Y coordinates in the memory 10.

[0052] The X, Y coordinates of a photosensitive element 12 within the sensor 6 are, for example, two integers that respectively define the horizontal and vertical positions of the photosensitive element 12 within the sensor 6 (this sensor being matrix-type, as already indicated). Other coordinate systems may, however, be used as an alternative.

[0053] The X, Y coordinates thus stored (initial coordinates) are then processed (by the processor 8) by three successive transformations described below in steps E4 to E8 in order to obtain, for each photosensitive element 12, modified coordinates X3, Y3. In other words, for each photosensitive element 12 of the sensor 6, the X, Y coordinates associated with this photosensitive element 12 in step E2 undergo successively the three transformations described in steps E4 to E8 in order to obtain modified coordinates X3, Y3 associated with this photosensitive element.

[0054] The processor 8 applies at step E4, for each photosensitive element 12 of the sensor 6, a first transformation of the X, Y coordinates relative to this photosensitive element 12.

[0055] This first transformation transforms input coordinates (here the initial coordinates X, Y) into output coordinates XI, Yl.

[0056] The first transformation compensates for the distortion caused by the optical element 4 in the plane of the photosensitive elements 12 of the sensor 6. To do this, the first transformation has the opposite effect of this distortion.

[0057] As seen in [Fig.3], the distortion caused by the optical element 4 has the effect of sending onto the sensor 6, at a distance d' from the optical axis of the optical element 4, a light ray which should have reached the sensor 6 at a distance D' from the optical axis of the optical element 4, with D' > d'.

[0058] The first transformation, in order to compensate for the effect of distortion, therefore causes an increase in distances around a central point (sensor point) 6 associated with the optical axis of the optical element 4), this increase in distances corresponding to the inverse effect of distortion in the plane of the photosensitive elements 12.

[0059] The first transformation is thus a transformation of the plane (i.e., defined in the plane) and transforms the input point (with coordinates X, Y) into an output point (with coordinates XI, Yl) defined as follows:

[0060] - the entry point, the exit point and the central point are aligned;

[0061] - the distance D' between the exit point and the central point is determined by application of a function g to the distance d between the entry point and the central point, the function g representing the inverse effect of the distortion in the plane of the photosensitive elements 12 and being such that D' = g(d') > d'.

[0062] The function g is independent of the desired magnification factor k and the first transformation carried out is therefore the same regardless of the desired magnification factor k.

[0063] The application of the function g is for example carried out by reading from a lookup table (stored here in memory 10). This lookup table stores a plurality of values ​​g(d') respectively associated with a corresponding plurality of values ​​d' and determined for example by means of prior measurements carried out on the optical element 4 concerned (or an optical element of the same type).

[0064] The processor 8 applies at step E6, for each photosensitive element 12, a second transformation of the coordinates (here XI, Yl) relative to this photosensitive element 12. This second transformation transforms input coordinates (here the coordinates XI, Yl produced at the output of step E4) into output coordinates X2, Y2.

[0065] The second transformation results in a magnification of distances by application of the magnification factor k.

[0066] The second transformation is for example carried out in practice by applying a homothety with ratio k centered on the central point defined above.

[0067] The processor 8 applies at step E8, for each photosensitive element 12, a third transformation of the coordinates (here X2, Y2) relative to this photosensitive element 12.

[0068] This third transformation is a transformation of the plane (i.e. a transformation defined in the plane) which transforms input coordinates (defining an entry point), here the coordinates X2, Y2 produced at the output of step E6, into output coordinates X3, Y3 (defining an output point).

[0069] This third transformation transforms the input point (with coordinates X2, Y2) into an output point (with coordinates X3, Y3) defined as follows:

[0070] - the entry point, the exit point and the central point are aligned;

[0071] - the distance d between the exit point and the central point is determined by application of a function fk at the distance D between the entry point and the central point.

[0072] In the example described here, it is assumed that the image to be constructed is centered on the sensor 6 and that the sensor 6 and the optical element 4 are aligned, so that the point associated with the optical axis of the optical element 4 (central point) is located at the center of the image to be constructed.

[0073] The function fk used during the current implementation of the third transformation depends on the desired magnification factor k.

[0074] We have represented on [Fig.4] three functions fkmin, fk, fkmax, where kmin is the minimum usable magnification factor and kmax is the maximum usable magnification factor; we therefore have kmin < k < kmax.

[0075] As can be seen in [Fig.4], all the fk functions used have the following properties:

[0076] - fk is increasing (here over its entire domain, this domain being the set of positive real numbers);

[0077] - for all x in its domain of definition, fk(x) < x (therefore, the curve representative of any function fk is located below the line with equation (d=D) as shown in [Fig.4]).

[0078] Thus, the distance d between the exit point and the central point is less (strictly) than the distance D between the entry point and the central point.

[0079] Furthermore, at the point with abscissa D=0, the graph of any function fk is tangent to the line with equation (d=D), as also shown in [Fig.4]. (In other words, the derivative of any function fk at the point with abscissa D=0 is 1.)

[0080] Thus, the third transformation does little to modify the entry points associated with a small distance D, that is to say located close to the central point.

[0081] Moreover, as can be seen in [Fig.4], the functions fk are distinct from one another so that, for each possible value of the distance D between the entry point and the central point, the ratio between the distance d = fk(D) and the distance D depends on the factor k.

[0082] Specifically, the ratio (less than 1) between the distance d (distance between the exit point and the central point) and the distance D (distance between the entry point and the central point) increases (approaching 1) when the desired magnification factor k increases.

[0083] In other words, the third transformation causes a significant contraction relative to the central point (and especially at a distance from the central point) when the desired magnification factor k is low, but a less significant contraction relative to the central point when the desired magnification factor is significant.

[0084] Moreover, as also shown in [Fig. 4], each function fk has as its image an interval [0; Ak[ (or, in other words, each function fk takes values ​​in the interval [0; Ak[) and the value fk(x) tends towards Ak as x tends towards infinity. In other words, for each function fk, the (horizontal) line with equation (y = Ak) is an asymptote to the graph of that function fk.

[0085] For example, the following values ​​are used:

[0086] - Ak min = r / 2, where r is the radius of the largest circle centered on the central point and inscribed in the image to be constructed (in the coordinate system linked to the image used for the image construction in step E10);

[0087] - Ak = Ak min.k / kmin for other values ​​of k.

[0088] The upper bound Ak of ​​the image [0; Ak[ of the function fk is therefore increasing here in function of the desired magnification factor k.

[0089] We can use, for example, the family of functions fk defined by:

[0090] fk(d) = (2.Ak / jt). atan(0.5.ir.d / Ak)

[0091] with here as already indicated Ak = (rk) / (2.kmin) and atan the trigonometric function "arc-tangent".

[0092] In practice, the application of the function fk is, for example, carried out by reading from a look-up table. For this purpose, memory 10 stores, for example, for each of a plurality of ranges of values ​​of k (these ranges of values ​​covering the allowed values ​​of the magnification coefficient k), a look-up table associating with distance values ​​D respectively the distance values ​​fk(D).

[0093] In this case, the application of the function fk includes the selection of a lookup table associated with the coefficient k and the reading of the distance d associated with the distance D in the selected lookup table.

[0094] The application of the third transformation makes it possible to compensate for the adverse effects of the distortion correction carried out by means of the first transformation (step E4).

[0095] Indeed, the third transformation tends to bring back towards the central point the pixel to which a given photosensitive element will be assigned.

[0096] The use of the value Ak min = r / 2 allows in this respect that with the minimum magnification coefficient, even distant points (in particular due to the distortion correction in step E4) are brought to the edge of the constructed image.

[0097] The processor 8 can then construct the image (step E10) by assigning the value produced by any photosensitive element defined by the initial coordinates X, Y to the pixel defined by the modified coordinates X3, Y3 determined for the photosensitive element concerned, provided that these modified coordinates X3, Y3 are located in the image.

[0098] Thanks to the first transformation performed in step E4, the value produced by a photosensitive element is assigned to a pixel corresponding to the arrival point of the light beam in question in the absence of distortion, and the distortion is thus compensated. The third transformation performed in step E8, however, partially compensates for this correction, depending on the desired magnification factor (magnification achieved by means of the second transformation in step E6), so as to avoid excessive distortion of objects in the image, particularly visible in the absence of magnification or in the case of low magnification.

[0099] The method can then include a step E12 of displaying the image constructed on a display device (not shown).

[0100] The embodiment described above is only one possible example of implementing the invention. It is also possible, in particular as an alternative, to carry out the transformations in a different order than that presented above, for example, to carry them out in the following order: second transformation, first transformation, third transformation.

[0101] Furthermore, in the case where the electronic device is a video camera, the processing described above for constructing an image is applied to each of the images of the video sequence taken by the video camera.

Claims

Demands

1. Method of constructing an image on the basis of a set of values ​​respectively associated with photosensitive elements (12) of a sensor (6) receiving a light flux through an optical element (4) causing a distortion of the light flux, comprising the following steps: - for each photosensitive element (12), determination (E4, E6, E8) of modified coordinates (X3, Y3) on the basis of coordinates (X, Y) of the photosensitive element concerned (12) within the sensor (6);- construction (E10) of the image by assigning the value associated with a given photosensitive element (12) to a pixel of the image defined by the modified coordinates (X3, Y3) determined for this given photosensitive element (12), characterized in that the modified coordinates (X3, Y3) are determined for each photosensitive element (12) on the basis of the coordinates (X, Y) of this photosensitive element (12) within the sensor (6) by successive application of a plurality of transformations comprising: - a first transformation (E4) compensating the effect of said distortion in the plane of the photosensitive elements (12) of the sensor (6); - a second transformation (E6) resulting in an increase in distances by application of a magnification factor (k);- a third transformation (E8) transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element (4) are aligned and that a first distance (d) between the output point and said associated point is less than a second distance (D) between the input point and said associated point, in which the ratio between the first distance (d) and the second distance (D) depends on the magnification factor (k).;

2. Method according to claim 1, wherein the first distance (d) is determined by applying to the second distance (D) an increasing function fk depending on the reduction factor.

3. Method according to claim 2, wherein the increasing function fk has as its image an interval [0 ; Ak [ and the values ​​fk(D) of the increasing function fk tend towards Ak as D tends towards infinity.

4. Method according to claim 3, wherein the upper bound (Ak) of said interval [0; Ak[ is increasing as a function of the magnification factor (k).

5. A method according to any one of claims 1 to 4, wherein the second transformation is carried out by means of a homothety with a ratio equal to the magnification factor.

6. A method according to any one of claims 1 to 5, wherein the first transformation transforms an input point into an output point such that the input point, the output point and said associated point are aligned and a third distance (D') between the output point and said associated point is greater than a fourth distance (d') between the input point and said associated point.

7. A method according to any one of claims 1 to 6, comprising a step (E12) of displaying the image on a display device.

8. Electronic device (2) designed to construct an image on the basis of a set of values ​​respectively associated with photosensitive elements (12) of a sensor (6) receiving a light flux through an optical element (4) causing a distortion of the light flux, comprising: - a memory (10) storing, for each photosensitive element, coordinates (X, Y) of the photosensitive element within the sensor (6);- a processor (8) configured to determine, for each photosensitive element, modified coordinates (X3, Y3) on the basis of the coordinates (X, Y) stored in the memory for that photosensitive element, and to construct the image by assigning the value associated with a given photosensitive element (12) to a pixel of the image defined by the modified coordinates (X3, Y3) determined for that given photosensitive element, characterized in that the processor (8) is configured to determine the modified coordinates (X3, Y3) for each photosensitive element (12) on the basis of the coordinates (X, Y) stored in the memory (10) for that photosensitive element by successive application of a plurality of transformations comprising: - a first transformation compensating for the effect of said distortion in the plane of the photosensitive elements (12) of the sensor (6); - a second transformation resulting in an increase in distances by application of a magnification factor (k);

9.

10. - a third transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element (4) are aligned and that a first distance (d) between the output point and said associated point is less than a second distance (D) between the input point and said associated point, in which the ratio between the first distance (d) and the second distance (D) depends on the magnification factor (k). Electronic device according to claim 8, comprising said sensor (6). Computer program comprising instructions executable by a processor (8) and designed to implement a method according to any one of claims 1 to 7 when these instructions are executed by the processor (8).

Images