Method for constructing an image, and associated electronic device and computer program

Abstract

Disclosed is a method for constructing an image, which method comprises the following steps: - for each photosensitive element of a sensor, determining (E4, E6, E8) modified coordinates (X3, Y3) on the basis of coordinates (X, Y) of the photosensitive element in question; - constructing (E10) the image by assigning the value associated with a given photosensitive element to a pixel of the image defined by the modified coordinates determined for the given photosensitive element. The modified coordinates are determined for each photosensitive element on the basis of the coordinates (X, Y) of the photosensitive element by successive application of a plurality of transformations comprising: - a first transformation (E4) compensating for the effect of a distortion caused by an optical element in the plane of the photosensitive elements; - a second transformation (E6) causing an increase in distances through application of a magnification factor; - a third transformation (E8) reducing distances with respect to a central point. The reduction in distances in the third transformation depends on the magnification factor.

Description

Image construction process, associated electronic device and computer program Technical field of the invention

[0001] The present invention relates to the technical field of image acquisition using a sensor.

[0002] It relates in particular to a process for constructing an image, as well as an associated electronic device and computer program. State of the art

[0003] It is known to use electronic devices (e.g., video cameras) that use a sensor with photosensitive elements to acquire an image.

[0004] An optical element, such as a lens, is generally placed in the path of the light flux incident on the sensor so as to image the environment to be observed by the sensor in the plane of the photosensitive elements.

[0005] However, the optical element can create distortion of the light flux incident on the sensor, particularly when the optical element has a reduced focal length to allow observation with a wide field of view.

[0006] Techniques to compensate for this distortion have been proposed, but the resulting image lacks realism due to a distortion of the objects present in the observed scene. Presentation of the invention

[0007] In this context, the present invention proposes a method for constructing an image based on a set of values ​​respectively associated with photosensitive elements of a sensor receiving a light flux through an optical element causing a distortion of the light flux, comprising the following steps:

[0008] - for each photosensitive element, determination of modified coordinates based on the coordinates of the photosensitive element concerned within the sensor;

[0009] - image construction by assigning (for at least some of the photosensitive elements, i.e. for example for the photosensitive elements for which the modified coordinates are located in the image) the value associated with a given photosensitive element to a pixel of the image defined by the modified coordinates determined for that given photosensitive element,

[0010] characterized in that the modified coordinates are determined for each photosensitive element based on the coordinates of that photosensitive element within the sensor by successive application of a plurality of transformations comprising:

[0011] - a first transformation compensating for the effect of said distortion (i.e. having the opposite effect of said distortion) in the plane of the photosensitive elements of the sensor;

[0012] - a second transformation resulting in an increase in distances by applying a magnification factor;

[0013] - a third transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element are aligned and that a first distance between the output point and said associated point is (strictly) less than a second distance between the input point and said associated point,

[0014] in which the ratio between the first distance and the second distance depends on the magnification factor.

[0015] The distortion caused by the optical element is corrected by the first transformation. The third transformation, however, corrects the resulting image to prevent distortion of the objects it contains, modulating this effect according to the magnification performed by the second transformation. Indeed, the distortion of objects caused by the first transformation occurs primarily at low image magnification.

[0016] The first distance is, for example, determined by applying an increasing function f to the second distance. k dependent on the reduction factor.

[0017] According to one possible implementation, the increasing function f k can have as its image an interval [0 ; Δ k [.

[0018] The values ​​f k (D) of the increasing function f k can then, for example, tend towards Δ k when D tends towards infinity.

[0019] The upper bound of said interval [0; Δ k [can also be increasing depending on the magnification factor.

[0020] The second transformation can be achieved by means of a homothety with a ratio equal to the magnification factor.

[0021] The first transformation can, for example, transform an input point into an output point such that the input point, the output point and said associated point are aligned and a third distance between the output point and said associated point is greater than a fourth distance between the input point and said associated point.

[0022] The process may further include a step of displaying the image on a display device.

[0023] The invention also proposes an electronic device designed to construct an image based on a set of values ​​respectively associated with photosensitive elements of a sensor receiving a light flux through an optical element causing a distortion of the light flux, comprising:

[0024] - a memory storing, for each photosensitive element, the coordinates of the photosensitive element within the sensor;

[0025] - a processor configured to determine, for each photosensitive element, modified coordinates based on the coordinates stored in memory for that photosensitive element, and to construct the image by assigning the value associated with a given photosensitive element to a pixel of the image defined by the modified coordinates determined for that given photosensitive element,

[0026] characterized in that the processor is configured to determine the modified coordinates for each photosensitive element based on the coordinates stored in memory for that photosensitive element by successively applying a plurality of transformations comprising:

[0027] - a first transformation compensating for the effect of said distortion in the plane of the photosensitive elements of the sensor;

[0028] - a second transformation resulting in an increase in distances by applying a magnification factor;

[0029] - a third transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element are aligned and that a first distance between the output point and said associated point is less than a second distance between the input point and said associated point,

[0030] in which the ratio between the first distance and the second distance depends on the magnification factor.

[0031] This electronic device may include said sensor (as well as possibly the optical element).

[0032] The invention finally proposes a computer program comprising instructions executable by a processor and designed to implement a process as defined above when these instructions are executed by the processor.

[0033] The optional process characteristics described above can also be applied to this electronic device.

[0034] Of course, the different features, variants and embodiments of the invention can be combined with each other in various ways as long as they are not incompatible or mutually exclusive. Detailed description of the invention

[0035] Furthermore, various other features of the invention become apparent from the attached description made with reference to the drawings which illustrate non-limiting embodiments of the invention and where:

[0036] represents the main elements of an example of an electronic device conforming to the invention;

[0037] is a logic diagram showing an example of a method for constructing an image according to the invention;

[0038] represents the distortion caused by an optical element of the electronic device; and

[0039] represents functions used in the process of the.

[0040] Lare represents the main elements of an example of an electronic device according to the invention.

[0041] Such an electronic device 2 (for example a video camera) includes an optical element 4, a sensor 6, a processor 8 and a memory 10. Such an electronic device is for example mounted in a vehicle (such as a motor vehicle) in order to acquire an image (or a plurality of images so as to form a video sequence) of the interior (for example of the passenger compartment) of the vehicle or of an environment outside the vehicle.

[0042] The sensor 6 comprises an array of 12 photosensitive elements.

[0043] The optical element 4 (for example, a lens, here a wide-angle and / or short focal length lens, or "fisheye" lens) images the environment observed by the camera in the plane of the photosensitive elements 12 of the sensor 6. The focal length of lens 4 is, for example, between 2 mm and 3 mm, and is 2.5 mm in the example described. The angle of view is, for example, 160° in the horizontal plane and 90° in the vertical plane.

[0044] The photosensitive elements 12 of the sensor thus receive a luminous flux through the optical element 4 which causes (due to the reduced focal length) a distortion of this luminous flux, particularly in the plane of the photosensitive elements 12.

[0045] Each photosensitive element 12 produces a value representative of the intensity of the light flux incident on the photosensitive element 12 concerned.

[0046] These values ​​produced respectively by the photosensitive elements 12 of the sensor 6 are processed by the processor 8 to construct (according to a process described below) an image to be displayed on a display device (not shown).

[0047] The processor 8 (for example a microprocessor) thus implements a process of processing the values ​​produced by the photosensitive elements 12 and of constructing the aforementioned image, as described below with reference to the, due to the execution by this processor 8 of computer program instructions (stored for example in memory 10).

[0048] This is a flowchart showing an example of a method for constructing an image according to the invention.

[0049] As indicated above, this process is implemented here by processor 8 due to the execution by this processor 8 of computer program instructions stored in memory 10.

[0050] We assume here that the user wants a magnification of the image by a factor k (the desired magnification factor being entered by the user for example using a user interface not shown).

[0051] This process begins with a step E2 during which the processor 8 defines a template including, for each photosensitive element 12 of the sensor 6, X, Y coordinates of this photosensitive element 12 within the sensor 6 and stores these X, Y coordinates in memory 10.

[0052] The X, Y coordinates of a photosensitive element 12 within the sensor 6 are, for example, two integers that respectively define the horizontal and vertical positions of the photosensitive element 12 within the sensor 6 (this sensor being matrix-based, as already mentioned). Other coordinate systems can, however, be used as alternatives.

[0053] The X, Y coordinates thus stored (initial coordinates) are then processed (by the processor 8) by three successive transformations described below in steps E4 to E8 in order to obtain, for each photosensitive element 12, modified coordinates X3, Y3. In other words, for each photosensitive element 12 of the sensor 6, the X, Y coordinates associated with this photosensitive element 12 in step E2 undergo successively the three transformations described in steps E4 to E8 in order to obtain modified coordinates X3, Y3 associated with this photosensitive element.

[0054] The processor 8 applies at step E4, for each photosensitive element 12 of the sensor 6, a first transformation of the X, Y coordinates relative to this photosensitive element 12.

[0055] This first transformation transforms input coordinates (here the initial coordinates X, Y) into output coordinates X1, Y1.

[0056] The first transformation compensates for the distortion caused by the optical element 4 in the plane of the photosensitive elements 12 of the sensor 6. To do this, the first transformation has the opposite effect of this distortion.

[0057] As can be seen in, the distortion caused by the optical element 4 has the effect of sending onto the sensor 6, at a distance d' from the optical axis of the optical element 4, a light ray which should have reached the sensor 6 at a distance D' from the optical axis of the optical element 4, with D' > d'.

[0058] The first transformation, in order to compensate for the effect of distortion, therefore causes an increase in distances around a central point (point of the sensor 6 associated with the optical axis of the optical element 4), this increase in distances corresponding to the inverse effect of distortion in the plane of the photosensitive elements 12.

[0059] The first transformation is thus a transformation of the plane (that is, defined in the plane) and transforms the input point (with coordinates X, Y) into an output point (with coordinates X1, Y1) defined as follows:

[0060] - the entry point, the exit point and the central point are aligned;

[0061] - the distance D' between the exit point and the central point is determined by applying a function g to the distance d' between the entry point and the central point, the function g representing the inverse effect of the distortion in the plane of the photosensitive elements 12 and being such that D' = g(d') > d'.

[0062] The function g is independent of the desired magnification factor k and the first transformation performed is therefore the same regardless of the desired magnification factor k.

[0063] The application of the function g is, for example, carried out by reading from a lookup table (stored here in memory 10). This lookup table stores a plurality of values ​​g(d') respectively associated with a corresponding plurality of values ​​d' and determined, for example, by means of prior measurements carried out on the optical element 4 concerned (or an optical element of the same type).

[0064] The processor 8 applies at step E6, for each photosensitive element 12, a second transformation of the coordinates (here X1, Y1) relative to this photosensitive element 12. This second transformation transforms input coordinates (here the coordinates X1, Y1 produced at the output of step E4) into output coordinates X2, Y2.

[0065] The second transformation results in a magnification of distances by applying the magnification factor k.

[0066] The second transformation is, for example, carried out in practice by applying a homothety with ratio k centered on the central point defined above.

[0067] The processor 8 applies at step E8, for each photosensitive element 12, a third transformation of the coordinates (here X2, Y2) relative to this photosensitive element 12.

[0068] This third transformation is a transformation of the plane (that is, a transformation defined in the plane) which transforms input coordinates (defining an entry point), here the coordinates X2, Y2 produced at the output of step E6, into output coordinates X3, Y3 (defining an output point).

[0069] This third transformation turns the input point (with coordinates X2, Y2) into an output point (with coordinates X3, Y3) defined as follows:

[0070] - the entry point, the exit point and the central point are aligned;

[0071] - The distance d between the exit point and the central point is determined by applying a function f k at the distance D between the entry point and the central point.

[0072] In the example described here, we consider that the image to be constructed is centered on sensor 6 and that sensor 6 and optical element 4 are aligned, so that the point associated with the optical axis of optical element 4 (central point) is located at the center of the image to be constructed.

[0073] The function f k used during the current implementation of the third transformation depends on the desired magnification factor k.

[0074] We have represented on the three functions f kmin , f k , f kmax , where kmin is the minimum usable magnification factor and kmax is the maximum usable magnification factor; therefore, we have kmin < k < kmax.

[0075] As can be seen on the, all the functions fk used have the following properties:

[0076] - f k is increasing (here over its entire domain, this domain being the set of positive real numbers);

[0077] - for all x in its domain, f k (x) < x (therefore, the graph of any function f k is located below the line with equation (d=D) as shown in).

[0078] Thus, the distance d between the exit point and the central point is less (strictly) than the distance D between the entry point and the central point.

[0079] Furthermore, at the point with abscissa D=0, the representative curve of any function f k is tangent to the line with equation (d=D), as also seen in. (In other words, the derivative of any function f k (at the point with abscissa D=0, the value is 1.)

[0080] Thus, the third transformation does little to modify the entry points associated with a small distance D, that is, those located close to the central point.

[0081] Furthermore, as can be seen on the, the functions f k are distinct from each other so that, for each possible value of the distance D between the entry point and the central point, the ratio between the distance d = f k (D) and the distance D depends on the factor k.

[0082] Specifically, the ratio (less than 1) between the distance d (distance between the exit point and the central point) and the distance D (distance between the entry point and the central point) increases (approaching 1) as the desired magnification factor k increases.

[0083] In other words, the third transformation causes a large contraction relative to the central point (and especially at a distance from the central point) when the desired magnification factor k is small, but a smaller contraction relative to the central point when the desired magnification factor is large.

[0084] Furthermore, as also visible in, each function f k has as its image an interval [0 ; Δ k [ (or, in other words, each function f k is to values ​​in the interval [0 ; Δ k [) and the value f k (x) tends towards Δ k as x tends towards infinity. In other words, for each function f k , the (horizontal) line with equation (y = Δ k ) is an asymptote to the graph of this function f k .

[0085] For example, the following values ​​are used:

[0086] - Δ k min= r / 2, where r is the radius of the largest circle centered on the central point and inscribed in the image to be constructed (in the coordinate system linked to the image used for the construction of the image in step E10);

[0087] - Δ k = Δ k min .k / kmin for other values ​​of k.

[0088] The upper bound Δ k of the image [0; Δ k [of the function f k is therefore increasing here as a function of the desired magnification factor k.

[0089] We can use, for example, the family of functions f k defined by:

[0090] f k (d) = (2.Δ k / π) . atan(0,5.π.d / Δ k )

[0091] with here as already indicated Δ k = (rk) / (2.kmin) and atan the trigonometric function "arc-tangent".

[0092] In practice, the application of the function f kThis is achieved, for example, by reading from a lookup table. For this purpose, memory 10 stores, for each of a plurality of ranges of values ​​of k (these ranges covering the allowed values ​​of the magnification coefficient k), a lookup table associating distance values ​​D with corresponding distance values ​​f. k (D).

[0093] In this case, the application of the function f k includes selecting a lookup table associated with the coefficient k and reading the distance d associated with the distance D in the selected lookup table.

[0094] Applying the third transformation makes it possible to compensate for the adverse effects of the distortion correction carried out using the first transformation (step E4).

[0095] Indeed, the third transformation tends to bring back towards the central point the pixel to which a given photosensitive element will be assigned.

[0096] The use of the value Δ k min = r / 2 allows in this respect that with the minimum magnification coefficient, even distant points (particularly due to the distortion correction in step E4) are brought to the edge of the constructed image.

[0097] The processor 8 can then construct the image (step E10) by assigning the value produced by any photosensitive element defined by the initial coordinates X, Y to the pixel defined by the modified coordinates X3, Y3 determined for the photosensitive element concerned, provided that these modified coordinates X3, Y3 are located in the image.

[0098] Thanks to the first transformation performed in step E4, the value produced by a photosensitive element is assigned to a pixel corresponding to the arrival point of the light beam in question in the absence of distortion, and the distortion is thus compensated. The third transformation performed in step E8, however, partially compensates for this correction, depending on the desired magnification factor (magnification achieved using the second transformation in step E6), in order to avoid excessive distortion of objects in the image, particularly noticeable in the absence of magnification or at low magnification.

[0099] The process can then include a step E12 of displaying the constructed image on a display device (not shown).

[0100] The embodiment described above is only one possible example of implementing the invention. As an alternative, the transformations can be carried out in a different order than that presented above, for example, in the following order: second transformation, first transformation, third transformation.

[0101] Furthermore, in the case where the electronic device is a video camera, the processing described above to construct an image is applied to each of the images of the video sequence taken by the video camera.

Claims

Method of constructing an image on the basis of a set of values ​​respectively associated with photosensitive elements (12) of a sensor (6) receiving a light flux through an optical element (4) causing a distortion of the light flux, comprising the following steps:- for each photosensitive element (12), determination (E4, E6, E8) of modified coordinates (X3, Y3) on the basis of coordinates (X, Y) of the photosensitive element concerned (12) within the sensor (6);- construction (E10) of the image by assigning the value associated with a given photosensitive element (12) to a pixel of the image defined by the modified coordinates (X3, Y3) determined for this given photosensitive element (12), characterized in that the modified coordinates (X3, Y3) are determined for each photosensitive element (12) on the basis of the coordinates (X, Y) of this photosensitive element (12) within the sensor (6) by successive application of a plurality of transformations including: - a first transformation (E4) compensating the effect of said distortion in the plane of the photosensitive elements (12) of the sensor (6); - a second transformation (E6) resulting in an increase in distances by application of a magnification factor (k);- a third transformation (E8) transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element (4) are aligned and that a first distance (d) between the output point and said associated point is less than a second distance (D) between the input point and said associated point, in which the ratio between the first distance (d) and the second distance (D) depends on the magnification factor (k).; A method according to claim 1, wherein the first distance (d) is determined by applying an increasing function f to the second distance (D). k dependent on the reduction factor. The method according to claim 2, wherein the increasing function f k has as its image an interval [0 ; Δ k [and the values ​​f k (D) of the increasing function f k tend towards Δ k when D tends towards infinity. Method according to claim 3, wherein the upper bound (Δ k ) of said interval [0 ; Δ k [is increasing as a function of the magnification factor (k). A method according to any one of claims 1 to 4, wherein the second transformation is carried out by means of a homothety with a ratio equal to the magnification factor. A method according to any one of claims 1 to 5, wherein the first transformation transforms an input point into an output point such that the input point, the output point and said associated point are aligned and a third distance (D') between the output point and said associated point is greater than a fourth distance (d') between the input point and said associated point. A method according to any one of claims 1 to 6, comprising a step (E12) of displaying the image on a display device. Electronic device (2) designed to construct an image on the basis of a set of values ​​respectively associated with photosensitive elements (12) of a sensor (6) receiving a light flux through an optical element (4) causing a distortion of the light flux, comprising: - a memory (10) storing, for each photosensitive element, coordinates (X, Y) of the photosensitive element within the sensor (6);- a processor (8) configured to determine, for each photosensitive element, modified coordinates (X3, Y3) on the basis of the coordinates (X, Y) stored in the memory for that photosensitive element, and to construct the image by assigning the value associated with a given photosensitive element (12) to a pixel of the image defined by the modified coordinates (X3, Y3) determined for that given photosensitive element, characterized in that the processor (8) is configured to determine the modified coordinates (X3, Y3) for each photosensitive element (12) on the basis of the coordinates (X, Y) stored in the memory (10) for that photosensitive element by successive application of a plurality of transformations comprising: - a first transformation compensating for the effect of said distortion in the plane of the photosensitive elements (12) of the sensor (6); - a second transformation resulting in an increase in distances by application of a magnification factor (k);- a third transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element (4) are aligned and that a first distance (d) between the output point and said associated point is less than a second distance (D) between the input point and said associated point, in which the ratio between the first distance (d) and the second distance (D) depends on the magnification factor (k).; Electronic device according to claim 8, comprising said sensor (6). Computer program comprising instructions executable by a processor (8) and designed to implement a method according to any one of claims 1 to 7 when these instructions are executed by the processor (8).

Images