Method for constructing an image, and associated electronic device and computer program

Abstract

Disclosed is a method for constructing an image, which method comprises the following steps: - for each pixel of the image, determining (E4, E6, E8) modified coordinates (X3, Y3) on the basis of coordinates (X, Y) of the pixel in the image; - constructing (E10) the image by assigning, to each pixel of the image, the value associated with a photosensitive element defined by the modified coordinates determined for the pixel of the image. The modified coordinates are determined for each pixel on the basis of the coordinates of the pixel by successive application of a plurality of transformations comprising: - a first transformation (E4) increasing distances with respect to a central point; - a second transformation (E6) causing a decrease in distances through application of a reduction factor; - a third transformation (E8) having the effect of a distortion caused by an optical element in the plane of the photosensitive elements. The increase in distances in the first transformation depends on the reduction factor.

Description

Image construction process, associated electronic device and computer program Technical field of the invention

[0001] The present invention relates to the technical field of image acquisition using a sensor.

[0002] It relates in particular to a process for constructing an image, as well as an associated electronic device and computer program. State of the art

[0003] It is known to use electronic devices (e.g., video cameras) that use a sensor with photosensitive elements to acquire an image.

[0004] An optical element, such as a lens, is generally placed in the path of the light flux incident on the sensor so as to image the environment to be observed by the sensor in the plane of the photosensitive elements.

[0005] However, the optical element can create distortion of the light flux incident on the sensor, particularly when the optical element has a reduced focal length to allow observation with a wide field of view.

[0006] Techniques to compensate for this distortion have been proposed, but the resulting image lacks realism due to a distortion of the objects present in the observed scene. Presentation of the invention

[0007] In this context, the present invention proposes a method for constructing an image based on a set of values ​​respectively associated with photosensitive elements of a sensor receiving a light flux through an optical element causing a distortion of the light flux, comprising the following steps:

[0008] - for each pixel of the image, determination of modified coordinates based on the coordinates of that pixel in the image;

[0009] - image construction by assigning to each pixel of the image the value associated with the photosensitive element defined by the modified coordinates determined for that pixel of the image,

[0010] characterized in that the modified coordinates are determined for each pixel of the image based on the coordinates of that pixel in the image by successive application of a plurality of transformations comprising:

[0011] - a first transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element are aligned and that a first distance between the output point and said associated point is (strictly) greater than a second distance between the input point and said associated point;

[0012] - a second transformation resulting in a reduction of distances by applying a reduction factor;

[0013] - a third transformation having the effect of said distortion in the plane of the photosensitive elements of the sensor,

[0014] in which the ratio between the first distance and the second distance depends on the reduction factor.

[0015] The distortion caused by the optical element is corrected by the third transformation. The first transformation, however, corrects the constructed image to avoid distortion of the objects it contains, modulating this effect according to the reduction performed by the second transformation. Indeed, the distortion of objects caused by the third transformation occurs primarily at low image magnification, that is, with a small reduction by the second transformation.

[0016] The first distance can be determined by applying an increasing function f to the second distance. k dependent on the reduction factor.

[0017] The increasing function f k is, for example, defined on an interval [0; Δ k [ ; the values ​​f k (d) of the increasing function f k in this case can tend towards infinity as d tends towards Δ k .

[0018] The reduction factor used can be the inverse of a desired magnification factor (entered for example by the user via a user interface).

[0019] The upper bound Δ k This interval can then be increasing according to the desired magnification factor. Thus, the effect of the function f k (and therefore of the first transformation) will be more important for low magnification factors.

[0020] The second transformation can be carried out using a homothety with a ratio equal to the reduction factor.

[0021] The third transformation can, for example, transform an input point into an output point such that the input point, the output point, and the associated point are aligned, and a third distance between the output point and the associated point is less than (strictly) a fourth distance between the input point and the associated point.

[0022] The process may also include a step of displaying the image on a display device.

[0023] The invention also proposes an electronic device designed to construct an image based on a set of values ​​respectively associated with photosensitive elements of a sensor receiving a light flux through an optical element causing a distortion of the light flux, comprising:

[0024] - a memory storing, for each pixel of the image, the coordinates of the pixel in the image;

[0025] - a processor configured to determine, for each pixel of the image, modified coordinates based on the coordinates stored in memory for that pixel, and to construct the image by assigning, to each pixel of the image, the value associated with the photosensitive element defined by the modified coordinates determined for that pixel of the image,

[0026] characterized in that the processor is configured to determine the modified coordinates for each pixel of the image based on the coordinates stored in memory for that pixel by successively applying a plurality of transformations comprising:

[0027] - a first transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element are aligned and that a first distance between the output point and said associated point is (strictly) greater than a second distance between the input point and said associated point;

[0028] - a second transformation resulting in a reduction of distances by applying a reduction factor;

[0029] - a third transformation having the effect of said distortion in the plane of the photosensitive elements of the sensor,

[0030] in which the ratio between the first distance and the second distance depends on the reduction factor.

[0031] This electronic device may include said sensor (as well as possibly the optical element).

[0032] The invention finally proposes a computer program comprising instructions executable by a processor and designed to implement a process as defined above when these instructions are executed by the processor.

[0033] The optional process characteristics described above can also be applied to this electronic device.

[0034] Of course, the different features, variants and embodiments of the invention can be combined with each other in various ways as long as they are not incompatible or mutually exclusive. Detailed description of the invention

[0035] Furthermore, various other features of the invention become apparent from the attached description made with reference to the drawings which illustrate non-limiting embodiments of the invention and where:

[0036] represents the main elements of an example of an electronic device conforming to the invention;

[0037] is a logic diagram showing an example of a method for constructing an image according to the invention;

[0038] represents functions used in the process of the ; and

[0039] represents the distortion caused by an optical element of the electronic device.

[0040] Lare represents the main elements of an example of an electronic device according to the invention.

[0041] Such an electronic device 2 (for example a video camera) includes an optical element 4, a sensor 6, a processor 8 and a memory 10. Such an electronic device is for example mounted in a vehicle (such as a motor vehicle) in order to acquire an image (or a plurality of images so as to form a video sequence) of the interior (for example of the passenger compartment) of the vehicle or of an environment outside the vehicle.

[0042] The sensor 6 comprises an array of 12 photosensitive elements.

[0043] The optical element 4 (for example, a lens, here a wide-angle and / or short focal length lens, or "fisheye" lens) images the environment observed by the camera in the plane of the photosensitive elements 12 of the sensor 6. The focal length of lens 4 is, for example, between 2 mm and 3 mm, and is 2.5 mm in the example described. The angle of view is, for example, 160° in the horizontal plane and 90° in the vertical plane.

[0044] The photosensitive elements 12 of the sensor thus receive a luminous flux through the optical element 4 which causes (due to the reduced focal length) a distortion of this luminous flux, particularly in the plane of the photosensitive elements 12.

[0045] Each photosensitive element 12 produces a value representative of the intensity of the light flux incident on the photosensitive element 12 concerned.

[0046] These values ​​produced respectively by the photosensitive elements 12 of the sensor 6 are processed by the processor 8 to construct (according to a process described below) an image to be displayed on a display device (not shown).

[0047] The processor 8 (for example a microprocessor) thus implements a process of processing the values ​​produced by the photosensitive elements 12 and of constructing the aforementioned image, as described below with reference to the, due to the execution by this processor 8 of computer program instructions (stored for example in memory 10).

[0048] This is a flowchart showing an example of a method for constructing an image according to the invention.

[0049] As indicated above, this process is implemented here by processor 8 due to the execution by this processor 8 of computer program instructions stored in memory 10.

[0050] We assume here that the user wants a magnification of the image by a factor k (the desired magnification factor being entered by the user for example using a user interface not shown).

[0051] This process begins with a step E2 during which the processor 8 defines a template including, for each pixel of the image to be constructed (or image to be displayed), X, Y coordinates of the pixel in the image and stores these X, Y coordinates in memory 10.

[0052] The X and Y coordinates of a pixel in the image are, for example, two integers that define the horizontal and vertical positions of the pixel in the image (this image being raster). Other coordinate systems can, however, be used as alternatives.

[0053] The X, Y coordinates thus stored (initial coordinates) are then processed (by processor 8) by three successive transformations described below in steps E4 to E8 in order to obtain, for each pixel of the image to be constructed, modified coordinates X3, Y3. In other words, for each pixel of the image to be constructed, the X, Y coordinates associated with this pixel in step E2 undergo successively the three transformations described in steps E4 to E8 in order to obtain modified coordinates X3, Y3 associated with this pixel.

[0054] In step E4, processor 8 applies a first transformation of the X, Y coordinates relative to each pixel of the image to be constructed. This first transformation is a transformation of the plane (that is, a transformation defined in the plane) which transforms input coordinates X, Y (defining an entry point) into output coordinates X1, Y1 (defining an exit point).

[0055] This first transformation turns the input point (with coordinates X, Y) into an output point (with coordinates X1, Y1) defined as follows:

[0056] - the entry point, the exit point and a point associated with the optical axis of the optical element 4 (called the central point in the following) are aligned;

[0057] - The distance D between the exit point and the central point is determined by applying a function f k at the distance d between the entry point and the central point.

[0058] In the example described here, we consider that the image to be constructed is centered on sensor 6 and that sensor 6 and optical element 4 are aligned, so that the point associated with the optical axis of optical element 4 (central point) is located at the center of the image to be constructed.

[0059] The function f kused during the current implementation of the first transformation depends on the desired magnification factor k.

[0060] We have represented on the three functions f kmin , f k , f kmax , where kmin is the minimum usable magnification factor and kmax is the maximum usable magnification factor; therefore, we have kmin < k < kmax.

[0061] As can be seen on the, all the functions f k used have the following properties:

[0062] - f k is increasing on its domain of definition;

[0063] - for all x in its domain, f k (x) > x (therefore, the graph of any function f k is located above the line with equation (D=d) as shown in).

[0064] Thus, the distance D between the exit point and the central point is greater (strictly) than the distance d between the entry point and the central point.

[0065] Furthermore, at the point with abscissa d=0, the graph of any function f k is tangent to the line with equation (D=d), as also seen in. (In other words, the derivative of any function f k (at the point with abscissa d=0, the value is 1.)

[0066] Thus, the first transformation does little to modify the entry points associated with a small distance d, that is to say located close to the central point.

[0067] Furthermore, as can be seen on the, the functions f k are distinct from each other so that, for each possible value of the distance d between the entry point and the central point, the ratio between the distance D = f k (d) and the distance d depends on the factor k.

[0068] Specifically, the ratio between the distance D (distance between the exit point and the central point) and the distance d (distance between the entry point and the central point) decreases when the desired magnification factor k increases.

[0069] In other words, the first transformation causes a significant dilation around the central point when the desired magnification factor is low, but a less significant dilation around the central point when the desired magnification factor is high.

[0070] Furthermore, as also visible in, each function f k is defined on an interval [0; Δ k [and the value f k (x) tends towards infinity as x tends towards Δ k In other words, for each function f k , the (vertical) line with equation (x = Δ k ) is an asymptote to the graph of this function f k .

[0071] For example, the following values ​​are used:

[0072] - Δ k min = r / 2, where r is the radius of the largest circle centered on the central point and inscribed in the image to be constructed (in the coordinate system used in step E2);

[0073] - Δ k = Δ k min .k / kmin for other values ​​of k.

[0074] The upper bound Δ k of the interval of definition [0; Δ k [of the function f k is therefore increasing here as a function of the desired magnification factor k.

[0075] We can use, for example, the family of functions f k defined by:

[0076] f k (d) = (2.Δ k / π) . tan(0.5.π.d / Δ k )

[0077] with here as already indicated Δ k = (rk) / (2.kmin) and tan is the trigonometric function "tangent".

[0078] In practice, the application of the function fk This is achieved, for example, by reading from a lookup table. For this purpose, memory 10 stores, for each of a plurality of ranges of values ​​of k (these ranges of values ​​covering the allowed values ​​of the magnification coefficient k), a lookup table associating distance values ​​d with corresponding distance values ​​f. k (d).

[0079] In this case, the application of the function f k includes selecting a lookup table associated with the coefficient k and reading the distance D associated with the distance d from the selected lookup table.

[0080] Applying the first transformation allows us to pre-compensate for the adverse effects of the distortion correction carried out by means of the third transformation (step E8).

[0081] Indeed, the first transformation tends to move away from the central point the points where the values ​​of the sensor's photosensitive elements will be read and therefore, conversely, to bring the positioning of these photosensitive elements in the image back towards the central point.

[0082] The use of the value Δ k min = r / 2 allows in this respect that with the minimum magnification coefficient, even distant points (particularly due to the distortion correction in step E8) are brought to the edge of the constructed image.

[0083] Processor 8 applies a second transformation of the coordinates (here X1, Y1) relative to each pixel of the image to be constructed at step E6. This second transformation converts the input coordinates (here the X1, Y1 coordinates produced at the output of step E4) into the output coordinates X2, Y2.

[0084] The second transformation results in a reduction of distances by applying the reduction factor 1 / k equal to the inverse of the magnification factor k.

[0085] Indeed, as explained below, it is the modified coordinates (produced at the output of steps E4 to E8) that indicate which photosensitive element must be read to construct (form) the image and a reduction of distances at step E6 therefore allows reading closer photosensitive elements on the sensor, i.e. the desired magnification.

[0086] The second transformation is, for example, carried out in practice by applying a homothety with a ratio of 1 / k centered on the central point defined above.

[0087] Processor 8 applies a third transformation of the coordinates (here X2, Y2) relative to each pixel of the image to be constructed at step E8. This third transformation transforms the input coordinates (here the X2, Y2 coordinates produced at the output of step E6) into the output coordinates X3, Y3.

[0088] The third transformation has the same effect as the distortion caused by the optical element 4 in the plane of the photosensitive elements 12 of the sensor 6.

[0089] As can be seen in, this distortion has the effect of sending a light ray to sensor 6, at a distance d' from the optical axis of optical element 4, which should have reached sensor 6 at a distance D' from the optical axis of optical element 4, with D' > d'.

[0090] The third transformation therefore causes a reduction of distances around the central point (associated with the optical axis of the optical element 4 as already indicated), this reduction of distances corresponding to the effect of distortion in the plane of the photosensitive elements 12.

[0091] The third transformation is a transformation of the plane (i.e., defined in the plane) and transforms the input point (with coordinates X2, Y2) into an output point (with coordinates X3, Y3) defined as follows:

[0092] - the entry point, the exit point and the central point are aligned;

[0093] - the distance d' between the exit point and the central point is determined by applying a function g to the distance D' between the entry point and the central point, the function g representing the effect of the distortion in the plane of the photosensitive elements 12 and being such that d' = g(D') < D'.

[0094] The function g is independent of the desired magnification factor k and the third transformation performed is therefore the same regardless of the desired magnification factor k.

[0095] The application of the function g is, for example, carried out by reading from a lookup table (stored here in memory 10). This lookup table stores a plurality of values ​​g(D') respectively associated with a corresponding plurality of values ​​D' and determined, for example, by means of prior measurements carried out on the optical element 4 concerned (or an optical element of the same type).

[0096] The processor 8 can then construct the image (step E10) by assigning, to each pixel (with coordinates X, Y) of the image, the value produced by the photosensitive element defined by the modified coordinates X3, Y3 determined for that pixel of the image.

[0097] Thanks to the third transformation performed in step E8, processor 8 reads the light beam associated with a pixel at the point where that light beam is incident (on sensor 6), and the distortion is thus compensated. However, the first transformation performed in step E4 partially compensates for this correction, depending on the desired magnification factor (magnification achieved using the second transformation in step E6), in order to avoid excessive distortion of objects in the image, particularly noticeable when there is no magnification or only low magnification.

[0098] The process can then include a step E12 of displaying the constructed image on a display device (not shown).

[0099] The embodiment described above is only one possible example of implementing the invention. As an alternative, the transformations can be carried out in a different order than that presented above, for example, in the following order: second transformation, first transformation, third transformation.

[0100] Furthermore, in the case where the electronic device is a video camera, the processing described above to construct an image is applied to each of the images of the video sequence taken by the video camera.

Claims

Method of constructing an image on the basis of a set of values ​​respectively associated with photosensitive elements (12) of a sensor (6) receiving a light flux through an optical element (4) causing a distortion of the light flux, comprising the following steps:- for each pixel of the image, determination (E4, E6, E8) of modified coordinates (X3, Y3) on the basis of coordinates (X, Y) of the pixel in the image;- construction (E10) of the image by assigning, to each pixel of the image, the value associated with the photosensitive element (12) defined by the modified coordinates (X3, Y3) determined for this pixel of the image, characterized in that the modified coordinates (X3, Y3) are determined for each pixel of the image on the basis of the coordinates (X, Y) of this pixel in the image by successive application of a plurality of transformations comprising: - a first transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element (4) are aligned and that a first distance (D) between the output point and said associated point is greater than a second distance (d) between the input point and said associated point; - a second transformation resulting in a reduction of the distances by application of a reduction factor;- a third transformation having the effect of said distortion in the plane of the photosensitive elements (12) of the sensor (6), in which the ratio between the first distance (D) and the second distance (d) depends on the reduction factor. A method according to claim 1, wherein the first distance (D) is determined by applying an increasing function f to the second distance (d). k dependent on the reduction factor. The method according to claim 2, wherein the increasing function f k is defined on an interval [0 ; Δ k [and the values ​​f k (d) of the increasing function f k tend towards infinity as d tends towards Δ k . A method according to claim 3, wherein the reduction factor is the inverse of a desired magnification factor (k) and wherein the upper bound (Δ k ) of said interval [0 ; Δ k[is increasing as a function of the desired magnification factor (k). A method according to any one of claims 1 to 4, wherein the second transformation is carried out by means of a homothety with a ratio equal to the reduction factor. A method according to any one of claims 1 to 5, wherein the third transformation transforms an input point into an output point such that the input point, the output point and said associated point are aligned and a third distance (d') between the output point and said associated point is less than a fourth distance (D') between the input point and said associated point. A method according to any one of claims 1 to 6, comprising a step (E12) of displaying the image on a display device. An electronic device (2) designed to construct an image based on a set of values ​​respectively associated with photosensitive elements (12) of a sensor (6) receiving a light flux through an optical element (4) causing a distortion of the light flux, comprising: - a memory (10) storing, for each pixel of the image, coordinates (X, Y) of the pixel in the image; - a processor (8) configured to determine, for each pixel of the image, modified coordinates (X3, Y3) based on the coordinates (X, Y) stored in the memory for that pixel, and to construct the image by assigning, to each pixel of the image, the value associated with the photosensitive element (12) defined by the modified coordinates determined for that pixel of the image, characterized in that the processor (8) is configured to determine the modified coordinates (X3, Y3) for each pixel of the image based on the coordinates (X,Y) stored in memory (10) for this pixel by successive application of a plurality of transformations comprising: - a first transformation transforming an input point into an output point and such that the input point, the output point and a point associated with the optical axis of the optical element (4) are aligned and that a first distance (D) between the output point and said associated point is greater than a second distance (d) between the input point and said associated point; - a second transformation resulting in a reduction of the distances by application of a reduction factor; - a third transformation having the effect of said distortion in the plane of the photosensitive elements (12) of the sensor (6), in which the ratio between the first distance (D) and the second distance (d) depends on the reduction factor. Electronic device according to claim 8, comprising said sensor (6). Computer program comprising instructions executable by a processor (8) and designed to implement a method according to any one of claims 1 to 7 when these instructions are executed by the processor (8).

Images