Method for representing a scene on a display medium, computer program, device, apparatus and vehicle implementing such a method

Abstract

The invention relates to a method (100) for representing a scene on a display medium (416) from a stack of images (PILs) of said scene, comprising at least one iteration of a display phase (110), the method comprising the following steps: - detecting (112) a target position of the scene; - identifying (114), in the stack of images (PILs), an image that sharply displays said target position, comprising the following steps for at least one of the other images in the stack (PILs), referred to as candidate image:  calculating (118), by a predetermined coordinate transformation function, the target position in said candidate image, and  when the target position in the candidate image is located in a sharp area of the candidate image, displaying (126) said image on the display medium. The invention also relates to a computer program and a device implementing such a method.

Description

DESCRIPTION Title: Method for representing a scene on a display medium, computer program, device, apparatus and vehicle implementing such a method.

[0001] The present invention relates to a method for representing a scene on a display medium, from a stack of images of said scene. It also relates to a computer program and a device implementing such a method. It further relates to an apparatus and a vehicle implementing such a method.

[0002] The field of the invention is the field of representing a scene on a display medium from images of said scene. State of the art

[0003] There are known techniques for representing a scene from a stack of images taken at different focusing distances. For example, the technique called focus bracketing, or focus stacking, is known, which allows capturing a stack of images of a scene, each image being captured at a different depth of field. The scene is then represented on a display medium, with a greater depth of field, using the stack of images.

[0004] However, current techniques, although improving the depth of field of the image of a scene, have drawbacks. For example, when acquiring the stack of images by a camera module comprising an image sensor and an optical lens, the distance between the image sensor and the optical lens is modified to change the focus, and this for each image. The inventors have noticed that this change in focus causes distortions, or shifts, in the images and therefore degrades the representation of the scene on a display medium from the image stack.

[0005] An aim of the present invention is to remedy at least one of the drawbacks of the state of the art.

[0006] Another aim of the invention is to propose a solution making it possible to improve the representation of a scene from images of said scene captured at different focuses. Statement of the invention

[0007] The invention proposes to achieve at least one of the aforementioned aims by a method of representing a scene on a display medium from a stack of images of said scene, each taken with a different focus so that each image of the stack has a zone of sharpness different from the other images, said method comprising at least one iteration of a display phase comprising the following steps: - detection of a target position of the scene, from an image, called the current image, chosen from said stack of images and currently displayed on said display medium; - identification, in the stack of images, of an image clearly displaying said target position, comprising the following steps for at least one of the other images in the stack, called a candidate image: ■ calculation, by a predetermined coordinate transformation function, of the target position in said candidate image, taking as input the target position in said current image, and ■ when the target position in the candidate image is within a sharpness area of ​​said candidate image, selecting said candidate image as the image to be displayed on said display medium; and - displaying said selected image on said display medium.

[0008] Thus, the method according to the invention proposes to identify, in the stack of images of the scene, an image to be displayed on which the target position is located in a clear zone. When this image to be displayed is identified, it is displayed on the display medium. Thus, for a given target position, the invention makes it possible to choose, on the fly, the image of the scene clearly displaying a region of the scene comprising said target position. In other words, the invention makes it possible to dynamically adjust the clear zone of a scene displayed on the display medium, by using a stack of images each comprising a clear zone of the scene.

[0009] Above all, the invention proposes to identify the image to be displayed, that is to say the image of the stack on which the target position is clear, by correcting the target position obtained on the current image, with a coordinate transformation function. This coordinate transformation function makes it possible to identify more precisely said target position on the other image by taking into account the geometric aberrations induced by a difference in focus between the images of the image stack. Thus, the method according to the invention makes it possible to take into account the geometric aberrations introduced during a change of focus in the imaging module used to capture the image stack, and therefore to improve the choice of the image to be displayed and therefore the display of the scene on a display medium.

[0010] By image we mean a digital image, and in particular a raster image.

[0011] According to the invention, the display medium can be any type of display medium.

[0012] For example, the display medium may be a display screen, such as a touch screen, etc.

[0013] For example, the display medium may be a surface onto which an image of the scene is projected, such as, for example, a display surface associated with a projector, etc.

[0014] The display medium may be a display medium equipping the apparatus implementing the invention, such as: - a display screen fitted to said device, etc. or - a projection surface equipping said device.

[0015] The display medium may be a medium independent of the apparatus implementing the invention, for example a display surface, such as a board, a wall, a curtain, etc., receiving images projected by a projector.

[0016] The target position preferably corresponds to the position, in the scene, to which a user observing the scene pays attention or wishes to pay attention.

[0017] According to embodiments, the target position may be a point position on the current image. In particular, the target position may correspond to a pixel of the current image.

[0018] According to embodiments, the target position may be an area of ​​the current image, called the target area. In this case, the position taken into account during the identification step may be a center of the target area, or a corner of the target area, etc.

[0019] The target position can be a position of a cursor-type pointer, moved by the user, in particular manually.

[0020] The target position can be: - a position touched by a user, or - a position aimed by the user in a contactless manner; for example by a part of the user's body such as a finger, a hand, an eye or eyes, or a face, etc.

[0021] According to embodiments, the method according to the invention may further comprise a step of resetting the selected image, relative to the current image, by an inverse coordinate transformation function, applied to each point of the image to be displayed.

[0022] Thus, the scene objects as they appear on the selected image will be positioned as they appear on the current image. This registration step makes it possible to avoid shifts or distortions that could have been caused by a difference in focus between the current image and the selected image. Indeed, the inventors have noticed that a change in focus induces aberrations resulting in the form of offset or deformation of objects in the scene. The registration step corrects these offsets and deformations, which improves the visual rendering when displaying the scene on a display medium, in particular when displaying the image to be displayed in place of the current image.

[0023] According to embodiments, for at least one other image, the transformation function may be a function: - the focus of the current image, and - the focus of the candidate image; so that a specific transformation function is associated with each focus pair {current image focus; candidate image focus}.

[0024] In this case, for an image stack comprising K images taken at K different focuses, there can be a total number of Kx(Kl) coordinate transformation functions.

[0025] Coordinate transformation functions can be stored in a database, each associated with a focus pair. During the identification step, for each {current image focus; candidate focus} pair, the coordinate transformation function can be read from said database.

[0026] According to embodiments, at least one transformation function can perform a projection of coordinates of the current image into the other image.

[0027] In this case, the coordinate transformation function takes as input the target position in the current image and outputs the target position in the candidate image. In other words, the conversion of the target position is done directly from the current image to the other image.

[0028] According to embodiments, at least one coordinate transformation function may comprise: - a first coordinate transformation function giving the target position in an image format, called reference format, previously defined, as a function of the target position in the current image; and - a second coordinate transformation function giving the target position in the image, as a function of the target position in the reference format.

[0029] In this case, the coordinate transformation function first obtains the target position in the reference format. This intermediate position is then used to obtain the target position in the other image.

[0030] According to non-limiting exemplary embodiments, the reference format may be the format of the image sensor of the camera module used to capture the image stack. In this case, the reference format may correspond to the pixel matrix forming the image sensor.

[0031] Using such a reference format makes it possible to use the same reference for each of the images in the image stack, which reduces errors when identifying the image to be displayed, and therefore improves the representation of the scene on the display medium. Of course, the reference format can be any other previously defined format.

[0032] Preferably, but without loss of generality, at least one coordinate transformation function may be specific to the camera module used for capturing the image stack. Thus, said at least one coordinate transformation function is a function of the aberrations specific to said camera module. This makes it possible to carry out a correction of the aberrations specific to said optical module, and therefore more precise.

[0033] Alternatively, or in addition, at least one coordinate transformation function may be specific to the design, and / or architecture, of the camera module used for capturing the image stack. In this case, said at least one coordinate transformation function is common to several camera modules having the same design, and / or the same architecture. This makes it possible to equip several devices with the same transformation function(s), which is faster to implement.

[0034] Alternatively, or in addition, at least one coordinate transformation function may be specific to the production batch of the camera module used for capturing the image stack. In this case, said at least one coordinate transformation function is common to several camera modules belonging to the same production batch, which allows for faster implementation while providing more precision for aberration correction.

[0035] Alternatively, or in addition, at least one coordinate transformation function may not be specific to a camera module.

[0036] At least one coordinate transformation function can be of any type, such as a mathematical relation.

[0037] According to preferred, but not limiting, embodiments, at least one transformation function may be a position correspondence matrix.

[0038] In this case, the position matching matrix can be indicated the position corresponding to a position given to it as input.

[0039] In particular, the correspondence matrix may have the same dimensions as the image sensor used to acquire the image stack. That is, if the image sensor has (UxV) pixels, at least one coordinate transformation function may be a matrix of size (UxV). For example, when the coordinate transformation function gives the target position directly in the other image (without going through the reference format), each (u,v) bin of the matrix may include values ​​indicating the (u,v) position in the candidate image.

[0040] The first function, respectively the second function, can also be a coordinate transformation matrix, as described above.

[0041] At least one inverse coordinate transformation function can also be a coordinate transformation matrix, as described above.

[0042] At least one coordinate transformation function may be predetermined and provided as input to the present invention, by example during a calibration phase which is not part of the present invention.

[0043] Alternatively, the method according to the invention may further comprise a calibration phase, to determine at least one coordinate transformation function.

[0044] The calibration phase can be carried out before or after production of the camera module.

[0045] The calibration phase can be performed before or after acquisition of the image stack.

[0046] The calibration phase can be carried out in the same device that acquired the stack of images of the scene.

[0047] The calibration phase can be carried out in a device other than the one that acquired the stack of images of the scene.

[0048] Preferably, but in no way limiting, the calibration phase can be carried out with the camera module used for capturing the stack of images of the scene so that at least one transformation function is specific to said camera module.

[0049] According to an exemplary embodiment, at least one coordinate transformation function can be determined by simulation, from a digital model of a camera module.

[0050] For example, it is possible to simulate using simulation software, for example of the Zemax ® type, taking as input a digital model of the camera module, the propagation of a light ray in said camera module. This simulation can be repeated: - for different focuses, and in particular the focuses corresponding to those used when acquiring the stack of images of the scene; and / or - for different points in space. It is then possible to identify by simulation the pixel receiving the light ray, for different points in space and for different focuses, which makes it possible to determine the coordinate transformation function(s).

[0051] The simulation can be carried out not with a point light beam, but with a reference pattern, such as a target. This allows the simulation to be carried out for a multitude of points in space, simultaneously, for a given focus.

[0052] According to an exemplary embodiment, at least one coordinate transformation function can be determined, from a stack of images actually acquired.

[0053] According to one embodiment, the calibration phase may be performed with a stack of calibration images. In this case, the calibration phase may comprise the following steps: - acquisition of a stack of images, called calibration, with at least the same focus as those of the images in the stack of images of the scene, and - deduction from said stack of calibration images of at least one coordinate transformation function.

[0054] The calibration image stack may be a stack of images of a scene. Alternatively, the image stack may be a stack of images of a calibration pattern such as a calibration target.

[0055] During the calibration phase, several images are captured, actually or by simulation, to constitute a stack of reference images, each at one focus among several focuses, such as for example the focuses used during the acquisition of the stack of images of the scene. The reference pattern being known, it is possible to determine, for each point of the reference pattern, the pixel to which it corresponds, and this for each of the focuses used. By knowing, for each point of the reference pattern, the pixel which corresponds to each of the focuses, it is then possible to establish the coordinate transformation functions, for example in the form of correspondence matrices, or in the form of mathematical functions.

[0056] Alternatively, the calibration phase can be performed with the stack of images of the scene, by analyzing the content of said images.

[0057] In this case, the image stack can be analyzed to detect the different objects in the scene on each image of the scene. The pixels corresponding to these objects are then determined, for each image of the scene, which then allows each coordinate transformation function to be deduced, in the form of a correspondence matrix and / or a mathematical function.

[0058] Regardless of the embodiment, the method according to the invention may comprise a storage of at least one coordinate transformation function. This at least one function may be used during one or more identification steps, for one or more image stacks, for one or more scenes.

[0059] In particular, as indicated above, at least one coordinate transformation function may be specific to the camera module used for capturing the image stack. Said at least one transformation function may be used for all image stacks captured with said apparatus within said apparatus. In addition, when an image stack is captured by said apparatus and transmitted to another apparatus, in this case the at least one transformation function may be transmitted to said other apparatus with the image stack.

[0060] According to embodiments, at least one coordinate transformation function can be received from a device other than that implementing the method according to the invention, in particular when the stack of images of the scene has been acquired by said other device.

[0061] In this case, preferably, said at least one transformation function may be specific to the camera module of said other device.

[0062] According to another aspect of the invention, there is provided a computer program comprising executable instructions which, when executed by a computer device, implement all the steps of the method according to the invention.

[0063] The computer program can be in any computer language, such as machine language, C, C++, JAVA, Python, etc.

[0064] According to another aspect of the invention, a device is provided comprising means configured to implement all the steps of the method according to the invention, except the display step.

[0065] The device according to the invention can be, or be integrated into, any type of device such as a smartphone, a tablet, a computer, a calculator, a processor, a computer chip, programmed to implement the method according to the invention, for example by executing the computer program according to the invention.

[0066] According to another aspect of the invention, there is provided an apparatus comprising: - at least one display means for displaying an image on a display medium, - at least one means for detecting a target position on an image displayed on said display medium, and - at least one computing unit; configured to implement the steps of the method according to the invention.

[0067] The device may not include an image acquisition means. In this case, the device is used to display one or more images acquired by another device.

[0068] Alternatively, the apparatus may comprise an image acquisition means, such as a camera module. In this case, the apparatus may be used to display one or more images acquired by said apparatus or by another apparatus.

[0069] The display medium can be any type of display medium.

[0070] For example, the display means may be a display screen, such as a touch screen, etc.

[0071] For example, the display means may be a projector projecting images onto a display surface.

[0072] According to the invention, the means for detecting the target position can be any type of means.

[0073] According to embodiments, the at least one means for detecting the target position may be a cursor-type pointer, designed to be moved by the user, in particular manually.

[0074] In this case, the user can move the cursor position by any type of input device, for example, mouse, trackball, or directional pad, etc.

[0075] According to embodiments, the at least one means for detecting the target position may be one or more sensors arranged to detect: - a position touched by a user, or - a position aimed by the user in a contactless manner; for example by a part of the user's body such as a finger, a hand, an eye or eyes, or a face, etc.

[0076] Such a detection sensor may comprise a detection surface, for example capacitive or resistive.

[0077] Such a detection sensor can be one or more cameras.

[0078] The at least one detection means may be arranged in / on / under the display means.

[0079] The at least one detection means may comprise at least one orientation detection means, such as a gyroscopic type sensor

[0080] In particular, the device may be a user device such as a smartphone, tablet, etc. comprising a display screen. In this case, the detection means may be, or may comprise, a touch surface, in particular integrated into, or associated with, the display screen of said apparatus.

[0081] In particular, the device may be a computer-type user device, comprising a display screen. In this case, the detection means may be, or may comprise, a touch-sensitive surface, in particular integrated into, or associated with, the display screen of said computer, or a pointer moved for example by a mouse, or a directional pad of said computer.

[0082] In particular, the device may be a television. In this case, the detection means may be a camera integrated into said television, detecting the gaze and the position of the head of the observer, or a pointer moved for example by a remote control of said television.

[0083] In particular, the device may be a virtual reality or augmented reality headset comprising a display screen or a projector associated with a projection surface onto which each image is projected. In this case, the detection means may be, or may comprise, a sensor, in particular an optical sensor, equipping said headset.

[0084] In particular, the device may be a medical imaging device.

[0085] In particular, the device may be an endoscope, an ultrasound device, etc.

[0086] Of course, the invention is not limited to the examples just given.

[0087] According to another aspect of the present invention, there is provided a vehicle comprising: - at least one display means for displaying an image on a display medium, and - at least one means for detecting a target position on an image displayed on said display medium, and - at least one computing unit; configured to implement the method according to the invention.

[0088] The vehicle according to the invention may not include an image acquisition means. In this case, the vehicle according to the invention is used to display one or more images acquired by another device.

[0089] Alternatively, the vehicle according to the invention may comprise an image acquisition means, such as a camera module. In this case, the vehicle may be used to display one or more images acquired by said vehicle or by another device.

[0090] The display medium can be any type of display medium.

[0091] For example, the display means may be a display screen, such as a touch screen, etc.

[0092] For example, the display means may be a projector projecting images onto a display surface.

[0093] According to the invention, the means for detecting the target position can be any type of means.

[0094] According to embodiments, the at least one means for detecting the target position may be a cursor-type pointer, designed to be moved by the user, in particular manually.

[0095] In this case, the user can move the cursor position by any type of input device, for example, mouse, trackball, or directional pad, etc.

[0096] According to embodiments, the at least one means for detecting the target position may be one or more sensors arranged to detect: - a position touched by a user, or - a position targeted by the user in a contactless manner; for example by a part of the user's body such as a finger, a hand, an eye or eyes, or a face, etc.

[0097] Such a detection sensor may comprise a detection surface, for example capacitive or resistive.

[0098] Such a detection sensor can be one or more cameras.

[0099] The at least one detection means may be arranged in / on / under the display means.

[0100] The at least one detection means may comprise at least one orientation detection means, such as a gyroscopic type sensor

[0101] According to embodiments, the vehicle may be a land vehicle, such as a car, autonomous or not.

[0102] According to embodiments, the vehicle may be a flying vehicle, such as a drone, an airplane, a helicopter, autonomous or not.

[0103] According to embodiments, the vehicle may be a maritime vehicle, such as a boat or a submarine, autonomous or not. Description of figures and embodiments

[0104] Other advantages and characteristics will appear on examining the detailed description of non-limiting embodiments, and the attached drawings in which: - FIGURES 1-3 are schematic representations of non-limiting exemplary embodiments of a method according to the invention; - FIGURE 4 is a schematic representation of a non-limiting exemplary embodiment of a device according to the invention; - FIGURES 5a-5c are schematic representations of non-limiting exemplary embodiments of an apparatus according to the invention; and - FIGURE 6 is a schematic representation of a non-limiting exemplary embodiment of a vehicle according to the invention.

[0105] It is understood that the embodiments which will be described below are in no way limiting. In particular, we can imagine variants of the invention comprising only a selection of features described below isolated from the other features described, if this selection of features is sufficient to confer a technical advantage or to differentiate the invention from the state of the prior art. This selection includes at least one preferably functional feature without structural details, or with only a part of the structural details if it is this part which is only sufficient to confer a technical advantage or to differentiate the invention from the state of the prior art.

[0106] In particular, all the variants and embodiments described can be combined with each other if there is no technical obstacle to this combination.

[0107] In the figures and in the rest of the description, the elements common to several figures retain the same reference.

[0108] FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of a method according to the present invention.

[0109] The method 100 of FIGURE 1 may be used to represent a scene on a display medium, such as a display screen or display surface.

[0110] The method 100 takes as input a stack of images of the scene, denoted PILs. This stack of PILs images can be captured by the apparatus implementing the method 100 or another apparatus. The stack of PILs images comprises several images of the scene, each captured with a different focus so that each image of the PILs stack has a different area of ​​sharpness for said scene. In other words, the scene is represented on each of the images with a different area of ​​sharpness. [YES] In a step 102 an image, selected from the image stack, is displayed on the display medium. This step 102 may be part of the method 100. Alternatively, step 102 may not be part of the method 100.

[0112] The image displayed during step 102 is called “current image”, for the first iteration of a display phase 110 of the method 100.

[0113] The purpose of the display phase 110 is to produce a representation of the scene, as a function of a target position. The target position preferably corresponds to the position, in the scene, to which a user observing the scene pays attention or wishes to pay attention. The display phase 104 can be repeated as many times as desired to produce a representation of the scene which changes and follows the position to which the user is interested, in the scene represented on the display medium.

[0114] The display phase 110 comprises a step 112 of detecting the target position POSc on the current image.

[0115] The target position POSc may be a point position on the current image. In particular, the target position POSc may correspond to a pixel of the image. Alternatively, the target position POSc may be an area of ​​the current image. For example, the target position POSc may correspond to a target area. In this case, the coordinates of the target area POSc taken into account for the rest of the method may be the coordinates of a center of the target area, or a corner of the target area, etc.

[0116] The target position POSc may be a position of a pointer, moved by the user, in particular manually. In this case, the target position POSc may be detected by detecting the position of said pointer.

[0117] The POSc target position can be: - a position touched by a user, or - a position targeted by the user in a contactless manner; for example by a part of the user's body such as a finger, a hand, an eye or eyes, or a face, etc. In this case, the target position POSc can be detected by at least one sensor, such as a capacitive sensor, a resistive sensor, a camera, etc.

[0118] The display phase 110 comprises a step 114 of identifying, in the stack of PILs images, the image clearly displaying the scene at the target position identified during step 112. This step 114 is carried out individually, in turn or simultaneously, for an image in the stack of PILs images (other than the current image), until identifying the one which clearly represents the scene at the target position POSc.

[0119] Step 114 includes a step 116 of selecting, as a candidate image, an image in the PILs of images, other than the current image.

[0120] In a step 118, the target position in the candidate image is calculated with at least one coordinate transformation function taking as input the target position in the current image. It should be noted that the coordinate transformation function depends on the focus of the current image, and the focus of the candidate image.

[0121] The data transformation function can be any mathematical relation. In the following, and without loss of generality, it is considered that the coordinate transformation function is a position correspondence matrix.

[0122] In the following, and without loss of generality, it is considered that the coordinate transformation function gives the target position directly in the candidate image. Alternatively, the coordinate transformation function may include: - a first transformation function giving an intermediate target position in a reference format, from the target position POSc in the current image; and - a second position transformation function giving the target position, in the candidate image, from the intermediate target position.

[0123] Then, in a step 120, it is determined whether the calculated target position in the candidate image is located in a sharpness zone of said candidate image.

[0124] To do this, the calculated target position can be compared with a sharpness area of ​​the candidate image. In this case, the sharpness area can be stored with the candidate image.

[0125] Alternatively, the scene depth at the calculated target position can be compared to a focus, or focus distance stored with the candidate image. In this case, the scene depth at the calculated position can be provided in the candidate image, by example stored in the candidate image pixel data corresponding to the calculated target position.

[0126] Of course, this comparison can be carried out using other techniques and the examples given above are in no way limiting.

[0127] If the calculated target position is in a sharp area on the candidate image, then the candidate image is selected as the image to be displayed in a step 122.

[0128] Otherwise, the identification step 114 is carried out for another candidate image selected in the stack of PILs images (other than the current image), and so on, until the image to be displayed is identified in the stack of PILs images.

[0129] The display phase 110 may optionally comprise an optional step 124 of resetting the image to be displayed on the current image by a coordinate transformation function inverse to the coordinate transformation function used in step 118. This optional resetting step 120 aims to adjust the image to be displayed on the current image and thus avoid a shift, or a jump, when displaying the image to be displayed in place of the current image.

[0130] The inverse coordinate transformation function can, for example, be a correspondence matrix giving, for each point in the image to be displayed, the corresponding point in the current image. The image to be displayed is then the realigned image.

[0131] The display phase 110 further comprises a step 126 of displaying the image to be displayed, possibly after resetting said image.

[0132] In the example of FIGURE 1, the at least one coordinate transformation function is not calculated. Indeed, the at least one transformation function may have been previously calculated. Alternatively, the at least one transformation function may have been provided with the PILs image stack.

[0133] FIGURE 2 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0134] The method 200 of FIGURE 2 may be used to represent a scene on a display medium, which may be, for example, a display screen or a display surface.

[0135] The method 200 of FIGURE 2 includes all of the steps of the method 100 of FIGURE 1.

[0136] The method 200 further comprises a phase 202, called the calibration phase, for calculating at least one coordinate transformation function.

[0137] The calibration phase 202 can be carried out by simulation from a digital model of a camera module. Alternatively, the calibration phase 202 can be carried out from images taken by a real camera module.

[0138] The calibration phase 202 can be performed with the camera module used for acquiring images of the scene. Alternatively, the calibration phase 202 can be performed with a camera module other than the one used for acquiring the stack of images of the scene. In this case, the other camera module can be a camera module having the same design as the camera module used for acquiring the stack of images of the scene, and / or a camera module produced in the same manufacturing batch as the camera module used for acquiring the stack of images of the scene.

[0139] The calibration phase 202 can be carried out with the stack of images of the PILs scene or a stack of images of another scene, or even a stack of images of a reference scene, such as for example a stack of images of a test pattern.

[0140] In the example described in FIGURE 2, it is considered, without loss of generality, that the calibration phase 202 is carried out, before the acquisition of the PILs image stack, with a reference scene. The reference scene can be any type of scene. Preferably, the reference scene is a target having a known pattern, and arranged at a given distance.

[0141] Furthermore, it is considered that the calibration phase 202 is carried out with the same camera module as that used for the acquisition of the stack of images of the PILs scene, or another same camera module having the same design and forming part of the same manufacturing batch as that used for the acquisition of the stack of images of the PILs scene.

[0142] The calibration phase 202 comprises a step 204 of acquiring a reference PILR image stack. Preferably, the PILR image stack comprises images taken with at least the same focus as those of the images in the image stack of the PILs scene. For example, the PILR image stack may be acquired by a focus bracketing technique.

[0143] Next, a step 206 calculates a coordinate transformation function between each image in the PILR image stack and each of the other images in the PILR image stack.

[0144] To do this, each image in the PILR image stack is considered the starting image. For each starting image, each of the other images in the PILR image stack is considered the ending image, in turn.

[0145] For each pair {starting image, ending image}, each point of the target is identified on the starting image and the ending image. By knowing the position of each point of the target on said starting and ending images, it is then possible to determine the coordinate transformation function giving for each position of the starting image, the corresponding position on the ending image.

[0146] In a step 208, the coordinate transformation function for each pair {starting image, ending image} is stored in association with the focus pair {focus of the starting image; focus of the ending image}. Thus, this coordinate transformation function can be used to perform a coordinate transformation between an image taken at the starting focus and an image taken at the ending focus, regardless of the scene.

[0147] Thus, if the image stack PILR comprises K images, step 206 provides (Kx(Kl)) coordinate transformation functions.

[0148] In a step 208, each coordinate transformation function is stored in association with the focus pair corresponding to the starting image and the arrival image focus.

[0149] As indicated above, the coordinate transformation functions obtained during the calibration phase 202 can be used, for one or more iterations of the display phase, and this for the representation of the same scene, or of different scenes.

[0150] FIGURE 3 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0151] The method 300 of FIGURE 3 may be used to represent a scene on a display medium, which may be, for example, a display screen or a display surface.

[0152] The method 300 of FIGURE 3 includes all of the steps of the method 200 of FIGURE 2.

[0153] The method 300 further comprises a step 302 of acquiring the stack of images of the PILs scene, each having a different focus so that each image has a different area of ​​sharpness. The stack of PILs images can be acquired using any known technique, for example by focus bracketing.

[0001] FIGURE 4 is a schematic representation of a non-limiting exemplary embodiment of a device according to the present invention.

[0002] The device 400 of FIGURE 4 comprises a module 402 implementing the display phase of the method according to the invention, and in particular the display phase 102.

[0003] The module 402 comprises a module 404 displaying an image of the scene given to it as input, on a display medium. This module is in particular configured / programmed to carry out step 126 of the methods 100, 200 and 300.

[0004] The module 402 further comprises a module 406 for detecting a target position on a currently displayed image. This module 406 takes as input a position of a pointer or of a part of the user's body, and determines the target position on the current image. This module 406 is in particular configured / programmed to carry out step 112 of the methods 100, 200 and 300.

[0005] The module 402 further comprises a module 408 for identifying, in the stack of images of the scene PILs, the image to be displayed as a function of the target position determined on the current image. This module 408 is in particular configured / programmed to carry out step 114 of the methods 100, 200 and 300.

[0006] The device 400 may further comprise an optional module 410 for resetting the image to be displayed on the current image. This module 410 is in particular configured / programmed to carry out step 124 of the methods 100, 200 and 300.

[0007] The device 400 may further comprise an optional calibration module 412 for determining the coordinate transformation functions. This module 412 is in particular configured / programmed to carry out phase 202 of the methods 200 and 300.

[0008] The device 400 may further comprise an optional module 414 for acquiring a stack of images of a scene, by cooperating with a camera module comprising an image sensor and an optical lens. This module 414 is in particular configured / programmed to carry out step 302 of the method 300. The optional module 414 may for example be a photo application.

[0009] At least one of these modules may be a module independent of the others.

[0010] At least two of these modules can be integrated within the same module.

[0011] At least one of these modules may be a hardware module.

[0012] At least one of these modules may be a software module, such as a computer program.

[0013] At least one of these modules may be a combination of at least one software module, such as a computer program, and at least one hardware module.

[0014] In particular, at least one of these modules can be integrated into an electronic chip, or even into an application installed in a user device.

[0015] The device 400 may further comprise, optionally, at least one display means 416, such as a display screen, touch-sensitive or not, or a means for projecting an image onto a support, for displaying an image of the scene.

[0016] Such a display means may be integrated into the device.

[0017] Such a display means is optional because the device may not include such a means. For example, the device 400 may be integrated into an apparatus that already has a display means and cooperate with said display means to display the image of the scene.

[0018] According to yet another alternative, the device 400 can be connected to an external display means, or to an external device having a display means or itself connected to a display means.

[0019] In the example shown, and without loss of generality, the display means may be an electronic display screen.

[0020] Optionally, the device 400 may further comprise at least one image acquisition means 418, such as a camera, or a camera module, comprising an optical lens and an image sensor, for acquiring an image or a stack of images of the scene.

[0021] Such an image acquisition means 418 may be integrated into the device, for example on a front face or on a rear face, or both.

[0022] According to yet another alternative, the device 418 can be connected to an external image acquisition means, or to an external apparatus having an image acquisition means 814 or itself connected to an image acquisition means 814.

[0023] The device 400 may further comprise, optionally, at least one sensor 420 for detecting: - the position of a pointer, - the contact position of a part of the user's body such as a hand or finger, or - the position targeted by a part of the user's body such as a hand, finger, eye, eyes or the user's face; to provide, or for the purpose of determining, the target position on a current image displayed on a display medium.

[0024] The sensor 420 can be any type of sensor such as a camera, a lidar, a detection surface for example of the capacitive type, etc.

[0025] In the example shown in FIGURE 4, and without loss of generality, the sensor may be in the form of a detection surface, of the capacitive type, integrated into the display means 416.

[0026] Such a sensor 420 is optional because the device may not include such a sensor. For example, the device 400 may be integrated into an apparatus that already has a sensor.

[0027] FIGURE 5a is a schematic representation of a non-limiting exemplary embodiment of an apparatus according to the present invention.

[0028] The apparatus 510 of FIGURE 5a comprises means configured to implement the invention, and in particular any one of the methods 100, 200 and 300.

[0029] The apparatus 510 of FIGURE 5a may comprise a device according to the invention, and in particular the device 400 of FIGURE 4.

[0030] In the example shown in FIGURE 5a, the device 510 is a smartphone, or a tablet, comprising the device 400 of FIGURE 4. In particular, the device 510 comprises a display screen 416 equipped with a detection surface 420, for example capacitive, and at least one camera 418.

[0031] FIGURE 5b is a schematic representation of another non-limiting exemplary embodiment of an apparatus according to the present invention.

[0032] The apparatus 520 of FIGURE 5b comprises means configured to implement the invention, and in particular any one of the methods 100, 200 and 300.

[0033] The apparatus 520 of FIGURE 5b may comprise a device according to the invention, and in particular the device 400 of FIGURE 4, without the camera 418.

[0034] In the example shown in FIGURE 5b, the apparatus 520 is a virtual reality, VR, headset, or an augmented reality headset, comprising the device 400 of FIGURE 4. In particular, the headset 520 comprises a display screen 416, a sensor (not visible in FIGURE 5b) for detecting the position aimed by an eye, or eyes, of the user on said display screen 416.

[0035] In the example shown in FIGURE 5b, the headset 520 does not include imaging means for capturing images of the scene. In this case, the image(s) of the scene to be displayed by the headset 520 are provided by another device to said headset 520.

[0036] Alternatively, the headset 520 may comprise at least one camera for capturing images of the scene in which it is located to display them on the screen 416, optionally after enriching said images, for example in the context of an augmented reality application.

[0037] FIGURE 5c is a schematic representation of a non-limiting exemplary embodiment of an apparatus according to the present invention.

[0038] The apparatus 530 of FIGURE 5c comprises means configured to implement the invention, and in particular any one of the methods 100, 200, and 300.

[0039] The apparatus 530 of FIGURE 5c may comprise a device according to the invention, and in particular the device 400 of FIGURE 4.

[0040] In the example shown in FIGURE 5c, the apparatus is a medical imaging apparatus, such as an endoscope, an ultrasound apparatus, etc. comprising the device 400 of FIGURE 4. In particular, the apparatus medical imaging device 530 comprises a display screen 416 equipped with a detection surface 420, for example capacitive. The medical imaging device 530 further comprises an imaging means formed by a distal objective connected to an imaging module (not shown).

[0041] FIGURE 6 is a schematic representation of a non-limiting exemplary embodiment of a vehicle according to the present invention.

[0042] The vehicle 600 of FIGURE 6 comprises means configured to implement the invention, and in particular any one of the methods 100, 200 and 300.

[0043] The vehicle 600 of FIGURE 6 may comprise a device according to the invention, and in particular the device 400 of FIGURE 4.

[0044] In the example shown in FIGURE 6, the vehicle 600 is a land vehicle, in particular a car, comprising the device 400 of FIGURE 4. In particular, the vehicle 600 comprises a display screen 416 equipped with a detection surface 420, for example capacitive, arranged in the passenger compartment of the vehicle 600. The vehicle 600 further comprises at least one camera 418, for example arranged on the windshield of the vehicle 600.

[0154] Of course, the invention is not limited to the examples which have just been described.

Claims

CLAIMS 1. Method (100; 200; 300) for representing a scene on a display medium (416) from an image stack (PIL S ) of said scene, each taken with a different focus so that each image in the stack (PIL S ) has a different sharpness zone from the other images, said method (100;200;300) comprising at least one iteration of a display phase (110) comprising the following steps: - detection (112) of a target position of the scene, from an image, called current image, chosen from said image stack (PIL S ) and currently displayed on said display medium (416); - identification (114), in the image stack (PIL S ), of an image clearly displaying said target position, comprising the following steps for at least one of the other images in the stack (PILs), called candidate image: ■ calculation (118), by a predetermined coordinate transformation function, of the target position in said candidate image, taking as input the target position in said current image, and ■ when the target position in the candidate image is within a sharpness area of ​​said candidate image, selecting (116) said candidate image as an image to be displayed on said display medium (416); and - displaying (126) said selected image on said display medium (416).

2. Method (100; 200; 300) according to the preceding claim, characterized in that it further comprises a step (124) of resetting the selected image, relative to the current image, by an inverse coordinate transformation function, applied to each point of the image to be displayed.

3. Method (100; 200; 300) according to any one of the preceding claims, characterized in that, for at least one candidate image, the transformation function is a function of: - the focus of the current image, and - the focus of the candidate image; so that a coordinate transformation function is associated with each focus pair {current image focus; candidate image focus}.

4. Method (100; 200; 300) according to any one of the preceding claims, characterized in that at least one coordinate transformation function comprises: - a first coordinate transformation function giving the target position in an image format, called reference format, previously defined, as a function of the target position in the current image; and - a second coordinate transformation function giving the target position in the candidate image, as a function of the target position in the reference format.

5. Method (100; 200; 300) according to any one of the preceding claims, characterized in that at least one transformation function is a position correspondence matrix.

6. Method (200; 300) according to any one of the preceding claims, characterized in that it further comprises, prior to the first iteration of the display phase (110), a phase (202), called the calibration phase, for determining at least one coordinate transformation function.

7. Method (200; 300) according to any one of the preceding claims, characterized in that the calibration phase (202) is carried out with the same camera module as that used for capturing the stack of images of the scene (PIL S) so that at least one transformation function is specific to said camera module.

8. Method (200; 300) according to any one of the preceding claims, characterized in that the calibration phase (202) comprises the following steps: - acquisition (204) of a stack of images (PILR), called calibration, with at least the same focus as those of the images of the stack of images of the scene (PILs), and - deduction (206) from said calibration image stack (PILR) of at least one coordinate transformation function.

9. Method (200; 300) according to any one of claims 1 to 7, characterized in that the calibration phase (202) is carried out with the stack of images of the scene (PILs), by analyzing the content of said images.

10. Method (100; 200; 300) according to any one of claims 1 to 5, characterized in that at least one coordinate transformation function is received from a device other than that implementing said method, and in particular when the stack of images of the scene (PILs) has been acquired by said other device.

11. Computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the method (100;200;300) according to any one of the preceding claims.

12. Processing device (400) comprising means configured to implement all the steps of the method (100; 200; 300) according to any one of claims 1 to 10.

13. Apparatus (510;520;530) comprising: - at least one display means (404) for displaying an image on a display medium (416), - at least one means (406) for detecting a target position on an image displayed on said display medium, and - at least one computing unit; configured to implement the method (100;200;300) according to any one of claims 1 to 10.

14. Apparatus (510; 520; 530) according to the preceding claim, characterized in that it is: - a smartphone (510), - a tablet - a computer, - a television, - a virtual reality or augmented reality headset (520), or - a medical imaging device (530).

15. Vehicle (600) comprising: - at least one display means (404) for displaying an image on a display medium (416), and - at least one means (406) for detecting a target position on an image displayed on said display medium, and - at least one computing unit; configured to implement the method (100; 200; 300) according to any one of claims 1 to 10.

Images