Method and apparatus for personalized display of scene based on one or more images of said scene

Abstract

The invention relates to a method (100) for presenting a scene on a display medium, said method (100) comprising at least one iteration of a display phase (102) comprising the following steps:-displaying (104) on the display medium an image (IC) of said scene having a clear area, referred to as the current image,-detecting (106) in the current image (IC) a position of interest for a user,-displaying (104) the image (IC) of said scene on the display medium, the position is referred to as a target position,-obtaining (108) a new image (IN) of the scene according to which a clear area of the new image (IN) is adjusted in order to create a clear display of the scene at the target position, and-displaying (110) the new image (IN). The invention also relates to a computer program, a device, a device and a vehicle implementing such a method.

Description

[0001] The present invention relates to an image display method allowing a personalized display of a scene on a display medium. The present invention also relates to a computer program, a device and an apparatus implementing such a method.

[0002] The field of the invention is the field of displaying digital images on a display medium such as a display screen or a projection surface. BACKGROUND

[0003] When a human eye scans a scene, it observes the scene with different focal points to perceive the whole scene. The focal point used to scan the scene depends directly on the depth of the objects in the scene. Generally, when a human eye observes a part of a scene, it adapts the focal point to the depth of this part in the scene to obtain a clear perception of the scene in this area.

[0004] However, the currently known methods for displaying an image of a scene display said image according to a single focal point, i.e. the focal point used when capturing said image. When the scene comprises parts of different depths, the focal point corresponds to one of said depths, and thus only one part of the scene is clear in the displayed image.

[0005] Such a display is very limited in the depiction of the scene.

[0006] Moreover, such a display does not at all adapt to the wishes of the user and limits the user to the choice made when obtaining the image of the scene to observe the scene.

[0007] An object of the present invention is to overcome at least one of the drawbacks of the prior art.

[0008] Another object of the present invention is to provide a solution for displaying an image of a scene that offers a greater degree of freedom to the viewer.

[0009] Another object of the present invention is to provide a solution for displaying an image of a scene that allows a richer depiction of the scene.

[0010] Another object of the present invention is to provide a solution for displaying an image of a scene that allows a greater degree of personalization of the display. SUMMARY

[0011] The present invention proposes to achieve at least one of the aforementioned objects by a method for displaying a scene on a display medium, said method comprising at least one iteration of a display phase, the display phase comprising the following steps:

[0012] - displaying on said display medium an image of said scene having a clear area, called current image,

[0013] - detecting in the current image a position of interest of the user, this position being called target position,

[0014] - obtaining a new image of the scene, the sharp area of this new image being adjusted according to the target position, so as to create a sharp display of the scene at this target position, and

[0015] - displaying this new image.

[0016] Thus, conventionally, the current image displayed on the display medium comprises a sharp area corresponding to a given depth of field: all the other areas of the current image are blurred. The method according to the invention proposes to determine a target position of interest of the user observing the current image: this target position corresponds to a target area of interest of the observer in the scene. Then, the method according to the invention proposes to modify the display so that said target area becomes sharp.

[0017] In this way, the scene display is customized and adapted to the observation of the user, so that the target area observed by the user is always sharp. In other words, the sharp area of the scene displayed on the screen follows the line of sight of the user. Unlike the prior art, the depth of field of the image of the scene is not imposed to the observer. On the contrary, according to the invention, the depth of field is adapted and adjusted according to the observer.

[0018] According to the invention, the display medium can be any type of display medium.

[0019] For example, the display medium can be a display screen, such as the screen of a smartphone, a tablet, a computer, a television, etc.

[0020] For example, the display medium can be a surface onto which the image of the scene is projected, such as a display surface associated with a projector, etc.

[0021] For example, the display medium can be a medium worn by the user, such as goggles or glasses, a virtual reality headset, an augmented reality headset, etc.

[0022] The display medium can be a display medium mounted on the device performing the display phase, such as:

[0023] - a display screen on the device, such as in the case where the device is a smartphone, a tablet, etc., or

[0024] - a projection surface on the device, such as in the case where the device is an augmented reality or virtual reality headset.

[0025] The display medium can be a medium independent of the device performing the display phase, such as part of a projector projecting the image onto a display surface.

[0026] According to an embodiment, the target position can be a point position on the current image.

[0027] According to some embodiments, the target position can be a region of the current image. For example, the target position can correspond to a target region.

[0028] According to some embodiments, the step of obtaining a new image can comprise generating the new image by computation.

[0029] In practice, the new image can be generated on the fly as soon as the target position is determined. In this case, the new image does not exist until the target position is known. The new image is generated as soon as the target position is known.

[0030] The new image can be generated from the current image.

[0031] For example, the new image can be generated by blurring the current sharp regions of the current image and making sharp the target region comprising or corresponding to the target position.

[0032] The current image can be an image captured by the same device as the device performing the display stage.

[0033] Alternatively, the current image can be an image captured by another device and sent to the device performing the display stage.

[0034] The new image can be generated from a source image that is not the current image.

[0035] For example, the source image can be an image previously obtained in which the scene is displayed with a sharp focus in all regions. In this case, a copy of the source image is made. The target region is kept sharp and all other regions are blurred to obtain the new image.

[0036] Such a source image can be obtained by taking several images of the scene with different depth of field, for example using a focus stacking technique. Then, the sharp regions of the scene are selected in each image. All the sharp regions obtained from all the images can be concatenated to obtain an image of the scene in which all regions are sharp.

[0037] Such a source image can also be obtained from a scene image by simulation or computation. For example, by using the transfer function of the optical lens used to acquire the image, it is possible to de-focus said image to restore full sharpness of all points. This can be achieved, for example, by inverting the optical transfer function (deconvolution).

[0038] The method according to the invention can comprise a step of obtaining such a source image before the first iteration of the display stage.

[0039] The step of obtaining the source image can be performed by the same device as the device performing the display stage.

[0040] Alternatively, the step of obtaining the source image can be performed by a device different from the device performing the display stage. In this case, the source image is transmitted to the device performing the display stage.

[0041] According to some embodiments, the step of obtaining the new image can comprise a step of selecting the new image from a stack of images of the scene, each image of the stack having a different sharp area.

[0042] In this case, the new image is not generated on the fly. It already exists before the target location is identified. This embodiment provides a higher responsiveness to the method according to the application.

[0043] In this case, the step of obtaining can comprise identifying the image of the stack in which the target location is displayed in sharp focus. This image is selected as the new image to be displayed in the display step.

[0044] The new image of the stack of images can be identified in various ways. For example, each image of the stack can be associated with information indicating the sharp area in this image. In this case, the step of identifying comprises identifying which sharp area includes the target location and selecting the image associated with this sharp area.

[0045] According to some embodiments, the stack of images can comprise at least one image captured by an imaging device, for example a camera.

[0046] In particular, the stack of images can have been captured by one or more cameras using a focus surround technique to provide a plurality of images of the scene, each image having a different depth of field and thus a different sharp area.

[0047] According to some embodiments, the stack of images can comprise at least one image obtained from at least one other image by calculation or simulation.

[0048] The other image can be an image having a sharp area and a blurred area. In this case, the computation of the image from the stack, including the generation of an image having a different depth of field than the other image. The sharp area of the other image is blurred, while another area of the scene is sharpened. To this end, the computation includes determining the transfer function of the original optics as much as possible based on the defocus case, in determining the transfer function, it is desirable to simulate according to the distance of the point to be displayed, i.e. based on the simulated "focus" distance and the distance of the target area. Typically, before applying the parametric refocusing, it can be tempting to restore the full sharpness of all points by inverting the optical transfer function (deconvolution). In order to minimize artifacts related to noise and modeling errors and the amount of computation needed to control these effects, the aim here is to only compute an incomplete deconvolution, restoring the transfer function of the original optics to a transfer function of the desired simulation that is also incomplete sharp in most areas of the image.

[0049] The other image can also be an image of the scene that is sharp in all areas, for example the source image as described above.

[0050] The method according to the application can comprise, before the first iteration of the display phase, a step of obtaining such a stack of images.

[0051] The step of obtaining the stack of images can be performed by the same device as the device implementing the display phase.

[0052] Alternatively, the step of obtaining the stack of images can be performed by a device different from the device implementing the display phase. In this case, the stack of images is transmitted to the device performing the display phase.

[0053] The new image can be selected in several ways.

[0054] According to some embodiments, after identifying the target position, the display phase can further comprise a step for identifying an object located at the target position. In this case, the new image can be obtained based on said object.

[0055] In particular, selecting the new image can comprise identifying the image on which said object is displayed sharply. To this end, each image in the stack can be associated with information indicating the object displayed in sharp focus in that image.

[0056] According to some embodiments, after identifying the target position, the display phase can comprise a step of identifying a region of the scene, called target region, which includes the target position. In this case, the new image can be obtained based on said target region.

[0057] In particular, the selection of the new image can comprise identifying an image on which said target region is sharply displayed. To this end, each image of the stack can be associated with information indicative of the regions that are sharply displayed in said image.

[0058] According to some embodiments, the display stage can comprise a step of determining depth data of the scene at said target location, the new image being obtained on the basis of said depth data.

[0059] Once the target location has been identified, the depth of the scene at this location can be determined. The scene depth information at the target location can be computed on the fly, for example by triangulation with another image of the scene. Alternatively, the scene depth information at the target location can have been measured by suitable means (e.g. time-of-flight camera, lidar, etc.) when the scene was imaged.

[0060] In the obtaining step, the depth information can be used to select a new image from the stack of images: in this case, an image whose depth of field corresponds to or includes the depth of the target location is identified. To this end, each image of the stack can be associated with information indicative of the depth of field of said image itself.

[0061] Alternatively, the depth information of the target location can be used to generate the new image on the fly from the current image or from a source image (e.g. the source image described above), by considering a calculation of the target location depth. In this case, a linear filter convolution of the image data is applied, the kernel of which reproduces the type of blur required (bokeh, Gaussian type): the kernel can be parameterized to control the intensity of the effect. In the case of Gaussian type, the formula is parameterized by a sigma width:

[0062]

[0063] The sigma width actually used in a point of the scene increases as the difference between the focal length to be simulated and the distance of the object to be imaged increases.

[0064] The target location of interest to the user can be detected in various ways.

[0065] According to some embodiments, the target location can be manually entered by the user.

[0066] In this case, the user manually selects the target location on the current image, for example using a pointer (such as a mouse, trackball or directional pad) to move a cursor, or any other position input device. The user can also enter the target location by making a tactile selection on a touch-sensitive surface arranged on or offset with respect to the display medium, for example in the case of a touch-sensitive display screen.

[0067] The target location detection step can comprise detecting a zoomed region of the current image.

[0068] Indeed, when the user applies a zoom on a region of the current image, this can indicate that he is focusing on a part of the scene located in the zoom region of the current image. In this case, the target position can be the zoom region, or can be a position within said zoom region, for example the center position of said zoom region.

[0069] The zoom can be applied in a known manner:

[0070] - for example with two fingers, by applying a movement for a tactile zoom. In this case, the target position can be a position equidistant from the positions of the zoom fingers;

[0071] - using a pointer around the region to be zoomed;

[0072] - etc.

[0073] The target position detection step can comprise detecting, by at least one sensor, the position aimed at by a body part of the user on the current image, or be performed in this way.

[0074] In particular, in this case, said body part can be held at a distance from the display medium, and not in contact with the display medium or control surface.

[0075] This part of the user's body can be a finger or a hand pointing at the target position on the current image. In this case, the position aimed at by the finger or hand at a distance from the display medium is detected.

[0076] This body part can be a single eye or both eyes of the user. In this case, the position aimed at by the single eye or both eyes of the user is detected.

[0077] This body part can be the face of the user. In this case, the orientation of the face and the position aimed at by a point on the face, for example the center of the face, are detected.

[0078] The position aimed at by the body part of the user can be detected, for example, by at least one sensor mounted on the display medium, or by a device integrated with the display medium. Alternatively, the sensor can be positioned at a distance from said medium or said device.

[0079] The at least one sensor for detecting the position can be an optical sensor, an acoustic sensor, a capacitive sensor, etc. For example, if the display medium is a touch screen, the sensor can be a capacitive sensor mounted on this touch screen.

[0080] Of course, other types of sensors can also be used, and the application is not limited to a particular type of sensor.

[0081] The display phase can be repeated as many times as desired, from one iteration to the next, with a different target position. In this way, the observer can scan the image to observe different parts of the image, each part being displayed with a sharp focus when the user observes it. In this way, the sharp area of the image is adjusted as the scene is scanned.

[0082] This makes navigation within the image more ergonomic and personalized. Above all, it allows a more realistic observation of the scene displayed on the display medium, as if it were actually in front of the user.

[0083] The application has just been described with reference to an image of a scene.

[0084] Of course, it can be applied to a plurality of images in a stream of images, for example a stream of images forming a video. In this case, the display phase is performed on at least two images of the scene.

[0085] According to another aspect of the application, a computer program is proposed comprising executable instructions which, when executed by a computer device, implement all the method steps according to the application.

[0086] The computer program can use any computer language, for example machine language, C, C++, JAVA, Python, etc.

[0087] According to another aspect of the application, a device is proposed comprising means configured to implement all the method steps according to the application.

[0088] The device according to the application can be any type of apparatus or integrated into any type of apparatus, for example a smartphone, a tablet, a computer, a calculator, a processor, a computer chip, which is programmed to implement the method according to the application, for example by running the computer program according to the application.

[0089] According to another aspect of the application, a device is proposed comprising:

[0090] - an image display means,

[0091] - at least one means for detecting a target position, and

[0092] - at least one computing means;

[0093] configured to implement all the method steps according to the application.

[0094] The device can not comprise an image acquisition means. In this case, the device is used to display one or more images acquired by another device.

[0095] Or, the device can comprise an image acquisition device, such as a camera or a camera module. In this case, the device can be used to display one or more images acquired by said device or by another device.

[0096] In particular, the device can be a user device comprising a display screen, such as a smartphone, a tablet, etc. In this case, the detection device can be or can comprise a touch-sensitive surface, in particular integrated into or associated with the display screen of said device.

[0097] In particular, the device can be a computer-type user device comprising a display screen. In this case, the detection device can be or can comprise a touch-sensitive surface, in particular integrated into or associated with the display screen of said computer, or said detection device can be or can comprise a pointer moved by a mouse, for example, or a directional pad of said computer.

[0098] In particular, the device can be a television. In this case, the detection device can be a camera integrated into said television for detecting the viewer's gaze and head position, or said detection device can be a pointer moved by a remote control of said television, for example.

[0099] In particular, the device can be a virtual reality or augmented reality headset comprising a display screen or a projector associated with a projection surface onto which each image is projected. In this case, the detection device can be or can comprise a sensor, in particular an optical sensor, mounted on said headset.

[0100] Of course, the device according to the application is not limited to the examples disclosed above.

[0101] In particular, the device can be a medical imaging device.

[0102] In particular, the device can be an endoscope, an ultrasound device, etc.

[0103] According to another aspect of the application, a vehicle is proposed, comprising:

[0104] - an image display device, and

[0105] - at least one device for detecting the position of a target,

[0106] - at least one computing device;

[0107] configured to implement all the method steps according to the application.

[0108] The vehicle can not comprise an image acquisition device. In this case, one or more images of the scene are provided by another device or by another vehicle.

[0109] Or, the vehicle can comprise an image acquisition device, such as a camera or camera module. In this case, the one or more images of the scene are captured by said image acquisition device, or provided by another device or another vehicle.

[0110] In some embodiments, the vehicle can be a land vehicle, such as a car, whether autonomous or not.

[0111] In some embodiments, the vehicle can be an aerial vehicle, such as a drone, an airplane or a helicopter, whether autonomous or not.

[0112] In some embodiments, the vehicle can be a watercraft, such as a boat or a submarine, whether autonomous or not.

[0113] According to some embodiments, the at least one image of the scene is a 2D image. In particular, the current image is a 2D image. In particular, the new image is a 2D image.

[0114] The image stack can comprise at least one 2D image. In particular, each image in the image stack is a 2D image.

[0115] In this case, the all-in-focus image is a 2D image.

[0116] According to some embodiments, the at least one image of the scene is a 3D image. In particular, the current image is a 3D image. In particular, the new image is a 3D image.

[0117] The image stack can comprise at least one 3D image. In particular, each image in the image stack is a 3D image.

[0118] In this case, the all-in-focus image is a 3D image.

[0119] BRIEF DESCRIPTION OF DRAWINGS AND EMBODIMENTS

[0120] Other advantages and features will become apparent from review of the detailed description of the completely non-limiting embodiments, taken in conjunction with the accompanying drawings, in which:

[0121] - Figure 1 is a schematic representation of a non-limiting exemplary embodiment of the method according to the present application;

[0122] - Figure 2 and Figure 3 is a schematic representation of a non-limiting exemplary embodiment of the steps for obtaining the new image that can be implemented in the present application;

[0123] - Figure 4 is a schematic representation of a non-limiting exemplary embodiment of the all-in-focus image;

[0124] - Figures 5-7is a schematic view of a further non-limiting exemplary embodiment of a method according to the present application;

[0125] - Figure 8 is a schematic view of a non-limiting exemplary embodiment of a device according to the present application;

[0126] - Figures 9a-9c is a schematic view of a non-limiting exemplary embodiment of an apparatus according to the present application; and

[0127] - Figure 10 is a schematic view of a non-limiting exemplary embodiment of a vehicle according to the present application.

[0128] It should be clearly understood that the embodiments to be described hereinafter are in no way limiting. In particular, variants of the present application comprising only a selection of the features disclosed hereinafter can be considered separately from the other disclosed features, if the selection of features is sufficient to bring about a technical benefit or to distinguish the present application from the prior art. This selection comprises at least one preferred functional feature, which feature does not require structural details, or only a part of the structural details, if only this part is sufficient to bring about a technical benefit or to distinguish the present application from the prior art.

[0129] In particular, all described variants and embodiments can be combined with each other, if no technical obstacles exist for such a combination.

[0130] In the remainder of the drawings and of the description, the same reference signs are used for features common to several figures.

[0131] Figure 1 is a schematic view of a first non-limiting exemplary embodiment of a method according to the present application.

[0132] Figure 1 The method 100 of Fig. 1 can be used to create a customized display of a scene on a display medium from at least one image of said scene.

[0133] The method comprises a display phase 102 for displaying an area of the scene in real time and in sharpness from at least one image of said scene.

[0134] The display phase 102 comprises a step 104 in which an image IC of the scene is displayed on a display medium. This image IC is hereinafter referred to as the current image.

[0135] The display medium can be an electronic screen, in particular a touch screen. Alternatively, the display medium can be a display surface, such as a wall, a board, a panel or a curtain, on which an image is projected, for example by a projector.

[0136] The current image can be an image captured by an imaging device, such as a camera or a camera module. The current image can also be an image generated in a computational manner. The current image displayed on the display medium has a given depth of field, so that a region of the scene is displayed in sharp focus, while the rest of the scene is displayed in blurred focus. However, the user can wish to observe another part of the scene in sharp focus, which is not displayed in sharp focus in the current image.

[0137] The display phase 102 comprises a step 106 for detecting on the current image a target position POS of the scene on which the user is focusing.

[0138] The target position POS can be a point position of the scene on which the user wishes to observe in sharp focus, or a region not limited to a point position.

[0139] The target position POS can be detected in different ways, some non-limiting examples of which are given above.

[0140] In particular, a pointer, such as a mouse pointer or the like, can be used to select the target position POS. In this case, the user moves the pointer to the desired target position.

[0141] The target position POS can be selected by a part of the user's body, such as a finger or a hand, by touching a touch-sensitive surface, which can or can not be integrated into the display medium. For example, the target position POS can be selected by touching a touchscreen forming the display medium.

[0142] The target position POS can be selected by a part of the user's body, such as a finger, a hand, a monocular eye, a binocular eye or a face, which is not in contact with a touch-sensitive surface. In this case, the position on which said part of the body is aimed on the current image, and in turn on the display medium, can be detected in any known manner, for example by means of at least one sensor, which can or can not be integrated into the display medium, such as a capacitive sensor, a camera, a sensor or the like.

[0143] Once the target position POS is known, it is known which part of the scene needs to be displayed in sharp focus.

[0144] The display phase 102 comprises a step 108 for obtaining a new image which displays the scene in sharp focus at the target position POS, or a target region of the scene comprising said target position POS.

[0145] Non-limiting examples of how to obtain such an image are described below, for example with reference to Figure 2 and Figure 3 .

[0146] The new image is displayed on the display medium in step 108 of the display phase 102.

[0147] The display phase 102 can be repeated as many times as needed to change the scene display, in particular to change the area of the scene displayed in sharp focus on the display medium.

[0148] The display phase 102 can be performed each time the user selects or confirms a new target position.

[0149] Alternatively, the display phase can be performed continuously, for example:

[0150] - as soon as the position of the pointer moved by the user changes,

[0151] - as soon as the position aimed at by the finger (or more generally, a body part of the user) on the current image changes; or

[0152] - as soon as the position of contact of the finger or hand of the user changes.

[0153] In some embodiments, the new image displayed in step 108 can be used as the current image for the next iteration of the display phase 102.

[0154] Figure 2 is a schematic representation of a first non-limiting exemplary embodiment of a step for obtaining a new image that can be implemented in the present application.

[0155] As Figure 2 illustrated, step 200 can be implemented to obtain, from a target position POS, a new image to be displayed in the method according to the application, in particular in the method 100 of Figure 1 .

[0156] Step 200 can be or can include step 108 of the method 100 of Figure 1 .

[0157] In the example illustrated, the new image is selected from an image stack PIL of the scene. Each of the images in the image stack presents the scene at a different depth of field. In this way, each image in the stack PIL presents the scene, but in each image a different part of the scene is in sharp focus, while all other parts of the scene are blurred. Each image can be associated with at least one of the following data items:

[0158] - data indicating the area of the image that is in sharp focus, for example by delimiting the coordinates of said area

[0159] - one or more objects that are in sharp focus in the image, for example by indicating identifiers of these objects or positions of these objects, and / or

[0160] - preferably, the depth of field of the image.

[0161] In the following, and without loss of generality, it is assumed that each image is associated with data indicative of the depth of field of the image, the depth of field thereby being indicative of the part of the scene that is rendered in sharp focus on the image.

[0162] The image stack PIL can be acquired by the imaging device using known focus bracketing techniques. In summary, the imaging device is controlled to obtain a plurality of images of the same scene in succession. Between each image, the focus of the device is changed so that each image is acquired at a different focus, and thereby at a different depth of field.

[0163] Of course, the image stack PIL can be computed from one or more images of the scene.

[0164] In step 204, the depth associated with the target position POS in the scene is determined. This target position POS has for example been determined beforehand in step 106.

[0165] When the scene is imaged and stored, this depth can have been measured by one or more sensors. For example, such a measurement can be performed by a sensor such as a laser radar sensor or a time-of-flight camera. In this case, the depth data is read in step 204.

[0166] Alternatively, the depth can be computed from images of at least two scenes, for example by triangulation techniques. These triangulation computation techniques are well known and are not detailed here. In this case, the depth data is computed in step 204, or read in step 204 if the computation has already been done.

[0167] In step 206, the depth of the target position POS is compared to the data indicative of the depth of field in each image associated with each image in the stack 202.

[0168] When the depth of the target position POS is within the depth of field associated with an image in the image stack PIL, this image in the stack is selected in step 206 as the new image. Indeed, since the depth of the target position is within the depth of field of this image of the stack, it means that the part of the scene including the target position is rendered in sharp focus on this image.

[0169] This image can then be selected as the new image.

[0170] Optionally, it can be processed in an optional image processing step 208.

[0171] In the example described with reference to Figure 2 In the example described, the new image is obtained from the depth associated with the target position.

[0172] Of course, many other possible embodiments exist. For example, an object located at the target position can be identified, or a target area comprising the target position can be identified. Then, the new image can be obtained based on said identified object or said identified target area. In this case, in the image stack PIL, the new image is identified and selected in steps 204 and 206 according to an identifier of said object or said target area.

[0173] Figure 3 is a schematic representation of a second non-limiting exemplary embodiment of a step for obtaining a new image that can be implemented in the present application.

[0174] As Figure 3 illustrated, step 300 can be implemented to obtain, from the target position, a new image to be displayed in the method according to the application, in particular in the method 100. Figure 1

[0175] Step 300 can be or can comprise step 108 in the method 100. Figure 1

[0176] In the illustrated example, the new image IN is generated on the fly from a full sharp image INP of the scene. In other words, in the full sharp image INP, the depth of field can cover the entire depth of the scene.

[0177] Such a sharp image INP over the entire scene can be obtained in different ways.

[0178] According to one example, the full sharp image INP can be obtained by stitching images from an image stack, for example the image stack PIL, each image having a limited sharp area of the scene.

[0179] According to another embodiment, the full sharp image INP can be computed from a single image of the scene.

[0180] Non-limiting examples of obtaining a full sharp image will be described in particular with reference to Figure 4 in the following.

[0181] Step 300 can optionally comprise step 202, as described above, which determines the depth of the scene at the target position POS.

[0182] ​​In step 302, a region of the all-in-focus image INP is selected, the region comprising the target position. The region can be selected arbitrarily, as long as it comprises the target position. Alternatively, the region can be selected according to predetermined rules: for example, it can correspond to the center of a region of predetermined size and shape. Alternatively, the region can be selected based on one or more objects located at the target position. According to yet another alternative, the region can be selected according to a depth map of the scene around the position and a predetermined depth interval: for example, it can be desirable that the region comprises an object located at the target position, the object being located in the scene at a depth that lies within an interval centered on the depth of the target position.

[0183] In step 304, the all-in-focus image INP is blurred everywhere except in the region selected in step 302. Such blurring can be performed by all known techniques. The generated image is used as the new image, optionally after an optional image processing step 306.

[0184] In the example described with reference to Figure 3 The new image is obtained according to a depth associated with the target position.

[0185] Of course, other possible designs exist. For example, an object located at the target position can be identified, or a target region comprising the target position can be identified. Then, the new image can be obtained based on said identified object or said identified target region. In this case, the all-in-focus image is blurred everywhere except for the part of it comprising the target object or the part of it corresponding to the target region.

[0186] Figure 4 is a schematic representation of a non-limiting exemplary embodiment of obtaining an all-in-focus image of a scene that can be implemented in the present application.

[0187] As Figure 4 illustrated, the method 400 can be implemented to obtain an all-in-focus image of a scene that can be used in the methods according to the present application, in particular in the method 100 of Figure 1 .

[0188] More specifically, the method 400 illustrated in Figure 4 can be implemented to obtain the all-in-focus image INP used in step 300 of the method 300 of Figure 3 .

[0189] In the example illustrated in Figure 4 , the all-in-focus image is obtained from a stack of images of the scene, the stack comprising a plurality of images, each image representing the scene at a different depth of field.

[0190] As mentioned above, the image stack PIL can be obtained by an imaging device implementing a focus surround technique, or by computation.

[0191] The method 400 comprises a step 402 of extracting, for each image of the image stack PIL, a sharp region of said image.

[0192] There are several ways to extract a sharp region from an image. For example, a local focus estimator (a possible implementation is to analogize it to an energy / variance / entropy operator) is constructed and computed over the whole surface of all images. It is assumed that for a given image region, the estimator is at its best when the image is sharpest. For each image of the stack, the region for which the value given by the local estimator is the largest over the whole image stack is thus determined. The content of these regions is directly copied into the desired result (considering geometric aberrations / focus breathing). By repeating this method for all regions of all images, the resulting image contains at each point the sharpest image information of this point.

[0193] The method 400 comprises a step 404 of constructing a full sharp image from the regions extracted at step 402. Indeed, once the sharp regions of each image of the stack have been extracted, it is possible to concatenate all the resulting sharp regions together.

[0194] The concatenation can be done using any known technique. For example, the concatenation can be based on overlapping regions between sharp regions, and possibly on objects of interest located in said sharp regions.

[0195] Alternatively, the sharp regions can be concatenated pixel by pixel to construct a full sharp image.

[0196] In an optional step 406, the full sharp image is stored.

[0197] Preferably, but in no way limitatively, the full sharp image can be enriched with scene depth data. In particular, depth data can be stored in association with at least one, in particular each, pixel or object of the scene. This or these depth data can be obtained, for example, from each image of the image stack PIL. Alternatively, the image stack can comprise an image or a map showing the depth of the scene at different, in particular each, point of the scene.

[0198] Figure 5 is a schematic representation of another non-limitative exemplary embodiment of a method according to the application.

[0199] Figure 5 The method 500 of can be used to perform a customized display of a scene on a display medium from at least one image of said scene.

[0200] Figure 5 The method 500 of comprises Figure 1all the steps of the method 100.

[0201] Figure 5 The method 500 can further comprise an optional step 502 of obtaining an image stack of the scene, each image showing the scene at a limited depth of field, such that only a portion of the scene is sharply rendered on said image.

[0202] The image stack can be obtained using known focus bracketing techniques. Alternatively, the image stack can be computed from one or more images of the scene.

[0203] For example, the image stack can be Figure 2 and Figure 4 the image stack PIL shown in Fig. 1 1.

[0204] In this case, and without loss of generality, the step 108 for obtaining a new image can be performed according to the example described with reference to Figure 2 Fig. 1 1.

[0205] Figure 6 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present application.

[0206] Figure 6 The method 600 can be used to perform a customized display of the scene on a display medium based on at least one image of said scene.

[0207] Figure 6 The method 600 comprises Figure 1 all the steps of the method 100.

[0208] Figure 6 The method 600 can further comprise an optional step 602 for obtaining a full- sharp image.

[0209] The full-sharp image can be computed from the image stack. For example, the full- sharp image can be obtained by Figure 4 the method 400.

[0210] The full-sharp image can be computed from a single image of the scene.

[0211] In this case, and without loss of generality, the step 108 for obtaining a new image can be performed according to the example described with reference to Figure 3 Fig. 1 1.

[0212] Figure 7 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present application.

[0213] Figure 7 The method 700 can be used to perform a customized display of the scene on a display medium based on at least one image of said scene.

[0214] Figure 7 The method 700 comprises all the steps of the method 600. Figure 6

[0215] Figure 7 The method 700 further comprises an optional step 502 of obtaining an image stack of the scene.

[0216] In all the examples described, the step 502 can be implemented in the same device as the device performing the displaying phase 102. Alternatively, the step 502 can be implemented in a different (in particular remote) device than the device performing the displaying phase 102.

[0217] In the examples described, the step 602 can be implemented in the same device as the device performing the displaying phase 102. Alternatively, when the device performing the image supplying step 502 is different than the device performing the displaying phase 102, the step 602 can be implemented in the same device as the device performing the image supplying step 502. In yet another alternative, the step 602 can be implemented in a different (in particular remote) device than the device performing the displaying phase 102, and / or in a different (in particular remote) device than the device performing the supplying step 502 of the image stack.

[0218] Figure 8 is a schematic representation of a non-limiting exemplary embodiment of a device according to the application.

[0219] Figure 8 The device 800 comprises a module 802 implementing the displaying phase of the method according to the application, in particular the displaying phase 102.

[0220] The module 802 comprises a module 804 for displaying an image of the scene as input. In particular, this module is configured / programmed to perform steps 104 and 110 of the displaying phase 102.

[0221] The module 802 further comprises a module 806 for detecting a target position on the currently displayed image. This module 806 takes as input the position of a pointer or of a body part of the user and determines the target position on the current image. In particular, this module 806 is configured / programmed to perform step 106.

[0222] The module 802 further comprises a module 808 for obtaining a new image to be displayed, e.g. according to any of the techniques described above. In particular, this module 808 is configured / programmed to perform step 108.

[0223] At least one of these modules can be a module independent from the other modules.

[0224] At least two of these modules can be integrated within a single module. ​

[0225] At least one of these modules can be a hardware module.

[0226] At least one of these modules can be a software module, such as a computer program.

[0227] At least one of these modules may be a combination of at least one software module (e.g., a computer program) and at least one hardware module.

[0228] In particular, at least one of modules 804-808 or module 802 may be integrated into an electronic chip or into an application installed in a user device.

[0229] The device 800 may also optionally include at least one display device (e.g., a touch-sensitive or non-touch-sensitive display screen) or means for projecting an image onto a medium to display a scene image.

[0230] This display device can be integrated into the equipment.

[0231] This display device is optional, as the device may not include such a device. For example, device 800 can be integrated into a device that already has a display device and cooperate with the display device to display an image of the scene.

[0232] According to another alternative, device 800 can be connected to an external display device, or to an external device that has a display device or is itself connected to a display device.

[0233] In the example shown, and without loss of generality, the display device may be an electronic display screen.

[0234] The device 800 may also optionally include at least one sensor 812 for detection:

[0235] - The position of the pointer

[0236] -The location of contact with the user's body part (e.g., hand or fingers), or

[0237] - The location to be aimed at a part of the user's body (e.g., hand or finger, one eye, both eyes, or the user's face);

[0238] To provide or determine the target position on the currently displayed image on the display medium.

[0239] The sensor can be any type of sensor, such as a camera, LiDAR, sensing surface (e.g., capacitive sensing surface), etc.

[0240] exist Figure 8 In the example shown, and without loss of generality, the sensor may take the form of a capacitive sensing surface integrated into the display 810.

[0241] Optionally, the device 800 may also include at least one image acquisition device 814, such as a camera or camera module, including an optical lens and an image sensor, for acquiring images or image stacks of the scene.

[0242] This image acquisition device 814 can be integrated into a device, for example, integrated into the front or rear, or integrated into both the front and rear.

[0243] This image acquisition device 814 is optional, as the device may not include it. For example, the device 800 can be integrated into a device that already has an image acquisition device (e.g., a camera module) and works in conjunction with it.

[0244] According to another alternative, device 800 can be connected to an external image acquisition device, or to an external device having image acquisition device 814 or being connected to image acquisition device 814 itself.

[0245] Optionally, the device 800 may also include at least one computing unit (not shown) for generating a fully sharp image based on an image of the scene or from an image stack of scenes with different depths of field, as described above.

[0246] Optionally, the device 800 may also include at least one

[0247] A computing unit (not shown) is used to generate image stacks of scenes, each with a different depth of field, from images of the scene, as described above.

[0248] Figure 9a This is a schematic diagram of a non-limiting exemplary embodiment of the device according to the present invention.

[0249] Figure 9a The apparatus 910 includes means configured to carry out the present invention, particularly any one of methods 100, 500, 600, and 700.

[0250] Figure 9a The device 910 may include the device according to the invention, in particular Figure 8 The equipment is 800.

[0251] exist Figure 9a In the example shown, device 910 includes Figure 8 The device 800 is a smartphone or tablet. In particular, the device 910 includes a display screen 810 and at least one camera 814, the display screen being equipped with a sensing surface 812 (e.g., capacitive).

[0252] Figure 9b This is a schematic diagram of another non-limiting exemplary embodiment of the device according to the present invention.

[0253] Figure 9b The device 920 comprises means configured to implement the application, in particular any one of the methods 100, 500, 600 and 700.

[0254] Figure 9b The device 920 can comprise at least one apparatus according to the application, in particular Figure 8 The apparatus 800 (without the camera 814).

[0255] In the example illustrated in Figure 9b , the device 920 is a virtual reality headset or an augmented reality headset comprising Figure 8 The apparatus 800. In particular, the headset device 920 further comprises a display screen 810, and a sensor (not visible in Figure 9B) for detecting the aiming position of a single eye or both eyes of a user on said display screen 810.

[0256] In the example illustrated in Figure 9b , the headset device 920 does not comprise an imaging device for capturing images of a scene. In this case, the images of a scene to be displayed by the headset device 920 are provided by another apparatus.

[0257] Alternatively, the headset device 920 can comprise at least one camera to capture images of a scene in which it is located, to be displayed on the screen 810, optionally after augmentation of said images, as part of for example an augmented reality application.

[0258] Figure 9c is a schematic view of a non-limiting exemplary embodiment of a device according to the application.

[0259] Figure 9c The device 930 comprises means configured to implement the application, in particular any one of the methods 100, 500, 600 and 700.

[0260] Figure 9c The device 930 can comprise at least one apparatus according to the application, in particular Figure 8 The apparatus 800.

[0261] In the example illustrated in Figure 9c , the device is a medical imaging device comprising Figure 8 The apparatus 800, for example an endoscope, an ultrasound apparatus, etc. In particular, the medical imaging device 930 comprises a display screen 810 equipped with a sensitive surface 812 (for example capacitive). The medical imaging device 930 further comprises an imaging device formed by a distal lens (not shown) connected to an imaging module.

[0262] Figure 10is a schematic view of a non-limiting exemplary embodiment of a vehicle according to the application.

[0263] Figure 10 The vehicle 1000 comprises means configured to implement the application, in particular any one of the methods 100, 500, 600 and 700.

[0264] Figure 10 The vehicle 1000 can comprise a device according to the application, in particular Figure 8 the device 800.

[0265] In the example shown, Figure 10 the vehicle 1000 is a land vehicle, in particular a car, comprising Figure 8 the device 800. In particular, the vehicle 1000 comprises a display screen 810 equipped with an inductive surface 812, for example capacitive, arranged inside the passenger compartment of the vehicle 1000. The vehicle 1000 also comprises at least one camera, for example on the windshield of the vehicle 1000.

[0266] Of course, the application is not limited to the examples disclosed above.

Claims

1. A method (100, 500, 600, 700) for presenting a scene on a display medium, said method (100, 500, 600, 700) comprising at least one iteration of a display phase (102), said display phase comprising the following steps: - Display (104) an image (IC) of the scene with a clear area, referred to as the current image, on the display medium. - Detect (106) a location of interest to the user in the current image (IC), the location being referred to as the target location. - Obtain a new image (IN) of the scene (108), the sharp area of ​​the new image being adjusted according to the target location to create a sharp display of the scene at the target location, and - Display (110) the new image (IN).

2. The method according to the preceding claim (100, 500, 600, 700), characterized in that, The step (108) of obtaining a new image includes generating the new image on the fly by calculation (302, 304).

3. The method according to the preceding claim (100, 500, 600, 700), characterized in that, The source image (INP) is a previously acquired image that shows the scene in sharp focus in all areas.

4. The method (100, 500, 600, 700) according to claim 1, characterized in that, The step of obtaining a new image (108) can include the step of selecting the new image from the image stack (PIL) of the scene (206), where each image in the image stack has a different area of ​​sharpness.

5. The method according to the preceding claim (100, 500, 600, 700), characterized in that, The image stack (PIL) includes at least one image captured by the imaging device.

6. The method (100, 500, 600, 700) according to any one of claims 3 or 4, characterized in that, The image stack (PIL) includes at least one image obtained from at least one other image through computation or simulation.

7. The method according to any one of the preceding claims (100, 500, 600, 700), characterized in that, After identifying the target location, the display stage (102) also includes a step for identifying an object located at the target location, based on which the new image is obtained.

8. The method according to any one of the preceding claims (100, 500, 600, 700), characterized in that, The display phase (102) also includes a step for identifying a region of the scene, including the target location, called the target region, upon which the new image is obtained.

9. The method according to any one of the preceding claims (100, 500, 600, 700), characterized in that, The display phase (102) includes the step of determining depth data of the scene at the target location, and the new image is obtained based on the depth data.

10. The method according to any one of the preceding claims (100, 500, 600, 700), characterized in that, The target location is manually entered by the user.

11. The method according to any one of the preceding claims (100, 500, 600, 700), characterized in that, The target location detection step includes detecting the zoomed area of ​​the current image.

12. The method according to any one of the preceding claims (100, 500, 600, 700), characterized in that, The target location detection step (106) includes detecting the location of the user's body part being targeted by at least one sensor.

13. The method according to any one of the preceding claims (100, 500, 600, 700), characterized in that, The display phase is repeatedly performed for multiple images in an image stream, particularly a video (102).

14. A computer program comprising executable instructions that, when executed by a computer device, perform all the steps of the method (100, 500, 600, 700) according to any one of the preceding claims.

15. An apparatus (800) comprising means configured to perform all steps of the method (100, 500, 600, 700) according to any one of claims 1 to 13.

16. An apparatus (910, 920, 930) comprising: -Image display device (810), and - At least one device (612) for detecting the target location. - At least one computing device (802); Configured to implement all steps of the method (100, 500, 600, 700) according to any one of claims 1 to 13.

Images