Method and device for the personalized displaying of a scene on the basis of one or more images of said scene

Abstract

The invention relates to a method (100) for representing a scene on a display medium, said method (100) comprising at least one iteration of a display phase (102) comprising the following steps: - displaying (104), on said display medium, an image (IC), referred to as current image, of said scene having a sharp area, - detecting (106) a position, referred to as target position, of interest to a user in the current image (IC), - obtaining (108) a new image (IN) of the scene, the sharp area of which is adjusted according to the target position so as to create a sharp display of the scene in the target position, and - displaying (110) the new image (IN). The invention also relates to a computer program, a device, an apparatus and a vehicle implementing such a method.

Description

DESCRIPTION Title: Method and device for personalized display of a scene from image(s) of said scene

[0001] The present invention relates to a method for displaying image(s) enabling a personalized display of a scene on a display medium. It also relates to a computer program, a device and an apparatus implementing such a method.

[0002] The field of the invention is the field of displaying a digital image on a display medium, such as a display screen or a projection surface. State of the art

[0003] When the human eye scans a scene, it observes the scene using different focal points to perceive the entire scene. The focal points used to scan a scene depend directly on the depth of the objects in the scene. Generally, to observe a part of a scene, the human eye adapts the focal point to the depth of the scene in that part, to obtain a clear perception of the scene in that area.

[0004] However, the methods for displaying an image of a scene currently known display said image according to a single focus, namely the focus used to capture said image. When the scene comprises parts located at different depths, the focus corresponds to one of said depths so that only one of the parts of the scene is clear in the displayed image.

[0005] Such a display gives a very limited representation of the scene.

[0006] Moreover, such a display does not adapt at all to the user's wishes and limits the user to observing the scene according to the choice made when acquiring the image of the scene.

[0007] An aim of the present invention is to remedy at least one of the drawbacks of the state of the art.

[0008] Another aim of the invention is to propose a solution for displaying an image of a scene which offers greater freedom for the observer.

[0009] Another aim of the invention is to propose a solution for displaying an image of a scene allowing a richer representation of the scene.

[0010] Another aim of the invention is to propose a solution for displaying an image of a scene allowing greater customization of the display. Statement of the invention

[0011] The invention proposes to achieve at least one of the aforementioned aims by a method of displaying a scene on a display medium, said method comprising at least one iteration of a display phase comprising the following steps: - displaying, on said display medium, an image, called current image, of said scene having a sharpness zone, - detection of a position, called target position, on said current image in which a user is interested, - obtaining a new image of said scene whose sharpness zone is adjusted according to said target position so as to produce a sharp display of said scene at said target position, and - displaying said new image.

[0012] Thus, in a conventional manner, the current image displayed on the display medium includes a sharpness zone corresponding to a given depth of field: all the other zones of the current image are blurred. The method according to the invention proposes to determine the target position in which a user observing the current image is interested: this target position corresponds to the target zone of the scene in which the observer is interested. Then, the method according to the invention proposes to modify the display so that said target area is made clear.

[0013] Thus, the display of the scene is personalized and adapts to the user's observation so that the target area observed by the user is always sharp. In other words, the sharp area of ​​the scene displayed on a screen follows the user's gaze. Unlike the prior art, the depth of field of an image of a scene is not imposed on the observer. On the contrary, with the invention, the depth of field adapts and adjusts to the observer.

[0014] According to the invention, the display medium can be any type of display medium.

[0015] For example, the display medium may be a display screen, such as, for example, a screen of a smartphone, a tablet, a computer, a television, etc.

[0016] For example, the display medium may be a surface onto which an image of the scene is projected, such as, for example, a display surface associated with a projector, etc.

[0017] For example, the display medium may be a medium worn by the user, for example, a goggle, or glasses, a virtual reality headset, an augmented reality headset, etc.

[0018] The display medium may be a display medium fitted to the device carrying out the display phase, such as: - a display screen fitted to said device, for example in the case of a smartphone, a tablet, etc. or - a projection surface equipping said device, for example in the case of an augmented reality or virtual reality headset.

[0019] The display medium may be a medium independent of the device carrying out the display phase, for example in the context of a projector projecting the images onto a display surface.

[0020] According to embodiments, the target position may be a point position on the current image.

[0021] According to embodiments, the target position may be an area of ​​the current image. For example, the target position may correspond to the target area.

[0022] According to embodiments, the step of obtaining the new image may comprise generating the new image, by calculation.

[0023] Indeed, the new image can be generated on the fly after the target position is determined. In this case, the new image does not exist until the target position is known. Once the target position is known, then the new one is generated.

[0024] The new image can be generated from the current image.

[0025] For example, the new image may be generated by blurring the currently sharp area of ​​the current image and sharpening the target area including, or corresponding to, the target position.

[0026] The current image can be an image captured by the same device as the one implementing the display phase.

[0027] Alternatively, the current image may be an image captured by another device and transmitted to the device performing the display phase.

[0028] The new image can be generated from a source image, which is not the current image.

[0029] For example, the source image may be a previously obtained image that represents the scene in sharpness in all areas. In this case, a copy of this source image is made. The target area is kept sharp and all other areas are blurred, to obtain the new image.

[0030] Such a source image can be obtained by taking several images of the scene with different depths of field, for example by the focus bracketing technique. Then, the sharp area of ​​the scene is selected in each image. All the sharp areas obtained from all the images can be concatenated to obtain an image of the scene sharp in all areas.

[0031] Such a source image can also be obtained, by simulation or calculation, from an image of the scene. For example, by using the transfer function of the optical lens used to capture the image, it is possible to defocus the image to regain total sharpness at all points. Such defocusing can, for example, be achieved by inverting the optical transfer function (deconvolution).

[0032] The method according to the invention may comprise a step of obtaining such a source image, prior to the first iteration of the display phase.

[0033] The step of obtaining the source image can be performed by the same device that implements the display phase.

[0034] Alternatively, the step of obtaining the source image may be performed by a device other than the device that implements the display phase. In this case, the source image is transmitted to the device performing the display phase.

[0035] According to embodiments, the step of obtaining the new image may comprise a step of selecting the new image from a stack of images of said scene, each having a different area of ​​sharpness.

[0036] In this case, the new image is not generated on the fly. It exists prior to the identification of the target position. This embodiment allows greater responsiveness of the method according to the invention.

[0037] In this case, the obtaining step may include identifying, in the image stack, the image on which the target position is clearly represented. This image is selected as the new image to be displayed during the displaying step.

[0038] Identifying the new image in the image stack can be done in different ways. For example, each image in the stack can be associated with information indicating the sharp area on said image. In this case, the identification step consists of identifying which of the sharp areas includes the target position and selecting the image associated with this sharp area.

[0039] In some embodiments, the image stack may include at least one image captured by an imaging device, such as a camera.

[0040] In particular, the image stack may have been captured by one or more cameras, through the implementation of the focus bracketing technique providing multiple images of a scene, each with a different depth of field and therefore a different sharp area.

[0041] According to embodiments, the image stack may comprise at least one image, obtained by calculation or by simulation, from at least one other image.

[0042] This other image can be an image with a sharp area and blurred areas. In this case, the calculation to obtain an image of the stack consists of generating an image with a different depth of field than that of the other image. The sharp area of ​​the other image is made blurred and another area of ​​the scene is made sharp. To do this, the calculation consists of determining as best as possible the transfer function of the original optics as a function of the defocus, and determining the transfer function that we wish to simulate given the distance of the points to be represented, that is to say as a function of the simulated 'focal' distance and the distance of the target area. Classically, we can be tempted to find total sharpness at all points by inverting the optical transfer function (deconvolution) before applying a parametric refloating.To minimize artifacts related to noise and modeling errors, and the amount of calculations that controlling these effects involves, we focus here on calculating only an incomplete deconvolution, which brings us back from the transfer function of the original optics to the transfer function that we wish to simulate, which is also incompletely clear over most of the image.

[0043] This other image can also be an image of the scene that is sharp in all areas, such as the source image described above.

[0044] The method according to the invention may comprise a step of obtaining such a stack of images, prior to the first iteration of the display phase.

[0045] The step of obtaining the image stack can be performed by the same device that implements the display phase.

[0046] Alternatively, the step of obtaining the image stack may be performed by a device other than the device that implements the display phase. In this case, the image stack is transmitted to the device performing the display phase.

[0047] The new image can be selected in several ways.

[0048] According to embodiments, the display phase may further comprise, after identifying the target position, a step of identifying an object located at the target position. In this case, obtaining the new image may be carried out as a function of said object.

[0049] In particular, the selection of the new image may include identifying the image on which said object is in focus. To do this, each image in the stack may be associated with information indicating the object(s) represented in focus on said image.

[0050] According to embodiments, the display phase may comprise, after identification of the target position, a step of identifying a zone, called target, of the scene comprising the target position. In this case, obtaining the new image may be carried out as a function of said target zone.

[0051] In particular, the selection of the new image may include identifying the image on which said target area is in focus. To do this, each image in the stack may be associated with information indicating the area represented in focus on said image.

[0052] According to embodiments, the display phase may comprise a step of determining depth data of the scene at said target position, the new image being obtained as a function of said depth data.

[0053] Indeed, once the target position is identified, the depth of the scene at said position can be determined. The scene depth information at the target position can be calculated on the fly, for example by triangulation with another image of the scene. Alternatively, the depth information of the scene at the target position may have been measured when the scene was imaged, by suitable measurement means, such as a time-of-flight camera, lidar, etc.

[0054] This depth information can be used, during the obtaining step, to select the new image in a stack of images: in this case, the image for which the depth of field corresponds to or includes the depth of the target position is identified. To do this, each image in the image stack can be associated with information indicating for said image the depth of field of said image.

[0055] Alternatively, the depth information of the target position can be used to generate the new image on the fly, by a calculation taking into account the depth of the target position, from the current image or from a source image, such as the source image described above. In this case, we apply a convolution of the image data by linear filtering whose kernel reproduces the desired type of blur (bokeh, Gaussian): this kernel can be parameterized to control the intensity of the effect. In the Gaussian case the formula is parameterized by the width sigma: The sigma width actually used at a point in the scene increases with the difference between the focus distance that we want to simulate, and the distance of the object to be represented.

[0056] The target position the user is interested in can be detected in different ways.

[0057] According to embodiments, the target position may be manually entered by the user.

[0058] In this case, the user manually selects the target position on the current image, for example using a pointer, such as a mouse, a trackball, or a directional pad moving a cursor, or any other position input device. The user can also enter the target position by touch selection on a surface touch screen arranged on the display support in the case, for example, of a touch screen display, or offset from the display support.

[0059] The target position detection step may include detecting a zoomed region of the current image.

[0060] Indeed, when the user zooms in on a region of the current image, this probably indicates that he is paying attention to the part of the scene located in said zoomed region of the current image. In this case, the target position may be the zoomed region, or may be a position in said zoomed region, such as for example a central position of said zoomed region.

[0061] Zoom can be applied in a known way: - tactilely by applying a movement, for example with two fingers. In this case, the target position can be the position equidistant from the positions of the fingers applying the zoom; - using a pointer surrounding the region to zoom; - etc.

[0062] The step of detecting the target position may comprise, or be carried out, by a detection, by at least one sensor, of a position targeted by a part of the user's body, on the current image.

[0063] In this case, in particular, said body part may be at a distance from the display medium, and does not come into contact with said display medium or a control surface.

[0064] This part of the user's body can be a finger, or a hand, pointing at the target position on the current image. In this case, it is the position aimed at by the finger or hand located at a distance from the display support that is detected.

[0065] This body part can be an eye, or the user's eyes. In this case, it is the position targeted by the user's eye or eyes that is detected.

[0066] This body part can be a user's face. In this case, the orientation of the face is detected and a position aimed at a point on the face, for example the center of the face, is detected.

[0067] The position targeted by the user's body part may be detected, for example, by at least one sensor fitted to the display support, or a device integrated into the display support. The sensor may alternatively be arranged at a distance from said support, or said device.

[0068] The at least one sensor for detecting the position may be an optical sensor, an acoustic sensor, a capacitive sensor, etc. For example, in the case where the display medium is a touch screen, then the sensor may be a capacitive sensor equipping said touch screen.

[0069] Of course, other types of sensors can be used and the invention is not limited to a specific type of sensor.

[0070] The display phase can be repeated as many times as desired with a different target position from one iteration to the next. Thus, an observer can browse an image to observe different parts of the image, each part being displayed sharply when observed by the user. Thus, the sharpness area of ​​the image is adjusted as the scene is browsed.

[0071] This allows for more ergonomic and personalized navigation within an image. Above all, it allows a scene displayed on a display medium to be observed more realistically, as if the scene were actually in front of the user.

[0072] The invention has just been described with reference to an image of the scene.

[0073] Of course, it can be applied to several images, within an image stream, for example a stream of images forming a video. In this case, the display phase is carried out for at least two images of the scene.

[0074] According to another aspect of the invention, there is provided a computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the method according to the invention.

[0075] The computer program can be in any computer language, such as machine language, C, C++, JAVA, Python, etc.

[0076] According to another aspect of the invention, a device is proposed comprising means configured to implement all the steps of the method according to the invention.

[0077] The device according to the invention can be, or be integrated into, any type of device such as a smartphone, a tablet, a computer, a calculator, a processor, a computer chip, programmed to implement the method according to the invention, for example by executing the computer program according to the invention.

[0078] According to another aspect of the invention, there is provided an apparatus comprising: - a means of displaying an image, - at least one means of detecting a target position, and - at least one calculation means; configured to implement all the steps of the method according to the invention.

[0079] The device may not include an image acquisition means. In this case, the device is used to display one or more images acquired by another device.

[0080] Alternatively, the apparatus may comprise an image acquisition means, such as a camera or a camera module. In this case, the apparatus may be used to display one or more images acquired by said apparatus or by another apparatus.

[0081] In particular, the device may be a user device such as a smartphone, tablet, etc. comprising a display screen. In this case, the detection means may be or may comprise the touch surface, in particular integrated into, or associated with, the display screen of said device.

[0082] In particular, the device may be a computer-type user device, comprising a display screen. In this case, the detection means may be or may comprise a touch surface, in particular integrated into, or associated with, the display screen of said computer, or a pointer moved for example by a mouse, or a directional pad of said computer.

[0083] In particular, the device may be a television. In this case, the detection means may be a camera integrated into said television, detecting the gaze and the position of the head of the observer, or a pointer moved for example by a remote control of said television.

[0084] In particular, the device may be a virtual reality or augmented reality headset comprising a display screen or a projector associated with a projection surface onto which each image is projected. In this case, the detection means may be or may comprise a sensor, in particular an optical sensor, equipping said headset.

[0085] Of course, the apparatus according to the invention is not limited to the examples which have just been given.

[0086] In particular, the device may be a medical imaging device.

[0087] In particular, the device may be an endoscope, an ultrasound device, etc.

[0088] According to another aspect of the present invention, there is provided a vehicle comprising: - a means of displaying an image, and - at least detection of a target position, - at least one calculation means; configured to implement all the steps of the method according to the invention.

[0089] The vehicle may not include a means of image acquisition. In this case, the image(s) of the scene are provided by another device or another vehicle.

[0090] Alternatively, the vehicle may comprise an image acquisition means, such as a camera or a camera module. In this case, the image(s) of the scene are captured by said image acquisition means, or provided by another device or another vehicle.

[0091] According to embodiments, the vehicle may be a land vehicle, such as a car, autonomous or not.

[0092] According to embodiments, the vehicle may be a flying vehicle, such as a drone, an airplane, a helicopter, autonomous or not.

[0093] According to embodiments, the vehicle may be a maritime vehicle, such as a boat or a submarine, autonomous or not.

[0094] According to embodiments, at least one image of the scene is a 2D image. In particular, the current image is a 2D image. In particular, the new image is a 2D image.

[0095] If applicable, the image stack includes at least one 2D image. In particular, every image in the image stack is a 2D image.

[0096] If the image is clear everywhere, it is a 2D image.

[0097] According to embodiments, at least one image of the scene is a 3D image. In particular, the current image is a 3D image. In particular, the new image is a 3D image.

[0098] If applicable, the image stack includes at least one 3D image. In particular, every image in the image stack is a 3D image.

[0099] If the image is clear everywhere, it is a 3D image. Description of figures and embodiments

[0100] Other advantages and characteristics will appear on examining the detailed description of non-limiting embodiments, and the attached drawings in which: - FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of a method according to the invention; - FIGURES 2 and 3 are schematic representations of non-limiting exemplary embodiments of a step of obtaining a new image which can be implemented in the present invention; - FIGURE 4 is a schematic representation of a non-limiting example of obtaining a clear image everywhere; - FIGURES 5-7 are schematic representations of other non-limiting exemplary embodiments of a method according to the invention; - FIGURE 8 is a schematic representation of a non-limiting exemplary embodiment of a device according to the invention; - FIGURES 9a-9c are schematic representations of non-limiting exemplary embodiments of an apparatus according to the invention; and - FIGURES 10 is a schematic representation of a non-limiting exemplary embodiment of a vehicle according to the invention.

[0101] It is understood that the embodiments which will be described below are in no way limiting. In particular, it is possible to imagine variants of the invention comprising only a selection of characteristics described below isolated from the other characteristics described, if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection includes at least one preferably functional characteristic without structural details, or with only part of the structural details if it is this part which is only sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art.

[0102] In particular, all the variants and embodiments described can be combined with each other if there is no technical obstacle to this combination.

[0103] In the figures and in the rest of the description, the elements common to several figures retain the same reference.

[0104] FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of a method according to the present invention.

[0105] The method 100 of FIGURE 1 can be used to provide a personalized display of a scene on a display medium from at least one image of said scene.

[0106] The method comprises a display phase 102 allowing the clear display, on the fly, of an area of ​​a scene from at least one image of said scene.

[0107] The display phase 102 comprises a step 104 during which an image IC of the scene is displayed on a display medium. This image IC is called the current image in the following.

[0108] The display medium may be an electronic screen, and in particular a touch screen. The display medium may alternatively be a display surface, such as a wall, a board, a panel or a canvas, onto which the image is projected, for example by a projector.

[0109] The current image may be one captured by an imaging device, such as a camera, or a camera module. The current image may also be a computationally generated image. The current image displayed on the display medium has a given depth of field such that a region of the scene is displayed sharply and the rest of the scene is displayed blurred. However, the user may wish to observe another part of the scene sharply, which is not the one displayed sharply in the current image.

[0110] The display phase 102 comprises a step 106 of detecting, on the current image, a target position POS of the scene on which the user is focusing his attention. [YES] This target position POS can be a point position, or an area not limited to a point position, that the user wants to observe in the scene, and in a clear manner.

[0112] The target position POS can be detected in different ways, some non-limiting examples of which are given above.

[0113] In particular, the target position POS can be selected with a pointer, mouse-like or similar. In this case, the user moves the pointer to the desired target position.

[0114] The POS target position may be selected by a part of the user's body, such as a finger or a hand, by contact on a touch-sensitive surface, whether or not integrated into the display medium. For example, the POS target position may be selected by contact on a touchscreen forming the display medium.

[0115] The target position POS may be selected by a part of the user's body, such as a finger, a hand, an eye, the eyes or the user's face, which does not come into contact with a touch-sensitive surface. In this case, the position targeted by said part of the body on the current image, and therefore on the display medium, may be detected in any known manner, for example by at least one sensor, integrated or not in the display medium, such as a capacitive sensor, a camera, a sensor, etc.

[0116] Once the target position POS is known, we then know which part of the scene should be displayed sharply.

[0117] The display phase 102 comprises a step 108 of obtaining a new image clearly displaying the scene at the target position POS, or a target area of ​​the scene comprising said target position POS.

[0118] Non-limiting examples of obtaining such an image are described below, for example with reference to FIGURES 2 and 3.

[0119] The new image is displayed on the display medium during a step 108 of the display phase 102.

[0120] The display phase 102 can be repeated as many times as desired to change the display of the scene, and in particular to change the area of ​​the scene displayed clearly on the display medium.

[0121] The display phase 102 can be carried out each time the user chooses, or validates, a new target position.

[0122] Alternatively, the display phase can be performed continuously, for example: - as soon as the position of a pointer moved by the user changes, - as soon as the position aimed at on the current image by a finger, and more generally by a part of the user's body, changes; or - as soon as the contact position of a user's finger or hand changes.

[0123] In some embodiments, the new image displayed in step 108 may be used as the current image for the next iteration of display phase 102.

[0124] FIGURE 2 is a schematic representation of a first non-limiting exemplary embodiment of a step of obtaining a new image, which can be implemented in the present invention.

[0125] Step 200, shown in FIGURE 2, can be implemented to obtain a new image to be displayed, from a target position POS, in the method according to the invention, and in particular in the method 100 of FIGURE 1.

[0126] Step 200 may be, or may be included in, step 108 of method 100 of FIGURE 1.

[0127] In the example shown, the new image is selected from a stack of PIL images of the scene. Each of the images in the image stack represents the scene with a different depth of field. Thus, each image in the PIL stack represents the scene, but in each image a different part of the scene is in focus and all other parts of the scene are blurred. Each image can be associated with at least one of the following data: - data indicating the area of ​​the image which is in focus, for example with coordinates delimiting said area, - one or more objects that are displayed clearly in the image, for example by indicating an identifier for these objects, or the position of these objects, and / or - preferably, the depth of field of the image. In the following, and without loss of generality, we consider that each image is associated with data indicating the depth of field of the image, said depth of field thus indicating the part of the scene which is clearly represented on the image.

[0128] The PIL image stack can be acquired using known focus bracketing techniques, also called "focus stacking" in French, with an imaging device. In short, the imaging device is controlled to obtain multiple images of the same scene one after the other. Between each image, the camera's focus is changed so that each image is acquired with a different focus, and therefore with a different depth of field.

[0129] Of course, the PIL image stack can be obtained by calculation, from one or more images of the scene.

[0130] In a step 204, the depth associated with the target position POS, in the scene, is determined. This target position POS has been previously determined, for example in step 106.

[0131] This depth may have been measured, by one or more sensors, when the scene is imaged and stored. Such a measurement can for example be carried out by sensors such as lidar sensors, or time-of-flight cameras, etc. In this case, the depth data is read during step 204.

[0132] Alternatively, the depth can be calculated, for example by triangulation techniques, from at least two images of the scene. These triangulation calculation techniques are well known and will therefore not be detailed here. In this case, the depth data is calculated in step 204, or read when the calculation has been carried out previously.

[0133] During a step 206, the depth of the target position POS is compared to the data associated with each image of the stack 202, and indicating the depth of field in said image.

[0134] When the depth of the target position POS is included in the depth of field associated with an image of the stack of images PIL, this image of the stack is selected, during a step 206, as a new image. Indeed, since the depth of the target position is within the depth of field of this image of the stack, this means that a part of the scene, including the target position, is represented clearly on this image.

[0135] The image from the stack can then be selected as the new image.

[0136] Optionally, it may be processed during an optional image processing step 208.

[0137] In the example described with reference to FIGURE 2, the new image is obtained from the depth associated with the target position.

[0138] Of course, other embodiments are possible. For example, it is possible to identify an object located at the target position, or it is possible to identify a target area comprising the target position. Obtaining the new image can then be carried out as a function of said identified object, or of said identified target area. In this case, in the PIL image stack, the new image is identified and selected during steps 204 and 206, as a function of an identifier of said object, or of said target area.

[0139] FIGURE 3 is a schematic representation of a second non-limiting exemplary embodiment of a step of obtaining a new image, which can be implemented in the present invention.

[0140] Step 300, shown in FIGURE 3, can be implemented to obtain a new image to be displayed, from a target position, in the method according to the invention, and in particular in the method 100 of FIGURE 1.

[0001] Step 300 may be, or may be included in, step 108 of method 100 of FIGURE 1.

[0002] In the example shown, the new IN image is generated, on the fly, from an INP image of the scene, sharp everywhere. In other words, in the sharp everywhere INP image, the depth of field potentially covers the entire depth of the scene.

[0003] Such a sharp INP image across the entire scene can be obtained in various ways.

[0004] According to an exemplary embodiment, the everywhere sharp image INP can be obtained by assembling images from a stack of images, each having a limited sharpness zone of the scene, such as for example the stack of images PIL.

[0005] According to another exemplary embodiment, the clear image everywhere INP can be obtained by calculation, from a single image of the scene.

[0006] A non-limiting example of obtaining a sharp image throughout will be described later, in particular with reference to FIGURE 4.

[0007] Step 300 may optionally comprise step 202 determining the depth of the scene at the target position POS, as described above.

[0008] In a step 302, an area of ​​the sharp image everywhere IM is selected, this area comprising the target position. This area can be selected arbitrarily, as long as it contains the target position. Alternatively, this area can be selected, according to a predetermined rule: for example, it can correspond to the center of an area of ​​predetermined size and shape. According to yet another alternative, this area can be selected, according to one or more objects located at the target position. According to yet another alternative, this area can be selected according to a depth map of the scene around the position, and a predetermined depth interval: for example it may be desirable for the area to include the objects located at the target position and whose depth, in the scene, is included in an interval centered on the depth of the target position.

[0009] During a step 304, the image IM that is sharp everywhere is blurred everywhere, except in the area selected during step 302. This blurring can be carried out by any known technique. The image thus obtained is used as a new image, possibly after an optional image processing step 306.

[0010] In the example described with reference to FIGURE 3, the new image is obtained from the depth associated with the target position.

[0011] Of course, other embodiments are possible. For example, it is possible to identify an object located at the target position, or it is possible to identify a target area comprising the target position. Obtaining the new image can then be carried out as a function of said identified object, or of said identified target area. In this case, the image, which is clear everywhere, is blurred everywhere except in its part comprising the target object, or in its part corresponding to the target area.

[0012] FIGURE 4 is a schematic representation of a non-limiting exemplary embodiment of obtaining an image of a scene that is sharp throughout, which can be implemented in the present invention.

[0013] The method 400, shown in FIGURE 4, can be implemented to obtain an image of a scene, clear everywhere, and can be used, in the method according to the invention, and in particular in the method 100 of FIGURE 1, to generate a new image of the scene.

[0014] More particularly, the method 400, shown in FIGURE 4, can be implemented to obtain the sharp image everywhere INP used in step 300 of FIGURE 3.

[0015] In the example shown in FIGURE 4, the sharp image throughout is obtained from a stack of images of the scene, comprising several images, each representing the scene with a different depth of field.

[0016] The PIL image stack can be obtained either by an imaging device implementing a focus bracketing technique, or by calculation, as described above.

[0017] The method 400 comprises a step 402 which extracts, for each image from the PIL image stack, the sharpness zone of said image.

[0018] Extracting the sharpness zone of an image can be done in different ways. For example, we build a local focus estimator (one possible implementation is to assimilate it to an energy / variance / entropy operator) that we calculate over the entire surface of all the images. We use the hypothesis that for a given image zone, the estimator is maximal when the image is sharpest. Therefore, we determine for each image in the stack, the region(s) in which the value given by the local estimator is the largest of the entire stack of images. The content of these regions is directly copied into the desired result (taking into account geometric aberrations / focus breathing). By repeating the process for all the zones of all the images, we build a result that at each point contains the information of the image that was sharpest at that location.

[0019] The method 400 comprises a step 404, constructing a sharp image everywhere from the areas extracted during step 402. Indeed, once the sharp area of ​​each image in the stack has been extracted, all the sharp areas thus obtained can be concatenated together.

[0020] The concatenation can be carried out using any known technique. For example, the concatenation can be based on overlapping areas between the net areas, and possibly on objects of interest located in said net areas.

[0021] Alternatively, the sharp areas can be concatenated pixel by pixel so as to construct a sharp image throughout.

[0022] In an optional step 406, the clear image throughout is stored.

[0023] Preferably, but in no way limiting, the sharp image throughout may be enriched by depth data of the scene. In particular, depth data may be stored in association with at least one, in particular each, pixel or object of the scene. This, or These depth data can be obtained in each of the images in the PIL image stack, for example. Alternatively, the image stack can include an image, or a map, giving the depth of the scene for different points, and in particular each point in the scene.

[0024] FIGURE 5 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0025] The method 500 of FIGURE 5 can be used to produce a personalized display of a scene on a display medium from at least one image of said scene.

[0026] The method 500 of FIGURE 5 includes all of the steps of the method 100 of FIGURE 1.

[0027] The method 500 of FIGURE 5 may further comprise an optional step 502 of obtaining a stack of images of the scene, each image representing the scene over a limited depth of field such that only a portion of the scene is clearly represented on said image.

[0028] The image stack may be acquired by known focus bracketing techniques. Alternatively, the image stack may be obtained computationally, from one or more images of the scene.

[0029] For example, the image stack may be the image stack PIL of FIGURES 2 and 4.

[0030] In this case, and without loss of generality, step 108 of obtaining the new image can be carried out in accordance with the example described with reference to FIGURE 2.

[0031] FIGURE 6 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0032] The method 600 of FIGURE 6 can be used to produce a personalized display of a scene on a display medium from at least one image of said scene.

[0033] The method 600 of FIGURE 6 includes all of the steps of the method 100 of FIGURE 1.

[0034] The method 600 of FIGURE 6 may further comprise an optional step 602 of obtaining a clear image throughout.

[0035] The sharp image everywhere can be obtained by calculation from a stack of images. For example, the sharp image everywhere can be obtained by the method 400 of FIGURE 4.

[0036] The sharp image everywhere can be obtained by calculation from a single image of the scene.

[0037] In this case, and without loss of generality, step 108 of obtaining the new image can be carried out in accordance with the example described with reference to FIGURE 3.

[0038] FIGURE 7 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0039] The method 700 of FIGURE 7 can be used to produce a personalized display of a scene on a display medium from at least one image of said scene.

[0040] The method 700 of FIGURE 7 includes all of the steps of the method 600 of FIGURE 6.

[0041] The method 700 of FIGURE 7 further comprises the optional step 502 of obtaining a stack of images of the scene.

[0042] In all the examples described, step 502 can be implemented in the same device as that carrying out the display phase 102. According to an alternative, step 502 can be implemented in a device other than, and in particular remote from, the device which carries out the display phase 102.

[0043] In the examples described, step 602 can be implemented in the same device as that carrying out the display phase 102. Alternatively, step 602 can be implemented in the same device as that which carries out the step 502 of providing images, when it is different from the device carrying out the display phase 102. According to yet another alternative, step 602 can be implemented in a device other than, and in particular distant from, that which carries out the display phase 102, and / or that which carries out the step 502 of providing the stack of images.

[0044] FIGURE 8 is a schematic representation of a non-limiting exemplary embodiment of a device according to the present invention.

[0045] The device 800 of FIGURE 8 comprises a module 802 implementing the display phase of the method according to the invention, and in particular the display phase 102.

[0046] The module 802 comprises a module 804 displaying an image of the scene given to it as input. This module is in particular configured / programmed to carry out steps 104 and 110 of the display phase 102.

[0047] The module 802 further comprises a module 806 for detecting a target position on a currently displayed image. This module 806 takes as input a position of a pointer or a part of the user's body, and determines the target position on the current image. This module 806 is in particular configured / programmed to carry out step 106.

[0048] The module 802 further comprises a module 808 for obtaining the new image to be displayed, for example according to any of the techniques above. This module 808 is in particular configured / programmed to carry out step 108.

[0049] At least one of these modules may be a module independent of the others.

[0050] At least two of these modules can be integrated within the same module.

[0051] At least one of these modules may be a hardware module.

[0052] At least one of these modules may be a software module, such as a computer program.

[0053] At least one of these modules may be a combination of at least one software module, such as a computer program, and at least one hardware module.

[0054] In particular, at least one of the modules 804-808, or the module 802, can be integrated into an electronic chip, or even into an application installed in a user device.

[0055] The device 800 may further comprise, optionally, at least one display means, such as a display screen, touch-sensitive or not, or a means for projecting an image onto a support, for displaying an image of the scene.

[0056] Such a display means may be integrated into the device.

[0057] Such a display means is optional because the device may not include such a means. For example, the device 800 may be integrated into an apparatus that already has a display means and cooperate with said display means to display the image of the scene.

[0058] According to yet another alternative, the device 800 can be connected to an external display means, or to an external device having a display means or itself connected to a display means.

[0059] In the example shown, and without loss of generality, the display means may be an electronic display screen.

[0060] The device 800 may further comprise, optionally, at least one sensor 812 for detecting: - the position of a pointer, - the contact position of a part of the user's body such as a hand or finger, or - the position targeted by a part of the user's body such as a hand or finger, an eye, eyes or the user's face; to provide, or for the purpose of determining, the target position on a current image displayed on a display medium.

[0061] The sensor can be any type of sensor such as a camera, a lidar, a detection surface for example of the capacitive type, etc.

[0062] In the example shown in FIGURE 8, and without loss of generality, the sensor may be in the form of a detection surface, of the capacitive type, integrated into the display screen 810.

[0063] Optionally, the device 800 may further comprise at least one image acquisition means 814, such as a camera, or a camera module, comprising an optical lens and an image sensor, for acquiring an image or a stack of images of the scene.

[0064] Such an image acquisition means 814 may be integrated into the device, for example on a front face or on a rear face, or both.

[0065] Such an image acquisition means 814 is optional because the device may not comprise an image acquisition means 814. For example, the device 800 may be integrated into an apparatus which already has an image acquisition means such as a camera module and cooperate with said image acquisition means.

[0066] According to yet another alternative, the device 800 can be connected to an external image acquisition means, or to an external apparatus having an image acquisition means 814 or itself connected to an image acquisition means 814.

[0067] Optionally, the device 800 may further comprise at least one computing unit (not shown) for producing a sharp image everywhere from an image of the scene, or from a stack of images of the scene each having a different depth of field, as described above.

[0068] Optionally, the device 800 may further comprise at least one computing unit (not shown) for producing a stack of images of the scene, each with a different depth of field, from an image of the scene, as described above.

[0069] FIGURE 9a is a schematic representation of a non-limiting exemplary embodiment of an apparatus according to the present invention.

[0070] The apparatus 910 of FIGURE 9a comprises means configured to implement the invention, and in particular any one of the methods 100, 500, 600 and 700.

[0071] The apparatus 910 of FIGURE 9a may comprise a device according to the invention, and in particular the device 800 of FIGURE 8.

[0072] In the example shown in FIGURE 9a, the device 910 is a smartphone, or a tablet, comprising the device 800 of FIGURE 8. In particular, the device 910 comprises a display screen 810 equipped with a detection surface 812, for example capacitive, and at least one camera 814.

[0073] FIGURE 9b is a schematic representation of another non-limiting exemplary embodiment of an apparatus according to the present invention.

[0074] The apparatus 920 of FIGURE 9b comprises means configured to implement the invention, and in particular any one of the methods 100, 500, 600 and 700.

[0075] The apparatus 920 of FIGURE 9b may comprise a device according to the invention, and in particular the device 800 of FIGURE 8, without the camera 814.

[0076] In the example shown in FIGURE 9b, the apparatus 920 is a virtual reality, VR, headset, or an augmented reality headset, comprising the device 800 of FIGURE 8. In particular, the headset 920 comprises a display screen 810, a sensor (not visible in FIGURE 9b) for detecting the position aimed by an eye, or eyes, of the user on said display screen 810.

[0077] In the example shown in FIGURE 9b, the headset 920 does not include imaging means for capturing images of the scene. In this case, the image(s) of the scene to be displayed by the headset 920 are provided by another device to said headset 920.

[0078] Alternatively, the headset 920 may comprise at least one camera for capturing images of the scene in which it is located to display them on the screen 810, optionally after enriching said images, for example in the context of an augmented reality application.

[0079] FIGURE 9c is a schematic representation of a non-limiting exemplary embodiment of an apparatus according to the present invention.

[0080] The apparatus 930 of FIGURE 9c comprises means configured to implement the invention, and in particular any one of the methods 100, 500, 600 and 700.

[0081] The apparatus 930 of FIGURE 9c may comprise a device according to the invention, and in particular the device 800 of FIGURE 8.

[0082] In the example shown in FIGURE 9c, the apparatus is a medical imaging apparatus, such as an endoscope, an ultrasound apparatus, etc. comprising the device 800 of FIGURE 8. In particular, the medical imaging apparatus 930 comprises a display screen 810 equipped with a detection surface 812, for example capacitive. The medical imaging apparatus 930 further comprises an imaging means formed by a distal objective connected to an imaging module (not shown).

[0083] FIGURE 10 is a schematic representation of a non-limiting exemplary embodiment of a vehicle according to the present invention.

[0084] The vehicle 100 of FIGURE 10 comprises means configured to implement the invention, and in particular any one of the methods 100, 500, 600 and 700.

[0085] The vehicle 1000 of FIGURE 10 may comprise a device according to the invention, and in particular the device 800 of FIGURE 8.

[0086] In the example shown in FIGURE 10, the vehicle 1000 is a land vehicle, in particular a car, comprising the device 800 of FIGURE 8. In particular, the vehicle 1000 comprises a display screen 810 equipped with a detection surface 812, for example capacitive, arranged in the passenger compartment of the vehicle 1000. The vehicle 1000 further comprises at least one camera, for example arranged on the windshield of the vehicle 1000.

[0087] Of course, the invention is not limited to the examples which have just been described.

Claims

CLAIMS 1. Method (100; 500; 600; 700) of representing a scene on a display medium, said method (100; 500; 600; 700) comprising at least one iteration of a display phase (102) comprising the following steps: - display (104), on said display medium, of an image (IC), called current image, of said scene having a sharpness zone, - detection (106) of a position, called target position, on said current image (IC) in which a user is interested, - obtaining (108) a new image (IN) of said scene whose sharpness zone is adjusted according to said target position so as to produce a sharp display of said scene at said target position, and - display (110) of said new image (IN).

2. Method (100; 500; 600; 700) according to the preceding claim, characterized in that the step (108) of obtaining the new image comprises an on-the-fly generation (302, 304) of the new image, by calculation.

3. Method (100; 500; 600; 700) according to the preceding claim, characterized in that the source image (INP) is an image, previously obtained, and representing the scene clearly in all areas.

4. Method (100; 500; 600; 700) according to claim 1, characterized in that the step (108) of obtaining the new image comprises a step (206) of selecting the new image from a stack of images (PIL) of said scene, each having a different area of ​​sharpness.

5. Method (100; 500; 600; 700) according to the preceding claim, characterized in that the image stack (PIL) comprises at least one image captured by an imaging device.

6. Method (100; 500; 600; 700) according to any one of claims 3 or 4, characterized in that the image stack (PIL) comprises at least one image, obtained by calculation or simulation, from at least one other image.

7. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the display phase (102) further comprises, after identification of the target position, a step of identifying an object located at the target position, the obtaining of the new image being carried out as a function of said object.

8. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the display phase (102) further comprises a step of identifying a zone, called target, of the scene comprising the target position, the obtaining of the new image being carried out as a function of said target zone.

9. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the display phase (102) comprises a step of determining depth data of the scene at said target position, the new image being obtained as a function of said depth data.

10. Method (100;500;600;700) according to any one of the preceding claims, characterized in that the target position is entered manually by the user.

11. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the step of detecting the target position comprises detecting a zoomed region of the current image.

12. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the step (106) of detecting the target position comprises a detection, by at least one sensor, of a position targeted by a part of the user's body.

13. Method (100;500;600;700) according to any one of the preceding claims, characterized in that the display phase (102) is repeated for several images in an image stream, in particular a video.

14. Computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the method (100;500;600;700) according to any one of the preceding claims.

15. Device (800) comprising means configured to implement all the steps of the method (100; 500; 600; 700) according to any one of claims 1 to 13.

16. Apparatus (910; 920; 930) comprising: - a means (810) for displaying an image, and - at least one means (612) for detecting a target position, - at least one calculation means (802); configured to implement all the steps of the method (100; 500; 600; 700) according to any one of claims 1 to 13.

Images