Method of immersive display of a scene on the basis of images of the scene

Abstract

The invention relates to a method for representing an imaged scene, called the method, comprising the steps of: - defining an area, called the the target area, of the imaged scene on the basis of a current image of the scene, - determining, from among a stack of images of the scene, an image of the scene, called the reference image, in which the target area has the highest sharpness, the stack of images comprising at least two images of the scene having a distinct focal distance, - selecting, from among the images of the stack of images, areas of the imaged scene that are located outside the target area and that have a blur strength greater than a blur strength of the target area, which areas are called blurred areas, - reconstructing a composite image of the imaged scene by concatenating the target area of the reference image and the selected blurred areas.

Description

DESCRIPTION Method for immersively displaying a scene from images of said scene technical field

[0001] The present invention relates to a method for displaying images, particularly in real time.

[0002] The field of the invention is the field of displaying a digital image on a display medium, such as a display screen or a projection surface. State of the art

[0003] Stacks or streams of images acquired using state-of-the-art imaging devices do not reproduce the depth of the imaged scene.

[0004] In addition, elements of the scene, which may be of significant size and / or of significant interest in the imaged scene, are obscured.

[0005] When directly observing a scene, an observer's binocular vision allows them to extract depth information from the scene. However, the observer needs to focus their gaze on a specific object within the scene to obtain a clear perception of the scene centered on that object.

[0006] Observing the given object in the scene allows it to extract limited depth information from the scene and thus to understand its environment, in a limited way, around and near said given object in the scene.

[0007] Also, the observer of a scene scans said scene by successively focusing his attention on several objects located at different depths and in different positions in the scene in order to best understand his environment.

[0008] In practice, an observer of a scene will successively focus their attention on different objects in the scene, naturally adapting the focal distance to the depth of the object in the scene on which they are focusing their attention at any given moment.

[0009] State-of-the-art display methods offer a scene display based solely on the optical axis of the camera used to acquire the scene image. Therefore, only the object or area of ​​the scene located on the optical axis, on which focus was achieved during image acquisition, appears sharp in the displayed scene image.

[0010] In addition, state-of-the-art display methods offer a scene display with a single focus corresponding to the focus distance used when acquiring the scene image, i.e., the focus distance used when focusing centered on the object located on the optical axis of the camera used to acquire the scene image.

[0011] Also, displaying a scene according to the state of the art provides a limited and restricted representation of the scene.

[0012] Furthermore, such a display forces the observer of the image of the displayed scene to a unique observation of the scene which is conditioned by the conditions of acquisition of the image of the scene.

[0013] One object of the present invention is to remedy at least one of the drawbacks of the prior art.

[0014] Another aim of the invention is to provide a display method: - offering an enhanced immersive experience to the observer, and / or - offering the observer freedom to observe the enhanced scene, and / or - improving the user experience of display devices for a user, and / or - allowing for an increased field of view of the scene, and / or - allowing the display of a scene to be interactive. Description of the invention

[0015] For this purpose, the invention relates to a method of representing an image scene, called the method.

[0016] The process includes the step of defining an area, called the target area, of the imaged scene. Preferably, the target area is defined from a current image of the scene.

[0017] The process includes the step of determining, from one or more stack(s) of scene images, a scene image, called the reference image, in which the target area has the highest sharpness.

[0018] Preferably, the image stack(s) include at least two images of the scene with distinct focal distances.

[0019] The process includes the step of selecting, from the images in the image stack(s), one or each of the areas of the imaged scene located outside the target area, called blur areas, exhibiting a blur intensity greater than a blur intensity of the target area.

[0020] The process includes the step of reconstructing a composite image of the imaged scene.

[0021] Preferably, the composite image reconstruction step is implemented by concatenating the target area of ​​the reference image and the selected blur areas.

[0022] Preferably, the process allows for a dynamic rendering of a scene. Preferably, there is no need to obtain, and the process does not seek to obtain, at least in some embodiments, a blur of high quality or representative of the blur of human vision, or a blur conforming to a specific blurring rule.

[0023] Preferably, the method allows for a sharp area, particularly in the area on which the observer focuses their attention, and / or a better rendering for the observer, which is sufficient to provide an enhanced sense of immersion compared to viewing an image displayed with a single focus point exhibiting a sharp area (and blur) imposed by the focus used during image acquisition of the scene. Preferably, the method does not require, at least in some embodiments, that the blur intensity be significant or follow a specific law or gradient (the blur intensity can be constant around the target area or can vary randomly or be inhomogeneous).

[0024] Preferably, the process allows for the dynamic rendering of a scene, providing a better sense of immersion without the need to improve image sharpness.

[0025] The scene's image stack can either be a match or originate from a scene image stream.

[0026] The images in the scene image stack, or the images in the scene image stream, can be acquired continuously or intermittently (for example, by a series of image stacks).

[0027] Preferably, the process is implemented, or even more preferably the steps for determining the reference image are implemented, from one stack of images of the scene or from several stacks of images of the scene.

[0028] Preferably, in the rest of the description, the characteristics relating to "a stack of scene images" include and also apply to multiple stacks of scene images.

[0029] The method according to the invention can also be defined as a method for selectively and / or dynamically blurring an image of a scene, preferably of a displayed scene and / or of a current image of a displayed scene.

[0030] The imaged scene can be described as an image of the scene contained within, or as the scene as acquired by, the images in the image stack. Preferably, the imaged scene can be contained, at least in part, within the, preferably each of the, images in the image stack of the scene.

[0031] Preferably, the image stack, or at least two images of the scene with distinct focal lengths, includes the current scene image. Preferably, the image stack, or at least two images of the scene with distinct focal lengths, includes the reference image.

[0032] Preferably, the images in the image stack were acquired by the same device. Preferably, the device images the same scene. Preferably, the device includes at least one image acquisition device. The image acquisition device is typically a camera. The device may include a single image acquisition device or may include several separate image acquisition devices. Also, the images in the image stack may come from the same image acquisition device or from several separate image acquisition devices.

[0033] Preferably, the at least two images acquired, preferably each with a different focal length, can be acquired by a technique known in the state of the art called "focus bracketing".

[0034] "Sharpness" can be understood as: a value or information, contained in the pixels of the image, relating to or representative of sharpness.

[0035] Preferably, the image in the image stack from which at least one blur area is selected (during the selection step) can be the current image, the reference image, or another image (distinct from the current and reference images) in the image stack. In other words, preferably, one or more blur areas are selected from, or originate from, either the current image, the reference image, or one of the images (distinct from the current and reference images) in the image stack.

[0036] Preferably, each blur area is selected or comes from only one of the images in the image stack.

[0037] Preferably, the sharpness of the target area in the reference image is greater than the sharpness of the target area in the current image. Preferably, the sharpness of the target area in the reference image is greater than the sharpness of the target area in each of the other images in the image stack.

[0038] Preferably, the reference image is distinct from the current image. Preferably, and in other words, the image in the (determined) image stack in which the target area has the highest sharpness, and in particular whose target area has a sharpness greater than the sharpness of the target area in the current image, is not the current image or is distinct from the current image.

[0039] Preferably, the target area includes or corresponds to an object in the current image.

[0040] Preferably, the selection step can be defined as selecting, for each area of ​​the imaged scene located outside the target area, in one of the images in the image stack, the area of ​​the imaged scene located outside the target area having a blur intensity greater than the target area.

[0041] Preferably, it is understood that the target area of ​​the reference image and of the other image(s) in the image stack, distinct from the reference image, correspond(s) to the target area of ​​the current image.

[0042] Preferably, the target area corresponds to the same part or object of the scene for the reference image, for the current image, and for several or each of the images in the image stack.

[0043] In other words, the part of the object, the object, parts of several objects or objects of the scene included in the target area of ​​the current image correspond to the part of the object, the object, parts of several objects or objects of the scene included in the target area of ​​the reference image and one or more other images in the image stack, distinct from the reference image.

[0044] The target area may be absent or not appear in, or not be included in or be outside the field of view of, one or more of the images in the image stack (distinct from the reference image and / or the current image).

[0045] Preferably, the target area of ​​the reference image is unchanged.

[0046] Preferably, the target area of ​​the reference image constitutes or forms the sharp part of the reconstructed composite image.

[0047] Preferably, the selected blurred areas constitute or form the blurred part of the reconstructed image.

[0048] Preferably, the focal distance of the current image of the scene is different from the focal distance of at least one other image in the image stack.

[0049] In this application, "focusing distance of an image (considered)" may be understood to mean the focusing distance of the imaging system (or the focusing distance of the optical lens of the imaging system) used to acquire the image (considered).

[0050] Preferably, the process includes a step of increasing the intensity of blur, called the blurring step, of at least one of the selected blur areas.

[0051] It can be understood as "a step consisting of increasing the intensity of blur": a blurring step.

[0052] Preferably, the step of blurring at least one selected blur area includes or consists of modifying at least one selected blur area, preferably further increasing the blur intensity of all or part of at least one selected blur area.

[0053] The blurring step is optional. It can be implemented to further enhance the image by adding extra blur for a more immersive experience.

[0054] The blurring step can be implemented, for example, in the unlikely event that none of the areas in any of the images in the image stack located outside the target area have a blur intensity greater than the blur intensity of the target area. However, at least some of the areas in the reference image located outside the target area should have a blur intensity greater than the blur intensity of the target area. Furthermore, given the difference in focal length between the images in the image stack, some image areas in the other images in the image stack should also have a blur intensity greater than the blur intensity of the target area.

[0055] Preferably, for a given blur area, preferably for a given blur area being selected, the selection step is carried out, preferably for one, several or each selected area(s), depending on the desired blur intensity.

[0056] Preferably, for a given blur area, preferably for a selected blur area, the blurring step is carried out, preferably for one, several or each selected area(s), depending on the desired blur intensity.

[0057] Preferably, the intensity of blur to be obtained increases with and / or is a function of the distance separating the target area from the area of ​​blur considered.

[0058] Preferably, the desired level of blur should be increased: - with, when, relatively, proportionally or as a function of the distance separating, in the image plane, the target area from the considered blur area, and / or - depending on, relatively, proportionally or depending on a depth difference between the target area (or, the, several or each of the objects in the target area) and the considered blur area (or, the, several or each of the objects in the considered blur area).

[0059] The term "image plane" can be understood to mean the plane of the reference image and / or the selected images and / or the images in the image stack. It can also be understood to mean the plane of the optical sensor of one or more imaging systems of one or more imaging devices from which the images in the image stack were acquired.

[0060] Preferably, selecting a blur area based on a desired blur intensity increases the user's perception of focus and / or sharpness of the target area.

[0061] Preferably, blurring at least one area of ​​blur according to a desired blur intensity increases the feeling of focus and / or blur around the target area for the user.

[0062] It can be understood by blur intensity to be obtained: the desired, desired or optimal intensity that can follow or be described by a particular law or function (for example describing the evolution of blur intensity within the image or relative to the target area).

[0063] Preferably, for each area of ​​the imaged scene and for a considered blur area, the selection includes or consists of selecting the image from the image stack (which may be the current image or the reference image or another image (distinct from the current image and the reference image) from the image stack) whose blur intensity of the considered blur area has the minimum deviation from the desired or optimal blur intensity to be obtained.

[0064] In this application, it can be understood as "in view of": or based on or taking into account or in relation to.

[0065] Preferably: the determination step is performed based on the focal length of the images in the image stack, preferably based on the focal length of the reference image, preferably based on a difference between the focal length of the reference image and the focal length of the images in the image stack, and / or the selection of all or part of the blurred areas is performed based on the focal length of the images in the image stack, preferably based on the focal length of the reference image, preferably based on a difference between the focal length of the reference image and the focal length of the images in the image stack, and / or the blurring of at least one of the blur areas, preferably of at least one selected blur area, is carried out according to the focal length of the images in the image stack, preferably, for a given blur area, according to the focal length of the image in the image stack including the given blur area, or the focal length of the image in the image stack in which the given blur area was selected.

[0066] Preferably, the step of determining and / or selecting all or part of the blur areas based on the focal length of the images in the image stack makes it easier and / or simpler and / or faster to calculate and / or reduce the resources needed to implement the selection step.

[0067] Preferably, blurring at least one of the blur areas based on the focal length of the images in the image stack improves the blur rendering.

[0068] Preferably: the determination step is performed based on a position, preferably a relative position, of the target area in the current image and / or a position, preferably a relative position, of the target area in one, several or each of the images in the image stack (distinct from the current image), and / or The selection of all or part of the blur areas is performed based on a position, preferably a relative position, of the target area in the reference image and / or a position, preferably a relative position, of the target area in one, several, or each of the images (distinct from the reference image but which may be the current image) of the image stack, preferably in each of the images of the image stack (distinct from the reference image and / or which may be the current image) including the target area, and / or for a given blur area, the selection of all or part of the blur areas is performed based on a position, preferably a relative position, of the considered blur area in the reference image (in the case where the reference image includes the considered blur area) and / or in the image to be reconstructed and / or a position, preferably the relative position.of the blur area considered in the image to be reconstructed and / or in the image (distinct from the reference image and / or which may be the current image) of the image stack containing the blur area considered (in the case where said image of the image stack contains the blur area considered) or of the image of the image stack in which the blur area considered has been selected (in the case where said image of the image stack contains the blur area considered), and / or, The blurring step is performed according to: • a position, preferably a relative position, of the target area in the reference image and / or, for a given blur area, a position, preferably the relative position, of the target area in the image (distinct from the reference image and / or which may be the current image) of the image stack containing the considered blur area (in the case where said image of the image stack contains the target area) or of the image of the image stack in which the considered blur area has been selected (in the case where said image of the image stack contains the target area), and / or • of a position, preferably a relative position, of the blur zone in the reference image (in the case where the reference image includes the blur zone in question) and / or in the image to be reconstructed and / or of a position, preferably the relative position, of the blur zone in the image to be reconstructed and / or in the image (distinct from the reference image and / or which may be the current image) of the image stack including the blur zone in question (in the case where said image of the image stack includes the blur zone in question) or of the image of the image stack in which the blur zone in question has been selected (in the case where said image of the image stack includes the blur zone in question).

[0069] Preferably: The determination step is performed based on the sharpness of the target area in the current image and / or in each of the images (distinct from the current image) in the image stack, and / or The selection of all or part of the blur areas is performed based on the sharpness of the target area in the reference image and / or based on the sharpness of the target area in one, several, or each of the images (distinct from the reference image and / or which may be the current image) in the image stack, preferably in each of the images in the image stack (distinct from the reference image and / or which may be the current image) containing the target area, and / or The blurring step is performed according to: • the sharpness of the target area in the reference image and / or, for a given blur area, the sharpness of the target area in the image (distinct from the reference image and / or which may be the current image) of the image stack containing the considered blur area (in the case where said image of the image stack contains the target area) or of the image of the image stack in which the considered blur area was selected (in the case where said image of the image stack contains the target area), and / or • for a given blur area, the sharpness of the blur area in the reference image and / or the sharpness of the blur area in the image (distinct from the reference image and / or which may be the current image) of the image stack including the blur area (in the case where said image of the image stack includes the blur area) or of the image of the image stack in which the blur area has been selected (in the case where said image of the image stack includes the blur area).

[0070] Preferably: the selection of all or part of the blur areas is implemented based on depth information contained in a pixel, in several pixels, or in one or more groups of pixels of the target area, preferably in each of the images in the image stack (distinct from the reference image and / or which may be the current image) comprising the target area, and / or The blurring step is implemented according to: • depth information contained in a pixel, several pixels, or one or more groups of pixels in the target area of ​​the reference image and / or, for a given blur area, depth information contained in a pixel, several pixels, or one or more groups of pixels in the target area of ​​the image (distinct from the reference image and / or which may be the current image) of the image stack containing the considered blur area (in the case where said image of the image stack contains the target area) or of the image of the image stack from which the considered blur area was selected (in the case where said image of the image stack contains the target area), and • for a given blur area, depth information contained in a pixel,in several pixels or in one or more groups of pixels of the blur area in the reference image and / or of depth information contained in a pixel, in several pixels or in one or more groups of pixels of the blur area in the image (distinct from the reference image and / or which may be the current image) of the image stack comprising the blur area in question (in the case where said image of the image stack includes the blur area in question) or of the image of the image stack in which the blur area in question has been selected (in the case where said image of the image stack includes the blur area in question).

[0071] Preferably, depth information contained in a pixel, several pixels or one or more group(s) of pixels is understood to mean: the depth of the point, object or group of objects in the scene.

[0072] Preferably, the step of determining and / or selecting all or part of the areas of blur based on depth information makes it easier and / or simpler and / or faster and / or reduces the resources needed to implement the selection step.

[0073] Preferably, blurring at least one of the blur areas based on depth information improves the blur rendering.

[0074] Preferably, the step of blurring at least one selected blur area is implemented according to a standard blur matrix.

[0075] Preferably, the typical blur matrix defines, for each pixel or group of pixels or areas of the reconstructed image or the image to be reconstructed, a variation or function or gradient of desired or optimal blur intensity to be obtained.

[0076] Preferably, the selection of blur areas is carried out according to a standard blur matrix.

[0077] Preferably, the standard blur matrix defines, for each pixel or group of pixels or areas of the reconstructed image or the image to be reconstructed, a desired or optimal blur intensity to be obtained.

[0078] Preferably, the blur matrix according to which the blurring step is implemented is identical to the blur matrix according to which the selection step is implemented.

[0079] Preferably, for one, several or each area of ​​the imaged scene, and for a considered blur area, the selection step includes or consists of selecting the image from the image stack whose blur intensity of the considered blur area has the minimum deviation from the blur intensity of the typical blur matrix, typically with respect to the coefficient of the blur matrix corresponding to or applied to the considered blur area.

[0080] Preferably, the process includes a step of calculating and / or determining the typical blur matrix.

[0081] Preferably, the calculation and / or determination step can be implemented by a data processing device.

[0082] Preferably, the standard blur matrix is ​​determined and / or calculated based on: a position, preferably a relative position, of the target area in the current image and / or in the reference image and / in at least one of the images in the image stack, distinct from the current image and the reference image and / or in the image to be reconstructed, and / or a position, preferably a relative position, of one, several, or each blur area in the current image and / or in the reference image and / or in at least one of the images in the image stack, distinct from the current image and the reference image, and / or in the image to be reconstructed, and / or the focal length of the current image and / or the reference image and / or in at least one of the images in the image stack, distinct from the current image and the reference image, and / or in the image to be reconstructed, and / or the sharpness of the target area in the current image and / or in the reference image and / or in at least one of the images in the image stack, distinct from the current image and the reference image, and / or in the image to be reconstructed, and / or depth information contained in one pixel, several pixels, or one or more groups of pixels in the target area and / or at least one blur area, and / or a reference or model fuzzy matrix, and / or a digital aperture or apertures of one or more optical lenses of an imaging device from which the image stack(s) were acquired, and / or a point spreading function, known as PSF, of one or more imaging systems of an imaging device from which the image stack was acquired.

[0083] Preferably, the standard blur matrix and / or the reference blur matrix includes at least two sets, each comprising a coefficient distinct from the desired or optimal blur intensity to be obtained.

[0084] The device may include a single imaging system from which the images in the image stack are or have been acquired.

[0085] Preferably, in the case where the device includes a single imaging system, at least two images are acquired, preferably each of the images can be acquired with a different focal length (in particular by focus bracketing).

[0086] Each optical lens of the device can belong to a separate imaging system, typically a camera or camera module. Preferably, each imaging system comprises an optical lens and an optical sensor, typically a photographic sensor or a photodetector. Preferably, at least two of the device's imaging systems, and preferably each of the imaging systems, have different focal lengths.

[0087] Preferably, each image of the scene or each of the images in the image stack or each of the at least two images of the scene was acquired by one of the imaging systems of the device.

[0088] Preferably, the device remains stationary during image stack acquisition. When the device has a single image acquisition system, the scene images are acquired from the same position or viewpoint. When the device has multiple image acquisition systems, the scene images are acquired from very close positions (a few centimeters apart), considered to constitute the same position and viewpoint, without impacting the field of view, the position of the imaged objects within the scene, or the imaged objects themselves (i.e., without requiring image stack processing to recognize or modify, particularly the shape, between corresponding objects).

[0089] The model blur matrix can define or represent a variation in blur intensity to be obtained conforming to or corresponding to the blur of human vision or to an average or standard blur generated by a standard imaging system.

[0090] Preferably, the process includes a step of displaying, on a display medium, the current image and / or the reconstructed image.

[0091] Preferably, the scene display step is implemented prior to the target area definition step.

[0092] Preferably, the target area is defined, selected, detected and / or identified from the currently displayed image.

[0093] Preferably, the reconstructed image displayed can be, or may constitute, the current image.

[0094] Preferably, the reconstructed image displayed can be substituted for the current image.

[0095] Preferably, the composite image reconstruction step is implemented from the reference image.

[0096] Preferably, the reconstruction step includes: a substitution of at least one area of ​​the reference image, located outside the target area of ​​the reference image, by at least one blur area selected from another image in the image stack, distinct from the reference image, and / or an addition of at least one blur area, called an additional area, selected from another of the images in the image stack, distinct from the reference image; the area of ​​the imaged scene corresponding to the additional blur area being absent from the reference image.

[0097] Preferably, the definition of the target area includes an identification of the target area in the current image, preferably the currently displayed image, by an observer of the current image.

[0098] Preferably, the target area is defined from at least one image of the observer's eyes.

[0099] Preferably, the target area corresponds to the area of ​​the currently displayed image, called the attention area, on which the observer's gaze is focused.

[0100] Preferably, the target area is defined by a means of detection, for example one or more cameras or one or more camera modules.

[0101] Preferably, the target area can be defined or determined by eye tracking.

[0102] Preferably, the definition of the target area includes the identification, preferably manually and / or voluntarily, of the target area by the observer of the current image.

[0103] According to the invention, a data processing device is also proposed. The data processing device comprises means arranged and / or programmed and / or configured to implement the method according to the invention.

[0104] According to the invention, a computer program is also proposed. The computer program comprises instructions which, when the program is executed by a computer, cause the computer to implement the method according to the invention.

[0105] According to the invention, a computer-readable medium is also proposed comprising instructions which, when executed by a computer, lead the computer to implement the process according to the invention.

[0106] The device according to the invention can be, or be integrated into, any type of device such as a smartphone, a tablet, a computer, a calculator, a processor, a computer chip, programmed to implement the method according to the invention, for example by executing the computer program according to the invention.

[0107] The device may not include a means of image acquisition.

[0108] In this case, the device is used to display one or more images acquired by another device.

[0109] Alternatively, the device may include an image acquisition means, such as a camera or camera module.

[0110] In this case, the device can be used to display one or more images acquired by said device or by another device.

[0111] In particular, the device can be a user device such as a smartphone, tablet, etc., comprising a display screen.

[0112] In this case, the detection means may be or include the touch surface, in particular integrated into, or associated with, the display screen of said device.

[0113] In particular, the device may be a computer-type user device, including a display screen.

[0114] In this case, the detection means may be or may include a touch surface, in particular integrated into, or associated with, the display screen of said computer, or a pointer moved for example by a mouse, or a directional pad of said computer.

[0115] In particular, the device could be a television.

[0116] In this case, the detection method may be a camera integrated into said television, detecting the gaze and head position of the observer, or a pointer moved for example by a remote control of said television.

[0117] In particular, the device may be a virtual reality or augmented reality headset comprising a display screen or a projector associated with a projection surface onto which each image is projected.

[0118] In this case, the detection means may be or may include a sensor, in particular optical, fitted to said helmet.

[0119] Of course, the device according to the invention is not limited to the examples just given. Brief description of the FIGURES

[0120] The invention will be better understood upon reading the following description, given solely by way of non-limiting example and made with reference to the accompanying drawings in which: - FIGURE 1 is a representation of a scene imaged by an imaging system, - Figure 2 is a curve illustrating a theoretical evolution curve of the inverse of sharpness as a function of depth in the imaged scene; - Figure 3 illustrates the curve showing the theoretical evolution of the inverse of sharpness according to Figure 2 and a real evolution curve of the inverse of sharpness within an image acquired by a real imaging system, as a function of the distance between the real imaging system and the imaged scene. -Figure 4 illustrates the theoretical evolution curve of the inverse of sharpness according to Figure 3, the actual evolution curve of the inverse of sharpness according to Figure 3, and an evolution curve, as a function of the depth in the imaged scene, of the inverse of sharpness to be obtained, as a function of the depth in the imaged scene. -Figure 5 illustrates the actual evolution curve of the inverse of sharpness according to Figures 3 and 4, a theoretical evolution curve of the inverse of sharpness as a function of depth in the imaged scene, and an evolution curve of the inverse of sharpness to be obtained as a function of depth in the imaged scene. -Figure 6 illustrates the actual evolution curve of the inverse of sharpness according to Figures 3 to 5, an example of the theoretical evolution of the inverse of sharpness as a function of depth in the imaged scene, and a curve of the evolution of the inverse of sharpness to be obtained as a function of depth in the imaged scene. -Figure 7 illustrates the actual evolution curve of the inverse of sharpness according to Figures 3 to 6, an example of the theoretical evolution of the inverse of sharpness as a function of the depth in the imaged scene and a curve of evolution of the inverse of sharpness to be obtained, as a function of the depth in the imaged scene.

[0121] It is understood that the embodiments described below are by no means exhaustive. In particular, variants of the invention may be conceived comprising only a selection of the features described below, isolated from the other features described, if this selection of features is sufficient to confer a technical advantage or to differentiate the invention from the prior art. This selection includes at least one preferably functional feature without structural details, or with only a portion of the structural details if that portion alone is sufficient to confer a technical advantage or to differentiate the invention from the prior art.

[0122] In particular, all the variants and embodiments described can be combined with each other if there are no technical obstacles to this combination.

[0123] In the figures and in the rest of the description, elements common to several figures retain the same reference. Detailed description of the FIGURES

[0124] According to the non-limiting embodiment, an example of implementation of the method for representing an image scene according to the invention, referred to as the method, is described.

[0125] The process includes a step of defining an area, called the target area, of the imaged scene from a current image of the scene.

[0126] The process also includes the step of determining a scene image, called a reference image, from a stack of scene images, in which the target area has the highest sharpness.

[0127] The image stack includes at least two images of the scene acquired with a distinct focal distance.

[0128] The process includes a step of selecting, from the images in the image stack, areas of the imaged scene located outside the target area, called blur areas, exhibiting a blur intensity greater than a blur intensity of the target area.

[0129] The process includes a step of reconstructing a composite image of the imaged scene by concatenating the target area of ​​the reference image with the selected blur areas.

[0130] With reference to FIGURES 1 to 7, an embodiment of the process according to the invention is described.

[0131] Figure 1 shows an example of a scene 1 imaged by a single imaging system Cl.

[0132] The imaging system Cl (or each imaging system) is, by way of non-limiting example, a camera Cl.

[0133] The Cl camera acquires a stream of images, preferably continuously.

[0134] In the illustration of picture scene 1 in FIGURE 1, scene 1 includes several objects located at different distances from camera Cl. Scene 1 includes an object 2 located at a distance z1 from camera Cl, an object 3 located at a distance z2 from camera Cl, an object 4 located at a distance z3 from camera Cl and an object 5 located at a distance z4 from camera Cl.

[0135] The z-axis (abscissa) represents the distance, originating from the position of the camera in question (Cl according to FIGURE 1), separating the camera from scene 1. The z-axis (abscissa) illustrates the distance separating scene 1, objects 2, 3, 4 and 5 of scene 1 according to FIGURE 1, from camera Cl. In other words, the z-axis represents the depth of the objects in scene 1.

[0136] The focal length of the camera (of the single camera Cl according to the embodiment shown in FIGURE 1) is denoted f. According to the invention, the focal length f can be understood as the distance between the camera in question, Cl according to FIGURE 1, and the plane of the scene exhibiting the best sharpness.

[0137] The camera's focusing distance Cl corresponds to the distance between the camera Cl and an object in scene 1.

[0138] According to this illustrative embodiment, the camera Cl comprises four focusing distances fl, f2, f3 and f4.

[0139] The target area and / or blur areas can correspond to an object (2, 3, 4 or 5) or a group of objects from the imaged scene 1.

[0140] In particular, the blur areas can correspond to all or part of the objects in the scene (distinct from the object(s) in the target area), for example to objects distinct from the background (the background can be defined as the objects in scene 1 having an "infinite" depth).

[0141] Also, with reference to FIGURE 1, once the target area is defined (for example, area 6), the blur areas can be limited or correspond, in particular or solely, to objects in scene 2, 3, 4 and 5 which can be objects of interest and / or objects distinct from the background.

[0142] An image stack can be described as consisting of a stack, preferably chronological, of images from an image stream. The images in an image stream (and therefore in the image stack) are acquired successively (at successive times t1, t2, t3...).

[0143] In general, the images in an image stream (and therefore in the image stack) are acquired successively at regular time intervals (typically every 0.8 seconds).

[0144] For example, the images in a given image stack constitute or form or are a set of successive images in an image stream.

[0145] For example, the number of images in an image stack can be chosen or selected (or, in other words, the number of images selected or chosen from an image stream to form the set of successive images in an image stream) so that the imaged scene is not, or only slightly, modified. In other words, the time lapse or interval during which the images in a given image stack were acquired is chosen so that the imaged scene is not, or only slightly, modified. In other words, the images in a given image stack (or the set of successive images in the image stream) primarily represent the same scene 1 (or are not too far apart in time) so that the objects in scene 1 (especially when one or more objects in scene 1 move during image acquisition) are in the same or immediately adjacent positions on the optical sensor(s) of the imaging system(s).

[0146] In the case of FIGURE 1, where scene 1 is imaged by a single camera Cl, at least two of the images of scene 1 are acquired with distinct focal lengths. In other words, preferably, two successive images of scene 1 (acquired at two successive or consecutive times tx and ty) within the image stack have different focal lengths (fx and fy). In other words, preferably, any given image (acquired at time tx), or preferably each given image, in the image stack has a focal length (fx) different from the focal length (fy) of the image that follows the one in question (the successive image acquired at time ty).

[0147] Thus, two successive images in the image stack have a different focal length.

[0148] In the case of FIGURE 1, the images in the image stack are acquired, for example, by varying or modulating continuously or cyclically the focal distance, i.e., by dynamically and / or continuously varying the focal distance of the camera's optical lens Cl.

[0149] However, the invention also includes the case where scene 1 is imaged by several imaging systems (or several cameras). In this case, the imaging systems are preferably arranged to image scene 1 from the same viewpoint (i.e., from primarily the same position). In this case, the method is implemented using several image stacks.

[0150] In cases where the process is implemented using multiple image stacks, each image stack can be acquired with a single focal length (f) or a constant focal length (f). The focal length (fx) of one of the image stacks is different from the focal length (fy, fz...) of one, several, or preferably each of the other image stacks. Therefore, two images from two distinct image stacks acquired at the same instant (or at sufficiently close instants that the scene 1 remains unchanged) will have different focal lengths.

[0151] In the case where the process is implemented from several image stacks, one, several or each of the image stacks can be acquired with a focal length (f) which varies or is modulated continuously or cyclically, that is to say by dynamically and / or continuously varying the focal length of the, of the or each of the optical lenses of the cameras.

[0152] Thus, when images from an image stack are acquired by continuously or cyclically varying or modulating the focusing distance of an optical lens in an imaging system, images from one or more image stacks can be acquired, for example: - by varying or dynamically modulating the focusing distance (fl, f4, f2, f4, f3, fl, f2...) in a random and / or sequential manner, or - by varying or dynamically modulating sequentially and / or incrementally the focus distance to a successive focus distance (e.g. (fl, f2, f3, f4, fl, f2, f3...)...), for example by focus bracketing, the focus distance of the optical lens of the camera or cameras.

[0153] The determination stage may include the selection of several reference images.

[0154] Each of the selected reference images may include a portion of the target area with the highest sharpness.

[0155] In this case, the process may include a step of concatenating the parts of the target area, each with the highest sharpness. In other words, the target area can be reconstructed (in a reconstruction step separate from the composite image reconstruction step) from several "subsets" or "sub-areas" of the target area, each originating from a different image in the image stack.

[0156] Sharpness can be contained within a single pixel, preferably each pixel, or within a group or block, preferably each block or group of pixels in an image. Sharpness can be information or a parameter relating to, comprising, or being a function of the entropy, energy, and / or variance of a pixel, or of a group or block of pixels.

[0157] The process may include the calculation, estimation, or determination of sharpness information.

[0158] The intensity of blur can be defined or correspond to the inverse of sharpness.

[0159] As a non-limiting example, depending on the embodiment, the sharpness may correspond to or be calculated from or be representative of the diameter of the PSF.

[0160] Furthermore, sharpness can be derived, estimated, calculated, or otherwise determined. By way of non-limiting example, sharpness can be determined by a local estimator (or operator), possibly a function of the camera's focal length, which is representative of or a function of energy, variance, focal length, and / or entropy. The estimator is maximal for the image(s) in which the target area is sharpest.

[0161] Also, we can determine, for each image in the stack or stacks of images, the value of the local estimator of the target area that is the largest among all the images in the stack or stacks of images.

[0162] Advantageously, the process includes a step of displaying, on a display medium, the current image and / or the reconstructed image.

[0163] According to this embodiment, the method is preferably iterative or comprises, preferably, an iteration of a sequence of steps. In particular, the method comprises an iteration of a sequence including, notably, the step of defining the target area, the step of determining the reference image, and the step of selecting the blurred areas.

[0164] According to this embodiment, the reconstructed image replaces the current image. The reconstructed image, obtained during a given iteration of the process, is displayed on the display medium. The reconstructed image thus displayed constitutes the current image used for implementing the process during the iteration following the one under consideration.

[0165] Advantageously, the method is implemented using one or more image streams displayed on a screen. The image stream(s) can be displayed in real time, that is, simultaneously with their acquisition by the camera(s). In this case, the current image is one of the images in the image stream(s). However, the method can also be implemented using one or more stored image streams, for example, on a data storage device.

[0166] The identification step can be implemented by a user of the video surveillance system observing the displayed image stream(s).

[0167] According to a non-limiting example, the blur area selection step and / or the blurring step is carried out according to a standard blur matrix.

[0168] The typical blur matrix defines, for each pixel, or for each block or group of pixels, of the image to be reconstructed, a blur intensity to be obtained.

[0169] Advantageously, the selection step and / or the blurring step can be implemented relative to or according to the typical blur matrix.

[0170] Thus, all characteristics relating to a blur intensity (to be obtained or reference) can be directly transposed to the blur matrix (type or reference) according to the invention, and vice versa.

[0171] By blur matrix, it can be understood as a matrix of determining or representative coefficients, or being associated or consistent with, by correspondence to the image or a part of the image, an intensity of blur to be applied or an intensity of blur to be obtained in each area, object, pixel or group(s) of pixels of the image.

[0172] Each coefficient of the matrix can correspond to, be transposed to, or be applied to a pixel or group of pixels in the image.

[0173] The intensity of blur can correspond to a full width at half maximum of a point dispersion function called PSF.

[0174] The blur intensity can be supplemented, if necessary, by other parameters defining, for example, the shape of the PSF function.

[0175] To obtain the blur, one can for example calculate the value of a considered pixel of the modified image by that obtained by the sum of the multiplications of the coefficients of the matrix representing the PSF with the pixels of the so-called current sharp image, in a neighborhood of the considered pixel.

[0176] Figure 2 illustrates a theoretical curve 7 showing the theoretical evolution of blur (or blur intensity) or the inverse of sharpness (1 / N) within an acquired image as a function of the distance between the imaging system and the imaged scene. Sharpness is denoted N.

[0177] According to the illustrative example in FIGURE 2, the theoretical curve 7 illustrates the theoretical evolution of the inverse of sharpness (1 / N) as it would be obtained within the image of scene 1 if it were acquired by an ideal camera Cl, with a focal length f2, as a function of the distance between the imaging system and the imaged scene 1.

[0178] With reference to the example illustrated in FIGURE 2, the target area 6 is presented, consisting of object 3 from scene 1.

[0179] In this case, the sharpness of object 3 is maximal for the images in the image stack acquired with the focal length f2 (compared to the sharpness of object 3 in the images acquired with the other focal lengths).

[0180] Thus, the sharpness of object 3 in the image stack images acquired with focal lengths f4 and fl is less than the sharpness of object 3 in the image stack images acquired with focal length f2, which is less than the sharpness of object 3 in the image stack images acquired with focal length f3.

[0181] The theoretical curve 7 shown in FIGURE 2 can be transposed into or converted into a theoretical fuzz matrix.

[0182] Such a theoretical blur matrix can constitute the typical blur matrix according to the invention.

[0183] Preferably, the theoretical blur matrix constitutes or corresponds to the reference blur matrix according to the invention.

[0184] With reference to FIGURE 3, the actual (or observed) curve of evolution of the inverse of sharpness 8 of the images obtained by the camera Cl with a focal length f3 is illustrated. It can be observed that the object 2 located at the distance zl exhibits an "infinite" blur intensity.

[0185] With reference to FIGURE 1, we consider that area 6, or object 3, of scene 1, is defined as target area 6. From here, the image(s) from the stack of images acquired by camera Cl with focus distance f3 will be determined as the reference images.

[0186] The theoretical blur intensity curve 7 is centered on the actual blur intensity curve 8 of the images obtained by the camera Cl with a focal length f3. In other words, the minimum of the theoretical blur intensity curve 7 is centered on, or corresponds to, the focal length f3. Thus, the theoretical blur intensity curve 7 shown in FIGURE 3 represents the theoretical blur intensity evolution 7 for images obtained with a (theoretical) imaging device having a focal length f3.

[0187] Figure 3 shows that the sharpness is mainly constant within the target area 6. In some cases where the sharpness varies noticeably or significantly within the target area 6, it may be considered to determine several reference images, from the image stack, each containing a sharp portion of the target area 6. In this case, the target area 6 can be reconstructed (in a reconstruction step separate from the composite image reconstruction step) from the different portions, for example by concatenating the different portions to obtain a sharp target area 6.

[0188] Thus, in particular, but not exclusively, in the case of FIGURE 1 where scene 1 is imaged by a single camera Cl, the process makes it possible to reduce the intensity of blur in the areas of the reconstructed image (or for the objects of scene 1 of the reconstructed image) located outside the target area 6 compared to the actual intensity of blur 8 of images obtained by the camera Cl with the same focal length f3.

[0189] This effect is achieved by using multiple images acquired with different focal lengths. This allows the selection, from among several images in the image stack with different focal lengths, of blurred areas with different focal lengths (and therefore different blur intensities for the same object in the scene). It is also possible to select blurred areas with a blur intensity lower than the actual blur intensity of the images obtained by the camera Cl with the same focal length f3.

[0190] In other words, with reference to FIGURE 3 illustrating an image acquired with a focal length of f3, the process makes it possible to obtain a reconstructed image in which the blurred areas exhibit a blur intensity lower than the actual blur intensity of the images obtained by the camera Cl with the same focal length of f3. Furthermore, obtaining such reconstructed images does not require time-consuming image processing (processing steps) and / or significant computing resources.

[0191] According to the non-limiting implementation method, the selection step can be implemented, for a given blur area, by selecting, from among the images in the image stack, the image whose blur area has a blur intensity greater than the intensity of the target area. Advantageously, for a given blur area, the selection step stops or ends at the first identified image in which the blur area has a blur intensity greater than the blur intensity of the target area.

[0192] The blur area in the identified image is concatenated with the reference image. In other words, the blur area in the identified image is substituted for the blur area in the reference image.

[0193] The composite image reconstruction step is implemented from the reference image.

[0194] The reconstruction step includes replacing at least one area of ​​the reference image, located outside the target area of ​​the reference image, with at least one blur area selected from another image in the image stack, distinct from the reference image.

[0195] Not all areas of the reference image that lie outside the target area are necessarily replaced by a selected area from another image in the image stack. Some areas of the reference image that lie outside the target area may have an intensity that matches (or is as close as possible to) the desired blur intensity.

[0196] If, for a given blur area, no image in the image stack has a blur intensity greater than the blur intensity of the target area; the given blur area is not modified in the reference image.

[0197] Since the process does not necessarily aim to produce a blur (around the target area 6) of a quality blur representative of human vision or a blur conforming to a specific blur rule, this embodiment allows for limiting the resources required to implement the process and / or accelerating the rendering of the reconstructed image. This enables the process to be implemented continuously, particularly during the display of a stream of images.

[0198] In the case where obtaining a quality blur is sought, the step of selecting the blur areas consists of selecting, among the images in the image stack, and for each of the objects 2, 4 and 5, the image for which the actual blur intensity of one of the objects considered is closest to, or presents the smallest deviation from, the theoretical blur intensity 7 that the object should present in the reconstructed image.

[0199] In some cases, the reference image might have a smaller field of view than one or more other images in the image stream(s). In this case, and optionally, the reconstruction step can include adding at least one blur area, called an additional area, selected from another image in the image stack, separate from the reference image. In this case, the area of ​​the imaged scene corresponding to the additional blur area is absent from the reference image.

[0200] According to one embodiment, the invention may include a step consisting of calculating and / or determining the typical blur matrix.

[0201] Advantageously, the typical blur matrix corresponds to or is equivalent to or represents the intensity of blur to be obtained.

[0202] Advantageously, the typical blur matrix corresponds to or is equivalent to or represents a reference blur intensity.

[0203] Advantageously, the typical blur matrix according to the invention, or the blur intensity to be obtained, can be obtained, calculated and / or determined, in particular by weighting, adjusting or modifying, the reference (or theoretical) blur matrix, or the reference blur intensity.

[0204] Advantageously, the typical blur matrix according to the invention, or the blur intensity to be obtained, can be obtained, calculated and / or determined, by combining the reference blur matrix, or the reference blur intensity, with one or more other blur matrix(ies), or one or more other blur intensities.

[0205] Obtaining, calculating, or determining the standard blur matrix, or the desired blur intensity, is not necessarily part of the process. Advantageously, obtaining, calculating, or determining the standard blur matrix, or the desired blur intensity, can be carried out independently of the process or prior to each of the other steps in the process. Indeed, once the camera characteristics are known (or for predetermined characteristics, sets, or sets of characteristics), it is possible to obtain, calculate, or determine the standard blur matrix, or the desired blur intensity.

[0206] With reference to FIGURE 4, a theoretical curve 7 of the evolution of the inverse of the sharpness of images obtained with an ideal imaging system having a focal length of f3 is illustrated.

[0207] A curve 9 is also shown representing the intensity of blur to be obtained.

[0208] According to the embodiment, the blur intensity curve to be obtained 9 is obtained from the theoretical evolution curve 7 of the blur intensity.

[0209] The blur intensity curve 9 to be obtained can be obtained by shifting or adjusting the minimum blur intensity value of the theoretical evolution curve 7.

[0210] According to the example shown in FIGURE 4, the theoretical evolution curve 7 is shifted so that the minimum blur intensity (the minimum) of the blur intensity curve 9 to be obtained (thus obtained) is equal to the minimum blur intensity (the minimum) of the actual blur intensity evolution curve 8 of the images obtained by the camera.

[0211] Alternatively or in combination, the blur intensity curve 9 to be obtained can be obtained by shifting or adjusting the position of the theoretical evolution curve 7 relative to the z-axis. In other words, in this case, the depth in scene 1 for which the minimum blur intensity of the theoretical evolution curve 7 is modified.

[0212] The blur intensity curve 9 to be obtained can also be obtained by linear (or affine) transformation of curve 7. For example, the linear transformation can be carried out relative or proportionally to the difference between the minimum of the actual (or observed) blur intensity evolution curve 8 and the minimum of curve 7.

[0213] The blur intensity curve to be obtained 9 can be obtained, from the theoretical blur intensity curve 7, by non-linear transformation(s).

[0214] According to a non-limiting embodiment, the process may include a step, called the blurring step, consisting of increasing the blur intensity of at least one of the selected blur areas.

[0215] Depending on the embodiment, the blurring step can be implemented for one, several or each of the blur areas, for example subsequent to the selection step or subsequent to the reconstruction step.

[0216] In one embodiment, the blurring step is implemented according to the standard blur matrix. From there, the blur intensity to be applied is implemented according to the desired blur intensity.

[0217] The desired blur intensity and the corresponding typical blur matrix can be used to implement the selection step and / or the blurring step.

[0218] The typical blur matrix or the reference blur matrix can be deduced and / or extrapolated and / or determined and / or calculated from the blur intensity to be obtained or the theoretical blur intensity.

[0219] Thus, the typical blur matrix advantageously includes, reflects or integrates the amount of blur to be applied (particularly during the blurring step) or to be taken into account during the implementation of the process (particularly the selection step).

[0220] The blurring step can be implemented, preferably subsequently, on the image thus reconstructed.

[0221] Alternatively or in combination with the blurring step, the blur intensity to be obtained and / or the typical blur matrix can be estimated, calculated and / or determined from a reference blur intensity, or a reference blur matrix.

[0222] With reference to FIGURES 5 to 7, a theoretical curve, respectively 71, 72, 73, of the evolution of the inverse of sharpness is illustrated. The actual evolution curve of the blur intensity 8 of the images obtained by the camera Cl is also shown.

[0223] The minimum blur intensity of the theoretical curves 7 of evolution of the inverse of sharpness are shifted relative to or as a function of the z-axis.

[0224] The minimum blur intensity of the theoretical curves 7 of evolution of the inverse of sharpness are shifted relative to or as a function of the target area 6.

[0225] In particular, for each of FIGURES 5 to 7, the theoretical curves 71, 72, 73 of evolution of the inverse of sharpness are shifted, relative to or as a function of the z-axis, so that the depth in scene 1 for which the minimum blur intensity of the theoretical evolution curves 71, 72, 73 corresponds or coincides with the average depth of the target area 6.

[0226] With reference to Figures 5 to 7, curves 91, 92, and 93 are also shown, representing the desired blur intensity. According to this embodiment, the desired blur intensity curves 91, 92, and 93 are derived from the respective theoretical blur intensity curves 71, 72, and 73.

[0227] Advantageously, target area 6 of the reference image is not modified. Therefore, target area 6 (and thus the sharpness of target area 6) of the reference image will correspond to the target area of ​​the reconstructed image.

[0228] With further reference to FIGURES 5 to 7, we note that the blur intensity to be obtained 91, 92, 93 in the blur zones (outside the target zone 6) is less than the blur intensity 8 of the reference image.

[0229] The minimum blur intensity of the theoretical evolution curves 71, 72 are greater, by the same value, compared to the minimum blur intensity of the actual evolution curve of the blur intensity 8 of the images obtained by the camera Cl.

[0230] In this way, the blur intensity of the reconstructed image increases more rapidly in the plane of the (reconstructed) image near the target area 6. In particular, the blur intensity of the reconstructed image increases more rapidly when the depth increases or decreases, for example relative to the depth of the target area 6, advantageously in the reference image, and / or relative to the focal distance of the camera used to acquire the reference image.

[0231] The theoretical evolution curve 71 (and therefore the desired blur curve 91) approaches an asymptote as one moves away (as the depth increases or decreases) from the target area 6. According to the non-limiting example in FIGURE 5, the theoretical evolution curve 71 (and therefore the desired blur curve 91) approaches an asymptote with a small slope, which may even tend towards zero. This limits the intensity of the blur in the reconstructed image as the depth increases or decreases relative to the target area 6.

[0232] The theoretical evolution curve 72 (and therefore the blur curve to be obtained 92) evolves in an increasing manner when moving away (when the depth increases or decreases) from the target zone 6. However, it evolves less pronouncedly in the vicinity of or near the target zone 6 compared to the theoretical curve 71.

[0233] According to the non-limiting example in FIGURE 6, the theoretical evolution curve 72 (and therefore the desired blur curve 92) evolves towards an oblique asymptote. This allows for a more significant increase in the blur intensity of the reconstructed image as the depth increases or decreases relative to the target area 6.

[0234] The minimum blur intensity of the theoretical evolution curve 73 is greater than the minimum blur intensity of the actual evolution curve of the blur intensity 8 of the images obtained by the camera Cl. The minimum blur intensity of the theoretical evolution curve 73 is greater than the minimum blur intensity of the theoretical evolution curves 71, 72.

[0235] In this way, the blur intensity of the reconstructed image increases more rapidly and relatively markedly in the plane of the (reconstructed) image near the target area 6. In particular, the blur intensity of the reconstructed image increases more rapidly when the depth increases or decreases, for example relative to the depth of the target area 6, advantageously in the reference image, and / or relative to the focal distance of the camera used to acquire the reference image.

[0236] The theoretical evolution curve 73 (and therefore the blur curve to be obtained 93) evolves in an increasing manner when moving away (when the depth increases or decreases) from the target area 6. It evolves more pronouncedly in the vicinity of or near the target area 6 compared to the theoretical curves 71, 72. The theoretical evolution curve 93 therefore offers a blur rendering closer to that offered by a real lens (especially when it has a shallow depth of field).

[0237] According to the non-limiting example in FIGURE 7, the theoretical evolution curve 73 (and therefore the desired blur curve 93) approaches an oblique asymptote. The theoretical evolution curve 73 (and therefore the desired blur curve 93) approaches an asymptote with a high slope. This allows for a more significant increase in the blur intensity of the reconstructed image as the depth increases or decreases relative to the target area 6.

[0238] According to the invention, a data processing device is also proposed comprising means arranged and / or programmed and / or configured to implement the process according to the invention.

[0239] According to the invention, a computer program is also proposed comprising instructions which, when the program is executed by a computer, lead the computer to implement the process according to the invention.

[0240] According to the invention, a computer-readable medium is also proposed comprising instructions which, when executed by a computer, lead the computer to implement the process according to the invention.

[0241] Of course, the invention is not limited to the examples just described, and many modifications can be made to these examples without departing from the scope of the invention. Thus, in combinable variations of the embodiments described above: - according to an advantageous, but not limiting, embodiment, the step of selecting all or part of the blur areas is performed based on, and / or the blurring step, and / or the blur intensity to be achieved is based on, and / or the blur intensity to be achieved is calculated and / or determined based on: • the blur intensity (or the sharpness or the inverse of the sharpness) of one, several, or each of the images in the image stack(s), and / or • the desired level of blur intensity (or the desired sharpness or inverse sharpness), and / or • the focal length of one, several, or each of the images in the image stack(s); preferably the focal length of the reference image and / or the focal length of the image(s) containing the selected areas, and / or • the depth information contained in one, several, or each pixel, or in one, several, or each block or group of pixels, of one, several, or each of the images in the image stack(s); preferably the depth information contained in one, several, or each pixel, or in one, several, or each block or group of pixels, of the specified image(s) and / or the selected image(s), and / or • the sharpness of the target area in one, several, or each pixel, or in one, several, or each block or group of pixels of one, several, or each of the images in the image stack(s); preferably, the sharpness of one, several, or each pixel, or the sharpness of one, several, or each block or group of pixels, of the specified image(s) and / or the selected image(s), and / or - advantageously, for a given blur zone, the step of selecting all or part of the blur zones is carried out as a function of, and / or the blurring step, and / or the intensity of blur to be obtained is a function of or increases as a function of or relative to, and / or the intensity of blur to be achieved is calculated and / or determined as a function of, for a given blur zone: • a difference in depth between the target area and the considered blur area, and / or • a considered selected image, a depth difference between the considered blur area in the determined image (or the object or objects in scene 1 included in or constituting the considered blur area of ​​the determined image) and the considered blur area in the selected image (or the object or objects in scene 1 included in or constituting the considered blur area of ​​the selected image), and / or • For a given selected image, a difference between a determined focal length and the selected image, and / or - advantageously, the determination step and / or the blurring step, is implemented as a function of: • the focal length of one, several, or each of the images in the image stack(s), and / or • the sharpness (or blur intensity) of the target area in the current image and in one, several, or preferably each of the images in the image stack, and / or - Advantageously, the typical fuzz matrix can be calculated and / or determined from: • the desired level of blur (or sharpness, or the inverse of sharpness), and / or • the focal length of one, several, or each of the images in the image stack(s); preferably the focal length of the reference image and / or the focal length of the image(s) containing the selected areas, and / or • the depth information contained in one, several, or each pixel, or in one, several, or each block or group of pixels, of one, several, or each of the images in the image stack(s); preferably the depth information contained in one, several, or each pixel, or in one, several, or each block or group of pixels, of the specified image(s) and / or the selected image(s), and / or • the sharpness of the target area in one, several, or each pixel, or in one, several, or each block or group of pixels of one, several, or each of the images in the image stack(s); preferably, the sharpness of one, several, or each pixel, or the sharpness of one, several, or each block or group of pixels, of the specified image(s) and / or the selected image(s), and / or • the numerical aperture(s) of the optical lens(es) from which the images in the image stack were acquired, and / or • a point spreading function, known as PSF, of the imaging system(s) from which the images in the image stack were acquired, and / or Advantageously, the blurring step includes increasing the blur intensity of a considered blur area: • as a function of, relative to, or proportionally to the distance separating, in the image plane, the target area from the considered blur area, and / or • depending on, relative to, or proportionally to a depth difference between the target area and the considered blur area, and / or • the focusing distance of the images in the image stack, and / or - advantageously, the blurring step is performed based on: • the sharpness of the target area in the reference image and / or, for a given blur area, the sharpness of the target area in the image stack containing the blur area, and • for a given blur area, the sharpness of the blur area in the reference image and / or the sharpness of the blur area in the image stack containing the blur area, and / or • depth information contained in a pixel, several pixels, or one or more groups of pixels in the target area of ​​the reference image and / or, for a given blur area, depth information contained in a pixel, several pixels, or one or more groups of pixels in the target area of ​​the image stack containing the given blur area, and • for a given blur area, depth information contained in a pixel, several pixels, or one or more groups of pixels of the blur area in the reference image and / or depth information contained in a pixel, several pixels, or one or more groups of pixels of the blur area in the image of the image stack comprising the blur area, and / or The determination step and / or the blurring step is performed according to the focusing distance of the images in the image stack, and / or the step of defining the target area includes identification of the target area in the current image by an observer of the current image, and / or The target area is defined, based on at least one image of the observer's eyes, the target area corresponding to the area of ​​the currently displayed image, called the area of ​​attention, on which the observer's gaze is focused, and / or The step of defining the target area includes the identification of the target area by the observer of the current image.

[0242] In particular, all the variants and embodiments described can be combined with each other if there are no technical obstacles to this combination.

Claims

DEMANDS 1. A method of representing a pictorial scene, called a method, comprising the steps of: - define an area, called the target area, of the imaged scene based on a current image of the scene, - to determine, from a stack of images of the scene, a scene image, called the reference image, in which the target area has the highest sharpness; the image stack includes at least two scene images with distinct focal distances, - select, from the images in the image stack, areas of the imaged scene located outside the target area, called blur areas, exhibiting a blur intensity greater than a blur intensity of the target area, reconstruct a composite image of the imaged scene by concatenating the target area of ​​the reference image and the selected blur areas.

2. A method according to the preceding claim, comprising a step of increasing the blur intensity of at least one of the selected blur areas.

3. A method according to the preceding claim, wherein, for a given blur area, the selection step and / or the blurring step is performed according to a desired blur intensity; the desired blur intensity increases: - with the distance separating, in the image plane, the target area from the considered blur area, and / or - depending on a difference in depth between the target area and the considered blur area.

4. A method according to claim 2 or 3, wherein the step of determining and / or selecting all or part of the blur areas and / or the blurring step is carried out as a function of the focal distance of the images in the image stack.

5. A method according to any one of the preceding claims, wherein the step of determining the reference image is performed as a function of a position of the target area in the current image and / or in one, several or each of the images in the image stack.

6. A method according to any one of the preceding claims, wherein the selection of all or part of the blurred areas is performed as a function of a position of the target area in the reference image and / or a position of the target area in one, several or each of the images in the image stack.

7. A method according to any one of the preceding claims, wherein, for a given blur zone, the selection of all or part of the blur zones is carried out as a function of a position of the considered blur zone in the reference image and / or a position of the considered blur zone in the image of the image stack comprising the considered blur zone.

8. A method according to any one of the preceding claims, wherein the determination step is carried out based on the sharpness of the target area in the current image and in each of the images in the image stack.

9. A method according to any one of the preceding claims, wherein the selection of all or part of the blur areas is carried out according to the sharpness of the target area in the reference image and in one, several or each of the images in the image stack.

10. A method according to any one of the preceding claims, wherein the selection of all or part of the blur areas is implemented based on depth information contained in one pixel, in several pixels or in one or more group(s) of pixels of the target area.

11. A method according to any one of claims 2 to 10, wherein the blurring step is carried out as a function of: - the sharpness of the target area in the reference image and / or, for a given blur area, the sharpness of the target area in the image stack containing the considered blur area, and / or for a given blur area, the sharpness of the blur area in the reference image and / or the sharpness of the blur area in the image stack containing the blur area.

12. A method according to any one of claims 2 to 11, wherein the blurring step is implemented based on: - depth information contained in a pixel, in several pixels, or in one or more groups of pixels of the target area in the reference image and / or, for a given blur area, depth information contained in a pixel, in several pixels, or in one or more groups of pixels of the target area in the image of the image stack comprising the given blur area, and / or for a given blur area, of a depth information contained in a pixel, in several pixels or in one or more group(s) of pixels of the considered blur area in the reference image and / or of a depth information contained in a pixel, in several pixels or in one or more group(s) of pixels of the considered blur area in the image of the image stack including the considered blur area.

13. A method according to any one of claims 2 to 12, wherein the blurring step is implemented according to a standard blur matrix defining, for each pixel or group of pixels or areas of the image to be reconstructed, a variation in blur intensity to be obtained.

14. A method according to any one of claims 1 to 12, wherein the selection of blur areas is carried out according to a standard blur matrix defining, for each pixel or group of pixels or areas of the image to be reconstructed, a blur intensity to be obtained.

15. Method according to claim 13 or 14, comprising a step of calculating and / or determining the typical blur matrix.

16. Method according to claim 15, wherein the typical blur matrix is ​​calculated and / or determined as a function of the focal length of the current image and / or the reference image and / or at least one of the images in the image stack, distinct from the current image and the reference image.

17. Method according to claim 15 or 16, wherein the typical blur matrix is ​​calculated and / or determined as a function of depth information contained in one pixel, several pixels or one or more group(s) of pixels of the target area and / or at least one blur area.

18. A method according to any one of claims 15 to 17, wherein the typical blur matrix is ​​calculated and / or determined as a function of the sharpness of the target area in the current image and / or in the reference image and / or in at least one of the images in the image stack, distinct from the current image and the reference image.

19. A method according to any one of claims 15 to 18, wherein the typical blur matrix is ​​calculated and / or determined as a function of a reference blur matrix.

20. A method according to any one of claims 15 to 19, wherein the typical blur matrix is ​​calculated and / or determined as a function of a numerical aperture or apertures of one or more optical lenses of an imaging device from which the images in the image stack were acquired.

21. A method according to any one of claims 15 to 20, wherein the typical blur matrix is ​​calculated and / or determined as a function of a point spread function, called PSF, of one or more imaging systems of an imaging device from which the images in the image stack were acquired.

22. A method according to any one of claims 15 to 21, wherein the typical blur matrix is ​​calculated and / or determined as a function of: a position of the target area in the current image and / or a position in the reference image and / or in at least one of the images in the image stack, distinct from the current image and the reference image, and / or a position of one, several or each blur area in the current image and / or in the reference image and / or in at least one of the images in the image stack, distinct from the current image and the reference image.

23. A method according to any one of the preceding claims, comprising a step of displaying, on a display medium, the current image and / or the reconstructed image.

24. A method according to any one of the preceding claims, wherein the composite image reconstruction step is implemented from the reference image; the reconstruction step comprises: a substitution of at least one area of ​​the reference image, located outside the target area of ​​the reference image, by at least one blur area selected from another image in the image stack, distinct from the reference image, and / or an addition of at least one blur area, called an additional area, selected from another of the images in the image stack, distinct from the reference image; the area of ​​the imaged scene corresponding to the additional blur area being absent from the reference image.

25. A method according to any one of the preceding claims, wherein the definition of the target area includes an identification of the target area in the current image by an observer of the current image.

26. Method according to the preceding claim, wherein the target area is defined, from at least one image of the observer's eyes, the target area corresponding to the area of ​​the current displayed image, called the attention area, on which the observer's gaze is focused.

27. A method according to any one of claims 25 or 26, wherein the definition of the target area includes the identification of the target area by the observer of the current image.

28. Data processing device comprising means arranged and / or programmed and / or configured to implement the method according to any one of claims 1 to 27.

29. Computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method according to any one of claims 1 to 27.

30. Computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of claims 1 to 27.

Images