Method for selectively blurring image

Abstract

The invention relates to a method for selectively blurring a current clear image, comprising the following steps: defining a region of the current image, referred to as a clear region (2), in which the image remains unchanged and around which a blurring, referred to as a region to be blurred, must be generated; -calculating a blurring matrix suitable for the region to be blurred as a function of the position of the clear region (2) in the current image, the blurring intensity to be applied depending on the distance d1 between the part of the region to be blurred considered and the clear region; and-modifying the current sharp image by applying the calculated blur matrix to the current sharp image.

Description

TECHNICAL FIELD

[0001] The present invention relates to the modification of a displayed image, in particular in real time.

[0002] The present invention relates to the dynamic visualization of a displayed image. The present invention also relates to the immersive viewing of a displayed image. PRIOR ART

[0003] Tools and software for modifying or retouching an image are known in the prior art. These functions or more or less programs offer functions including image blur filters. There are several commonly used blur filters or methods such as radial blur, average blur, or even the most commonly used Gaussian filter. Depending on the software, the parameters of the blur filter can be customized to a greater or lesser extent, for example, by modifying the area to be blurred or the intensity of the blur.

[0004] The methods and processes of the prior art aim to provide the user with the function of customizing and controlling the blur that he wishes to apply to the image.

[0005] One aim of the present invention is to propose a method for blurring an image which allows:

[0006] - selective blurring of a displayed image being viewed by a user, and / or

[0007] - blurring of a portion and / or certain areas of the displayed image being viewed, and / or

[0008] - rendering of a specific blur effect on the displayed image, and / or

[0009] - real-time modification of the displayed image to reproduce a specific blur effect, and / or

[0010] - real-time modification of a stream of displayed images to reproduce a specific blur effect. SUMMARY

[0011] To this end, a method for selectively blurring a current clear image is proposed, called the method. The method comprises the following steps:

[0012] - defining a region of the current image called the clear region in which the image remains unchanged and around which a blur is generated, called the region to be blurred,

[0013] - calculating, generating or determining a blur matrix to be applied or applicable to the region to be blurred as a function of the position of the clear region in the current image, wherein the blur intensity to be applied is a function of the distance between the considered part, portion or partition of the region to be blurred and the clear region,

[0014] - Modify the currently sharp image by applying the calculated blur matrix to the currently sharp image.

[0015] The method may include the step of displaying the current image, preferably on a display medium or monitor.

[0016] Preferably, the method includes the step of displaying the image after modifying the current clear image, preferably on a display medium or monitor, thereby displaying the modified image.

[0017] Preferably, the display is view-oriented. Preferably, the user's attention to the image is defined by a fuzzy law. Depending on the application, the image may be displayed on a monitor other than the viewer's monitor, and / or transmitted, and / or archived.

[0018] In this application, "region" can be understood as a pixel, pixel group, or object in the current image.

[0019] Preferably, the blur is generated across the entire image except for the sharp areas.

[0020] The current clear image can change over time.

[0021] A blur matrix can be understood as a coefficient matrix that determines the blur intensity to be applied to each region and each pixel of an image by matching the image or a portion thereof. This blur intensity can be matched to the full width at half maximum (FWHM) of a point spread function (PSF). If needed, the blur intensity can be supplemented by defining other parameters, such as the shape of the PSF function. To obtain the blur, for example, the modified value of the pixel in the image under consideration can be calculated, which is obtained by multiplying the matrix coefficients representing the PSF by the pixels of the currently so-called sharp image in the vicinity of the pixel under consideration.

[0022] A currently sharp image may not be perfectly sharp everywhere. This method may include taking into account the degree of lack of sharpness in certain areas of the image to correspondingly reduce the intensity of the blur coefficient, thereby obtaining a modified image blur that depends essentially only on the blur laws explained in the following paragraphs, and little or no on the initial blur of the currently sharp image.

[0023] In this invention, an image, particularly a part, region, area, or partition of the current image, can be understood as referring to a pixel or group of pixels in the image.

[0024] A blur matrix can be understood as a matrix, grid, or table of coefficients. This matrix can contain a set of coefficients with different properties and / or different intensities. The coefficients of the blur matrix can correspond to pixels or groups of pixels in the pixel table of the current sharp image and / or the modified image.

[0025] According to the present invention, "...function", for example "the fuzzy matrix applicable to the region to be blurred is a function of the distance between the considered part of the region to be blurred and the clear region": evolves according to a law depending on one or more parameters, for example, the fuzzy matrix applicable to the region to be blurred evolves according to a law depending on the distance between the considered part of the region to be blurred and the clear region.

[0026] The method can be implemented for the current image stream or from the current image stream.

[0027] Preferably, the blur matrix to be applied and / or the distance d1 is a function of depth information contained in or associated with at least some of the pixels in the current image, or calculated based on depth information contained in or associated with at least some of the pixels in the current image, and / or calculated based on the angle formed between the axis connecting the sharp region and the position referred to as the viewing position and the axis connecting the viewing position and the considered portion of the region to be blurred, such that the observer's eye is estimated to be located at the viewing position relative to the displayed current image and / or relative to the sharp region and / or relative to the region to be blurred.

[0028] Viewing position can be the estimated or intended or located position of the observer's eye relative to the display medium displaying the current image.

[0029] Preferably, the fuzzy matrix to be applied is a function of depth information, or is calculated based on depth information, or is calculated based on a comparison of depth information contained in or associated with a portion of the clear region and a portion of the region to be blurred (preferably each).

[0030] Preferably, each coefficient of the blur matrix is ​​calculated based on a distance called the viewing distance between a position called the viewing location and a clear region, whereby the observer's eye is estimated relative to the current image, preferably the currently displayed image, and more preferably the currently displayed image located at that viewing location.

[0031] Preferably, the blur matrix is ​​calculated based on the distance between the viewing position and the considered portion of the region to be blurred.

[0032] Preferably, each coefficient of the fuzzy matrix is ​​calculated based on a comparison of two distances, which can be equated to the concept of depth.

[0033] Preferably, each coefficient of the blur matrix is ​​calculated based on a comparison of distance or depth obtained from the position of the lens of the optical system acquiring the image or video (referred to as the acquisition position) with a corresponding one or more objects in the scene or in the sharp area of ​​the current image, and from the distance between the acquisition position and a corresponding one or more objects in the scene or in the considered part of the area to be blurred.

[0034] Furthermore, the depths of these sharp and blurred areas can be matched with the depth obtained or evaluated in a reference frame of the image capture or imaging scene (i.e., the so-called true depth), or even with the depth estimated in a reference frame of the image presentation or display. For greater realism, the depth obtained or evaluated in a reference frame of the image capture or imaging scene is preferred.

[0035] Preferably, each coefficient of the fuzzy matrix is ​​calculated based on distance or depth, or a comparison of two or more distances, which can be equivalent to the concept of depth.

[0036] Preferably, each coefficient of the blur matrix is ​​calculated based on a comparison between the distance between the acquisition location and a corresponding object or one or more objects in the scene or in a sharp region of the current image, and between distances or depths obtained from one or more of the following distances or depths:

[0037] -Viewing distance, and / or

[0038] - The distance between the acquisition location and one or more corresponding objects in the scene or within the considered portion of the area to be blurred, and / or

[0039] - The distance between the viewing position and the considered portion of the blurred area of ​​the current image (preferably the currently displayed image).

[0040] Preferably, the method includes calculating the partitions or dimensions of the clear area (2) based on the viewing distance.

[0041] Preferably, the calculation of each coefficient of the blur matrix is ​​also based on the distance between the region to be blurred and the clear region, the distance or depth between the region to be blurred and the viewing position, and / or the comparison between the clear region and the depth of the viewing position.

[0042] The clear area and / or the partitions or dimensions of the clear area can be determined or calculated based on the viewing distance and based on at least one image from the observer's eye or based on the image stream from the observer's eye.

[0043] Preferably, the fuzzy matrix is ​​calculated based on the optical parameters of the human eye.

[0044] Preferably, the optical characteristics of the human eye may include:

[0045] - The focal length matches the distance between the lens and the retina, for example, from 17mm to 25mm.

[0046] - The diameter of the pupil.

[0047] - The response of the retina, i.e., the visual field.

[0048] Preferably, the retinal response can be a function of the preferred position of the image focus on the retina relative to the fovea. Preferably, the position of the image focus on the retina is a function of the distance between a portion of the region to be blurred or a region under consideration, the region to be blurred matching or coinciding with the position of the image focus on the retina and the sharp region.

[0049] Preferably, the method includes the step of obtaining the viewing position.

[0050] Preferably, a clear region is obtained from at least one image from the observer's eye using a detection device. Preferably, the clear region matches the region, referred to as the region of interest, where the observer's gaze is focused on the currently displayed image.

[0051] Preferably, the sharp region and / or viewing position are obtained from at least one image of the observer (including the observer's eyes) using a detection device. Preferably, the sharp region and / or viewing region are obtained from an image stream of the observer's eyes. Preferably, the sharp region and / or viewing position are obtained by analyzing at least one image of the observer's eyes.

[0052] An optical system (e.g., a camera) can be used to acquire the at least one image and / or the at least one image stream.

[0053] A clear region can be determined or obtained by eye tracking or based on pupil and / or eye information, such as by imaging the observer's eye, including pupil size and / or pupil orientation and / or the position of the eye within the eye socket. The clear region can be determined or obtained based solely on at least one image of the observer's eye or based on an image stream of the observer's eye.

[0054] Preferably, the method is implemented in real time, such that each change in the location of the region of interest results in the display of a modified image that takes into account the change.

[0055] Preferably, for each point on the perimeter of the clear area, an angle of 1° to 6° is formed by connecting the considered point on the perimeter of the clear area to the axis of the viewing position and connecting the center of the clear area to the axis of the viewing position.

[0056] Preferably, for each point on the perimeter of the clear area, the angle formed by connecting the considered point on the perimeter of the clear area to the axis of the viewing position and connecting the center of the clear area to the axis of the viewing position is greater than or equal to 1°, preferably equal to 2°, and / or less than or equal to 6°, preferably equal to 5°, and even more preferably equal to 4°. For each point on the perimeter of the clear area, the angle formed by connecting the considered point on the perimeter of the clear area to the axis of the viewing position and connecting the center of the clear area to the axis of the viewing position can be 3°.

[0057] Preferably, the fuzzy matrix comprises at least two sets, each set containing different fuzzy coefficients and / or different fuzzy intensities to be applied to the region to be fuzzed.

[0058] Preferably, the fuzzy matrices are continuously variable depending on different parameters, or are represented by discretization of these parameters, in order to achieve, for example, faster computation and minimize the memory footprint of representing them.

[0059] Preferably, a fuzzy matrix is ​​calculated, or a set of fuzzy matrices is arranged such that each set of fuzzy matrices generates a different fuzzy region in the modified image.

[0060] Preferably, the at least two sets and / or the at least two fuzzy coefficients are predetermined and / or taken from a database containing a list of fuzzy matrices and / or a list of sets and / or a list of fuzzy coefficients.

[0061] Preferably, the one or more fuzzy coefficients control and / or define the fuzzy intensity to be applied.

[0062] Preferably, for each set of the matrix, the fuzziness coefficients and / or fuzziness intensity of the set under consideration are calculated by applying a point spread function (referred to as PSF) to the set under consideration.

[0063] Preferably, each fuzziness coefficient and / or fuzziness intensity is calculated by applying different PSFs.

[0064] Preferably, the PSF describes the response of the imaging system that obtains the current image, where the PSF is a function of the following:

[0065] - The distance between the optical sensor of the imaging system and an object in the scene imaged by the optical sensor of the imaging system, and / or

[0066] - The distance between a part of the imaging system and the optical sensor of the imaging system, and / or

[0067] - The distance between a portion of the imaging system and an object in the scene imaged by the optical sensor, and / or

[0068] - The status of the imaging system, such as the zoom or focus or digital aperture settings of the imaging system, and / or

[0069] -One or more pieces of information contained in each pixel, group of pixels, or block of pixels in a convolutional image, and / or

[0070] - Each location in the image field, specifically as a function of object depth, and / or

[0071] - Each pixel or group of pixels or block of pixels, particularly as depth information contained in each pixel or group of pixels, and / or

[0072] - One or more angles between the optical sensor and the imaging system.

[0073] Preferably, in order to obtain a blur effect originating from or generated by an imaging system or camera module, or even the human eye, the method includes the step of generating a PSF function, wherein the PSF function is a function of the following:

[0074] - The distance between the optical sensor of the imaging system and an object in the scene imaged by the optical sensor of the imaging system, and / or

[0075] - The distance between a part of the imaging system and the optical sensor of the imaging system, and / or

[0076] - The distance between a portion of the imaging system and an object in the scene imaged by an optical sensor, and / or

[0077] - The status of the imaging system, such as the zoom or focus or digital aperture settings of the imaging system, and / or

[0078] -One or more pieces of information contained in each pixel, group of pixels, or block of pixels in a convolutional image, and / or

[0079] - Each location in the image field, specifically as a function of the object's depth, and / or

[0080] - Each pixel or group of pixels or block of pixels, specifically as a function of the depth information contained in each pixel or group of pixels, and / or

[0081] - One or more angles between the optical sensor and the imaging system.

[0082] Additional blurring effects related to the angular deviation between the sharp region and the region to be blurred can be added to the resulting PSF matrix.

[0083] Preferably, the fuzzy matrix is ​​calculated based on or from the following:

[0084] - At least one image of the observer's eye or a stream of images of the observer's eye, and / or

[0085] -Viewing distance, and / or

[0086] - The location of the sharp area in the current image, and / or

[0087] - One or more PSFs, preferably describing one or more responses of one or more imaging systems that obtained the current image, and / or

[0088] - At least some of the pixels in the current image contain one or more pieces of information, and / or

[0089] - At least some of the pixels in the current image contain depth information, and / or

[0090] - The angle formed between the axis perpendicular to the current image plane containing the center of the sharp area and the axis connecting the center of the sharp area to the viewing position.

[0091] The information contained in a pixel may be, for example, intensity, color data and / or texture data, and / or brightness data, and / or hue data, and / or saturation data, and / or RGB data detected by the photodiode of an optical sensor.

[0092] The fuzzy coefficients of the set of matrices, and / or the fuzzy intensity of the set, or each fuzzy coefficient of each set, and / or each fuzzy intensity of each set, are variable within the set.

[0093] According to another aspect of the invention, an apparatus is proposed that includes means configured to perform all the steps of the method according to the invention.

[0094] According to the present invention, a data processing apparatus is also proposed, comprising means arranged and / or programmed and / or configured to implement the method according to the present invention.

[0095] According to the present invention, a computer program including instructions is also provided, which, when executed by a computer, causes the computer to implement the method according to the present invention.

[0096] According to the present invention, a computer-readable medium (e.g., a recording medium) comprising instructions is also provided, which, when executed by a computer, causes the computer to implement the method according to the present invention.

[0097] According to the present invention, a computer-readable data medium having a computer program according to the present invention recorded thereon is also proposed.

[0098] The device according to the invention can be or integrated into any type of device, such as a smartphone, tablet computer, computer, calculator, processor, or computer chip, which is programmed to implement the method according to the invention, for example by executing a computer program according to the invention.

[0099] The device may not include an image acquisition unit. In this case, the device is used to display one or more images acquired by another device.

[0100] Alternatively, the device may include an image acquisition device, such as a camera or camera module. In this case, the device can be used to display one or more images acquired by the device or another device.

[0101] In particular, the device can be a user device including a display screen, such as a smartphone, tablet, etc. In this case, the detection device can be or can include a touch-sensitive surface, particularly integrated into or associated with the display screen of the device.

[0102] Specifically, the device can be a user device including a display screen, such as a computer. In this case, the detection device can be or can include, in particular, a display screen integrated into the computer or a touch-sensitive surface associated with it, or even a pointer moved by, for example, a mouse or the computer's directional pad.

[0103] Specifically, the device can be a television set. In this case, the detection device can be a camera integrated into the television set to detect the observer's line of sight and head position, or a pointer moved by, for example, the television set's remote control.

[0104] Specifically, the device can be a virtual reality or augmented reality head-mounted device, which includes a display screen or a projector associated with a projection surface onto which each image is projected. In this case, the detection device can be or may include sensors, particularly optical sensors mounted on the helmet.

[0105] Of course, the apparatus according to the invention is not limited to the examples disclosed above.

[0106] In particular, the device can be a medical imaging device.

[0107] Specifically, the device can be an endoscope, an ultrasound machine, etc.

[0108] According to another aspect of the invention, a vehicle is provided, comprising:

[0109] -Image display device, and

[0110] -At least a device for detecting the target location,

[0111] - At least one computing device;

[0112] Configured to implement all steps of the process according to the invention.

[0113] The vehicle may not include an image acquisition device. In this case, one or more images of the scene are provided by another device or vehicle.

[0114] Alternatively, the vehicle may include an image acquisition device, such as a camera or camera module. In this case, one or more images of the scene are captured by the image acquisition device, or provided by another device or vehicle.

[0115] In some embodiments, the vehicle may be a land vehicle, such as an automobile, and may be autonomous or non-autonomous.

[0116] In some embodiments, the vehicle may be a marine vehicle, such as a drone, an airplane, or a helicopter, and may be autonomous or non-autonomous.

[0117] In some embodiments, the vehicle may be a water vehicle, such as a boat or submarine, and may be autonomous or non-autonomous.

[0118] In some embodiments, at least one image of the scene is a 2D image. Specifically, the current image is a 2D image. Specifically, the modified image is a 2D image.

[0119] If applicable, the image stack includes at least one 2D image. Specifically, all images in the image stack are 2D images.

[0120] If applicable, a fully clear image is a 2D image.

[0121] In some embodiments, at least one image of the scene is a 3D image. Specifically, the current image is a 3D image. Specifically, the new image is a 3D image.

[0122] If applicable, the image stack includes at least one 3D image. Specifically, all images in the image stack are 3D images.

[0123] If applicable, a fully clear image is a 3D image. Attached Figure Description

[0124] Further benefits and features will become apparent upon review of the detailed description of the non-limiting embodiments and implementations, and from the following figures, wherein:

[0125] Figure 1 This is a schematic depiction illustrating the observer's position relative to the scene image displayed on the display medium and relative to the scene in question.

[0126] Figure 2 The plotted curve illustrates the radius of the region to be blurred, generated by the blurring matrix, for each considered region of the region to be blurred.

[0127] Figure 3 A schematic diagram depicts the displayed image modified by the blurring method according to the present invention.

[0128] Figures 4a-4c This is an illustrative depiction of a non-limiting exemplary embodiment of the device according to the present invention.

[0129] Figure 5 This is an illustrative depiction of a non-limiting exemplary embodiment of a vehicle according to the present invention. Detailed Implementation

[0130] The embodiments described below are not limiting in any way. In particular, variations of the invention may be considered if the selection of features is sufficient to provide a technical benefit or to distinguish the invention from the state of the prior art. These variations may contain only the selection of the disclosed features, isolated from the other disclosed features (even if the selection is isolated within a phrase containing the other features). The selection includes at least one preferred functional feature that lacks structural detail or has only partial structural detail, provided that part itself is sufficient to provide a technical benefit or to distinguish the invention from the level of the prior art.

[0131] Human vision consists of a clear area and a surrounding blurred area. The clear area corresponds to the light rays from objects or points in the scene being viewed, which strike a region in the eye called the fovea, where vision is at its maximum. When viewing a scene or image of a scene, a user focuses their gaze on an object or point in the scene at a given time t, causing a blurred area to appear around the area of ​​attention on which the observer is focusing. The observer's saccades of the scene shift both the clear and blurred areas within the viewed scene.

[0132] In addition, the image or video of the scene itself has a sharp area that matches the blur in one or more parts of the scene object located at the focal distance of the lens of the optical system that acquires the image or video, as well as the blur in the rest of the image, that is, the blur in one or more parts of the scene object located at a focal distance less than or greater than the focal distance of the lens of the optical system that acquires the image or video.

[0133] This invention aims to present images or videos with a blurriness representative of or equivalent to human vision by adding additional blur to the inherent blurriness of the displayed image or video. According to the invention, the image can be displayed on any type of known display medium (e.g., a screen, such as a television, monitor, smartphone, tablet, glasses, or virtual reality headset). Furthermore, the image or video display medium is typically located at a distance from the image observer, a distance smaller than the distance between the object in the imaged scene and the imaging system acquiring the image or video. According to the invention, the image can originate from any type of known imaging system, particularly a camera, such as a dedicated camera or a smartphone or tablet camera.

[0134] In summary, this invention aims to selectively blur an image displayed to a user during viewing, according to a so-called blur law that depends on parameters related to the viewing position of the image and parameters related to the position of the region of the image to be blurred. This invention also aims to render a blurring effect on the displayed image according to this blur law. Consideration is given to modifying the displayed image in real time to reproduce the blur of this blur law, and / or modifying the displayed image stream in real time to reproduce the blur of this law.

[0135] According to a non-limiting embodiment, the blur law aims to keep image regions close to and located on the so-called region of interest sharp, and to obtain / generate other increasingly blurred regions as the region of interest moves away from it, i.e. as the distance between a given part of the image to be blurred and the region of interest increases.

[0136] In a preferred case, the displayed image includes depth information and the gap to the region of interest preferably depends on the depth of the image region.

[0137] However, if the image does not contain depth information, then the depth of the region to be blurred in the observed image can be considered equal to the depth of the region of interest.

[0138] Similarly, if an image does not contain depth information, it can be replaced with parameters related to the vertical coordinates in the image. In other words, this means that the larger the vertically displayed image size, the greater the depth, which is indeed the case.

[0139] Furthermore, according to an advantageous method, it is beneficial to use a distance that separates the viewing position from the area of ​​the image to be blurred.

[0140] The gap to the region of interest can also depend on the angular deviation parameter between the angle of the region of interest and the angle of the region to be blurred. These angles and angular deviations are evaluated relative to the estimated spatial position of the observer's eye and the reference axis connecting the display medium, as well as the angle formed by the screen area relative to that reference axis.

[0141] The dependence of blur on depth information contained in an image can be calculated relative to the lens of the imaging system that acquires the displayed image, which is focused on the depth of the region of interest. This lens produces a blur effect related to the aperture (i.e., the lens's focusing diameter). Furthermore, the dependence of blur on depth information can be calculated relative to this blur effect, and / or relative to the lens's focus, proportional to this effect, or at least minimal or zero at regions in the image where the depth equals the depth of the region of interest, and increasing for lower and higher depths.

[0142] Preferably, the blur dependency can be equal to or proportional to the accommodation blur of human vision, that is, equal to or proportional to the accommodation blur of a lens with a given focal length and a given distance from the image capture plane. The diameter of the condenser aperture can be equal to that of the lens accommodating in front of the retina, and its diameter can be equal to a specific diameter of the iris.

[0143] The two parameters of the angle dependence of this law can be obtained by positioning the coordinates of points or regions of an image on two axes (e.g., x and y). These parameters can be expressed as angles α and β by dividing x and y by the distance between the display medium and the viewer's eye. This distance can be the distance between the display medium and the observer's eye, or it can be an estimated distance between the observer's eye and an estimated position of the scene actually represented on the display medium. Based on this position, the angular coordinates of each region of the image can be denoted as αi and βi, and the angular coordinates of the observed region can be denoted as αo and βo. This makes it possible to calculate the parameters of the fuzzy dependence law, such as the differences between these angles (αi - αo) and (βi - βo). This fuzzy dependence law can be inspired by the decreasing density of cone cells in the human eye, such that as one of the angular differences increases, or more precisely, as (αi - αo) increases... 2 +(βi-βo) 2 As the size increases, vision becomes less clear.

[0144] refer to Figures 1 to 3 An embodiment of a method for selectively blurring a currently sharp image is illustrated. According to a non-limiting embodiment, the currently sharp image is displayed on a display medium visible to an observer. The current image modified by this method is also displayed on the display medium after modification.

[0145] A sharp image should be understood as an image in which at least a portion is sharp, or the entire image is sharp, or an image that is almost sharp across the entire image set. The method includes the step of acquiring region 2 of the current image, referred to as sharp region 2, in which the image remains unchanged, and a blur must be generated around this sharp region, referred to as the region to be blurred.

[0146] According to a non-limiting embodiment, a clear region 2 can be obtained based on at least one image of an observer's eye 4. The clear region 2 matches the area where the observer's gaze is focused on in the displayed image. This clear region 2 can be derived from data related to the observer's gaze. Data related to the user's gaze can be obtained through eye tracking without requiring an acquisition step, and even the step of determining the clear region 2 from this data does not necessarily constitute part of the method according to the invention. In some cases, the clear region 2 can be known or predetermined data. Those skilled in the art are familiar with eye-tracking techniques and also with data that can be derived from these techniques, such as the foveal path. The method may include the step of acquiring data related to the observer's gaze, for example, by means of one or more optical systems that may include one or more cameras.

[0147] The method further includes calculating a blur matrix applicable to the region to be blurred based on the position of the sharp region 2 in the displayed image, wherein the applied blur intensity increases according to the distance d1 between the considered pixel to be blurred or the considered pixel group to be blurred and the sharp region 2. According to an embodiment, the applied blur intensity increases radially from the sharp region 2 towards the image edge 5.

[0148] The method also includes the step of modifying the current sharp image by applying a calculated blur matrix to the current sharp image to obtain a modified image 1. The modified image 1 replaces the current sharp image. In other words, the modified image 1, instead of the previously displayed sharp image, is displayed on the display medium 6.

[0149] According to a particularly advantageous embodiment, the blur matrix is ​​calculated based on depth information P contained in at least a portion of the pixels of the displayed image. For example, the depth P is matched with a distance relative to the imaging system or optical sensor that acquires the displayed image, at which an object in the imaging scene is located.

[0150] Special Reference Figure 1 The observer adjusts their gaze to the plane containing the display 6. However, they focus their attention on the area of ​​interest 7 located on the plane of the display medium 7, which includes the displayed image. When the observer directly views the displayed image, i.e., their eyes 4 are located at the position of the optical sensor of the imaging system that acquires the displayed image, they will focus their gaze on one or more objects in the scene located on a virtual plane 8 at a fictitious distance d4 that matches the depth P0 of the imaging scene where the object their gaze will focus on is located, thus adjusting their gaze to the relevant one or more objects.

[0151] Furthermore, the calculation of the blur matrix takes into account the depth P of different objects in the imaging scene contained in the displayed image. For example, the blur matrix is ​​calculated based on the object depth contained in virtual plane 9 located in front of virtual plane 8 (i.e., depth P or virtual distance d5 is less than P0 or d4), and the object depth contained in virtual plane 10 located behind virtual plane 8 (i.e., depth P or virtual distance d6 is greater than P0 or d4).

[0152] Advantageously, the step of calculating the blur matrix is ​​performed based on the viewing distance d2, which extends between the viewing position (where the observer's eye 4 is estimated to be located relative to the displayed image) and the clear region 2.

[0153] According to a non-limiting embodiment, the viewing position is such that the axis connecting the sharp region 2 to the viewing position coincides with the axis perpendicular to the region of the display medium including the sharp region 2. It should be noted that, according to the embodiment, the display medium is planar, but it can also be curved. Additionally, an angle β may exist between the axis connecting the sharp region 2 to the viewing position and the axis perpendicular to the region of the display medium including the sharp region 2. In this case, the blur matrix is ​​calculated based on angle β.

[0154] Advantageously, the step of calculating the blur matrix is ​​performed based on the distance d3 between the viewing position and the considered portion (4) of the region to be blurred.

[0155] Advantageously, and as an alternative or in combination with the calculation of the blur matrix based on distances d1 and / or d2 and / or d3, the calculation steps of the blur matrix are performed according to the angle λ, which is formed between the axis connecting the clear region (2) and the viewing position (i.e., the axis along which distance d2 extends) and the axis connecting the viewing position and the considered part of the region to be blurred (i.e., the axis along which distance d3 extends).

[0156] According to a non-limiting embodiment, for each considered region to be blurred in the displayed image, the radius of the light spot, denoted as tau, which is the radius of the considered region to be blurred by the blur matrix in each considered region, can be expressed according to the following formula 1:

[0157] , Formula 1,

[0158] Where f is the focal length of the observer's eye, OD is the aperture radius of the iris of the observer's eye 4, and L is the difference between the depth of the region to be blurred and the depth of the region of interest in the image (L=d3-d4).

[0159] According to the described non-limiting embodiments, Figure 2An example of a Tau curve is depicted, in pixels, as a function of distance d3 (in meters or other parameter units) for several aperture radii of the observer's eye 4 iris. The Tau curve is calculated with d4 equal to 1 meter, f equal to 0.017 meters, and aperture radii of 0.5 mm, 1.5 mm, 5 mm, and 9 mm.

[0160] The method includes the step of acquiring information about the viewing position. The viewing position can be known or predetermined data. According to a non-limiting embodiment, the viewing position is determined based on at least one image of the observer, or the observer's head or eyes 4, obtained by means of one or more imaging systems. The steps of acquiring at least one image and determining the viewing position are not necessarily part of the method according to the invention. The one or more imaging systems used to acquire at least one image can be the same as or different from the one or more imaging systems used to acquire data for determining the sharp region 2. Preferably, but not necessarily, the imaging systems are arranged and / or positioned to acquire objects from the observer's environment, particularly from the display medium 6 and / or the displayed images.

[0161] The method includes calculating the partitions, surface area, or shape of the clear region 2 based on the viewing distance d2. This step is also optional because the viewing position and / or the position of the display medium 6 and / or the clear region 2 may be known or predetermined. The partitions of the clear region 2 are calculated such that, for each point of the perimeter 21 of the clear region 2, the axis connecting the considered point of the perimeter 21 of the clear region 2 to the viewing position forms an angle of 1° to 6° with the axis connecting the center of the clear region 2 to the viewing position. According to an embodiment, the angle formed between the axis connecting the considered point of the perimeter 21 of the clear region 2 to the viewing position and the axis connecting the center of the clear region 2 to the viewing position is 3°. According to this embodiment, the perimeter 21 of the clear region 2 forms a circle. According to an embodiment, the clear region 2 forms a disk.

[0162] According to an embodiment, the fuzzy matrix includes at least two sets. Each set includes different fuzzy coefficients to be applied to the region to be fuzzified. In the step of calculating the fuzzy matrix, each set and each fuzzy coefficient are calculated such that the region to be fuzzified includes at least two concentric and adjacent fuzzy regions, adjacent to each other in pairs, with one of the two adjacent fuzzy regions at least partially surrounding the other fuzzy region. In the step of calculating the fuzzy matrix, each set and each fuzzy coefficient are calculated such that the fuzzy coefficients increase radially from the fuzzy region under consideration to fuzzy regions located outside the fuzzy region under consideration.

[0163] refer to Figure 2 The coefficients of the fuzzy matrix, denoted as C(i,j), can be expressed according to the following formula 2:

[0164] , Formula 2,

[0165] Where i and j are subscripts from 1 to N, which locally describe the pixels of the image (each pixel has at least three color channels, R, G, B (red, green, blue)), and i0 and j0 are the centers of the blur function (e.g., A is the conversion coefficient between the discretization step size and the physical distance on the optical sensor, and B is the renormalization coefficient used to ensure that the sum of the coefficients equals 1, so that the local multiplication of the image and the blur matrix does not modify the average brightness of the image region.

[0166] according to Figure 3 The illustrated non-limiting embodiment, in addition to parameters related to distances d1 to d6, angles λ and / or β, and depth P, may also consider human visual parameters to further enrich the blur matrix and represent the inherent blur of human vision. Indeed, light stimuli are converted into neural signals by cone and rod cells at the retina. The retinal response is complex and reaches its maximum in the central region of the retina (called the fovea), located at the macula, which contains the highest concentration of cone cells. Therefore, the response can be considered to be maximum at the center of the clear region 2 and then decreases with distance from this center.

[0167] Advantageously, the proportion of the object being viewed on the displayed image should also be considered, as well as the optional magnification factor applied during display. In practice, the image may be taken at a magnification, meaning that it is displayed on a screen at a distance D from the observer and with a width of L, and does not reproduce the blur that the observer would experience if the object displayed in the clear area 2 were actually located at a distance d4 (which is the distance between it and the optical sensor in the imaging scene).

[0168] Furthermore, as a non-limiting example, to reproduce the decrease in visual acuity as one moves away from the sharp region 2, an additional fuzziness law can be applied to the blur matrix. This is based on the parameter tauV, which has a lower value in the sharp region 2 and increases with angle λ. TauV can be considered constant between 0° and 1°, decreasing before reaching a certain angle, and then remaining constant after that angle (e.g., after angle λ exceeds 10°). This TauV parameter can be added to the distance, angle, and depth Tau, preferably using a sum of squares approach. For example, the effective Tau of the blur matrix can be equal to the square root of the sum of squares of Tau and TauV.

[0169] As an illustration, and for reference Figure 3An example of the modified image 1 shown is obtained from a blur matrix comprising three sets. A first set is computed to generate a first blurred region 31 adjacent to and surrounding the sharp region 2. A second set is computed to generate a blurred region 35, referred to as the peripheral blurred region 35, which is adjacent to and surrounding the first blurred region 31. A third set is computed to generate a blurred region 36, referred to as the edge blurred region 36, which is adjacent to and surrounding both the peripheral blurred region 35 and the first blurred region 31. The first blurred region 31 is partially surrounded by the peripheral blurred region 35 and partially surrounded by the edge blurred region 36. According to the illustrated embodiment, the sharp region 2 is completely surrounded by the first blurred region 31. According to the illustrated embodiment, the peripheral blurred region 35 is completely surrounded by the edge region 36. Figure 3 The perimeter 41 of the first blurred region 31 and the perimeter 45 of the outer blurred region 35 are also depicted.

[0170] Figure 9a is a schematic depiction of a non-limiting exemplary embodiment of the device according to the present invention.

[0171] The apparatus 910 of FIG9a includes means configured to implement the present invention.

[0172] The apparatus 910 of FIG9a may include the device according to the invention.

[0173] In the example shown in FIG9a, device 910 is a smartphone or tablet computer, including the device according to the invention. In particular, device 910 includes a display screen 810 equipped with a detection surface 812 (e.g., capacitive) and at least one camera 814.

[0174] Of course, the present invention is not limited to the examples just described, and many modifications can be made to these examples without departing from the scope of the present invention.

[0175] Therefore, in the interchangeable variations of the embodiments described above:

[0176] - Calculate the blur matrix based on the viewing distance, and / or

[0177] - Calculate the blur matrix based on depth information contained in at least a portion of the pixels of the displayed image, and / or

[0178] - Calculate one or more blur coefficients to be applied to the region to be blurred based on depth information contained in at least a portion of the pixels of the displayed image. This allows, for example, the blur coefficients to be modulated and / or varied differently depending on whether an object in the imaging scene is behind or in front of the sharp region 1.

[0179] - The blur matrix is ​​calculated based on the angle formed between an axis perpendicular to the displayed image plane (which contains the center of the sharp area) and an axis connecting the center of the sharp area to the viewing position, and / or

[0180] - For each set of the matrix, the ambiguity coefficients of the considered set are calculated by applying the point spread function (called PSF) to the considered set, and / or

[0181] - Calculate each fuzzy coefficient by applying different PSFs, and / or

[0182] -PSF describes the response of the imaging system from which it obtains the displayed image, and / or

[0183] -PSF is a function of the following:

[0184] • The distance between the optical sensor of the imaging system (if it is an imaging system) and the object in the scene imaged by the optical sensor of the imaging system, and / or

[0185] • The distance between a portion of the imaging system and the optical sensor of the imaging system, and / or

[0186] • The distance between a portion of the imaging system and an object in the scene imaged by the optical sensor, and / or

[0187] • The status of the imaging system, such as the zoom or focus or digital aperture settings of the imaging system, and / or

[0188] • One or more pieces of information contained in each pixel, group of pixels, or block of pixels in a convolutional image, and / or

[0189] • Each location in the image field, specifically as a function of object depth, and / or

[0190] • Each pixel or group of pixels or block of pixels, in particular as a function of the depth information contained in each pixel or group of pixels, and / or

[0191] • One or more angles between the optical sensor and the imaging system

[0192] - The fuzzy coefficients of the set of matrices, or each fuzzy coefficient of each set, varying within the set, and / or

[0193] - The fuzziness intensity to be applied is controlled and / or defined by the fuzziness coefficients, and / or

[0194] - The fuzzy coefficient is controlled and / or defined by PSF, and / or

[0195] - Implement the method from the displayed image stream, and / or

[0196] - This invention proposes a device, system, or any apparatus comprising arranged means, such as a processing unit, to implement any of the embodiments of the methods of the invention just described, so as to reproduce the blurring effect of human vision on a displayed image acquired by one or more imaging systems of the device, system, or apparatus, presenting a modified image, or presenting a modified image 30 based on changes in the observer's viewpoint of the displayed image 20 reproduced from images stored in the device, system, or apparatus, wherein the device, system, or apparatus is, as a non-limiting example: a smartphone, computer, camera, vehicle, glasses, or virtual reality headset, and / or

[0197] - Any of the uses of the embodiments of the device according to the invention just described, and / or the uses of any of the embodiments of the method according to the invention just described, based on images, preferably image streams, from the device, system, machine, or apparatus just described, or within a processing unit.

[0198] This invention proposes a computer program including instructions that, when executed by a computer, cause the computer to perform a method according to any of the disclosed embodiments, and / or

[0199] - The present invention proposes a readable medium comprising instructions, which are in particular readable by a computer or any device containing a processing unit, and which, when executed by the computer or device, cause the computer or device to perform a method according to any of the described embodiments.

[0200] Figure 4a This is a schematic depiction of a non-limiting exemplary embodiment of the device according to the present invention.

[0201] Figure 4a The apparatus 910 includes means configured to implement the present invention.

[0202] Figure 4a The device 910 may include the apparatus according to the invention.

[0203] exist Figure 4a In the example shown, device 910 is a smartphone or tablet computer, including the device according to the invention. Specifically, device 910 includes a display screen 810 equipped with a detection surface 812 (e.g., capacitive) and at least one camera 814.

[0204] Figure 4b This is an illustrative depiction of another non-limiting exemplary embodiment of the device according to the invention.

[0205] Figure 4bThe apparatus 920 includes means configured to implement the present invention.

[0206] Figure 4b The device 920 may include the device according to the invention, but not the camera 814.

[0207] exist Figure 4b In the example shown, device 920 is a virtual reality (VR) headset or augmented reality headset, comprising the device according to the invention. Specifically, headset 920 includes a display screen 810 and sensors for detecting an area of ​​interest, i.e., the position on the display screen 810 where one or both of the user's eyes are aimed.

[0208] exist Figure 4b In the example shown, the head-mounted device 920 does not include an imaging device for capturing images of the scene. In this case, one or more images of the scene to be displayed by the head-mounted device 920 are provided to the head-mounted device 920 by another device.

[0209] Alternatively, the head-mounted device 920 may include at least one camera for capturing images of its surroundings to display on screen 810, optionally after augmenting the images, as part of, for example, an augmented reality application.

[0210] Figure 4c This is a schematic depiction of a non-limiting exemplary embodiment of the device according to the present invention.

[0211] Figure 4c The apparatus 930 includes means configured to implement the present invention.

[0212] Figure 4c The device 930 may include the apparatus according to the invention.

[0213] exist Figure 4c In the example shown, the device is a medical imaging apparatus including the device according to the invention, such as an endoscope, ultrasound equipment, etc. Specifically, the medical imaging apparatus 930 includes a display screen 810 equipped with a detection surface 812 (e.g., capacitive). The medical imaging apparatus 930 also includes an imaging device formed by a distal lens connected to an imaging module (not shown).

[0214] Figure 5 This is an illustrative representation of a non-limiting exemplary embodiment of a vehicle according to the present invention.

[0215] Figure 5 The vehicle 100 includes means configured to implement the present invention.

[0216] Figure 5 The vehicle 1000 may include the equipment according to the invention.

[0217] exist Figure 5 In the example shown, vehicle 1000 is a land vehicle, particularly an automobile, that includes the device according to the invention. Specifically, vehicle 1000 includes a display screen 810 equipped with a detection surface 812 (e.g., capacitive), which is arranged in the passenger compartment of vehicle 1000. Vehicle 1000 also includes at least one camera (e.g., arranged on the windshield of vehicle 1000).

[0218] Furthermore, the various features, forms, variations, and embodiments of the present invention can be combined with each other in various ways, as long as they are not mutually incompatible or mutually exclusive.

Claims

1. A method for selectively blurring a currently sharp image, comprising the following steps: - Define a region of the current image called the sharp region (2), in which the image remains unchanged, and a blur must be generated around the sharp region (2), called the region to be blurred. - Based on the location of the sharp region in the current image, calculate a blur matrix applicable to the region to be blurred, where the applied blur intensity depends on the distance d1 between the considered portion of the region to be blurred and the sharp region. - Modify the current sharp image by applying the calculated blur matrix to the current sharp image.

2. The method according to the preceding claim, wherein the blur matrix is ​​calculated based on depth information P associated with at least some of the pixels of the current image.

3. The method according to claim 1 or 2, wherein the blur matrix is ​​calculated based on an angle λ, the angle λ being formed between an axis connecting the clear region (2) to a position referred to as the viewing position and an axis connecting the viewing position to a considered portion of the region to be blurred, wherein the observer's eye (4) is estimated to be located at the viewing position relative to the currently displayed image.

4. The method according to claim 1 or 2, comprising calculating a partition or size of the clear region (2) based on a distance d2 between a position referred to as the viewing position and the clear region, the distance d2 being referred to as the viewing distance, wherein the observer's eye (4) is estimated to be located at the viewing position relative to the currently displayed image.

5. The method according to claim 1 or 2, wherein the blur matrix is ​​calculated based on a distance d3 between a position referred to as the viewing position and a considered portion of the region to be blurred, wherein the observer's eye (4) is estimated to be located at the viewing position relative to the currently displayed image.

6. The method according to any one of the preceding claims, wherein the blur matrix is ​​calculated based on the optical parameters of the human eye.

7. The method of claim 6, in combination with any one of claims 3 to 5, further comprising the step of obtaining the viewing position.

8. The method according to any one of the preceding claims, wherein the clear region (2) is obtained from at least one image of the observer's eye (4), and the clear region matches the region on the currently displayed image where the observer's gaze is focused, referred to as the region of interest (7).

9. The method of claim 8, in combination with any one of claims 3 to 7, wherein, For each point of the perimeter (21) of the clear area (2), the considered point of the perimeter of the clear area is connected to the axis of the viewing position, and the center of the clear area is connected to the axis of the viewing position to form an angle of 1° to 6°.

10. The method according to any one of the preceding claims, wherein the fuzzy matrix comprises at least two sets, each set comprising different fuzzy coefficients to be applied to the region to be fuzzed.

11. The method according to the preceding claim, wherein, For each set of matrices, the fuzziness coefficient of the set under consideration is calculated by applying a point spread function called PSF to the set under consideration, and each fuzziness coefficient is calculated by applying a different PSF.

12. The method according to the preceding claim, wherein the PSF describes the response of the imaging system, wherein a current image is obtained from the imaging system, and wherein the PSF is a function of: - The distance between the optical sensor of the imaging system and an object in the scene imaged by the optical sensor of the imaging system, and / or - The distance between a portion of the imaging system and the optical sensor of the imaging system, and / or - The distance between a portion of the imaging system and an object in the scene imaged by the optical sensor, and / or - The state of the imaging system, such as the zoom or focus or digital aperture settings of the imaging system, and / or -One or more pieces of information contained in each pixel, group of pixels, or block of pixels in a convolutional image, and / or - Each location in the image field, specifically as a function of the object's depth, and / or - Each pixel or group of pixels or block of pixels, specifically as a function of the depth information contained in each pixel or group of pixels, and / or - One or more angles between the optical sensor and the imaging system.

13. The method according to any one of claims 8 to 12, wherein the fuzzy coefficients of the set of matrices, or each fuzzy coefficient of each set, vary within the set.

14. A data processing apparatus comprising means arranged and / or programmed and / or configured to implement the method according to any one of claims 1 to 13.

15. A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 13.

16. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 13.

Images