Method and system for personalized image enhancement

Abstract

A method and system for providing personalized image enhancement may include presenting at least one calibration image to a user via a User Interface (UI); receiving from the user, via the UI, one or more modifications to one or more respective visual parameters of the at least one calibration image; defining a personalized profile of the user, based on the one or more modifications; receiving a target image; adjusting the target image based on the personalized profile; and presenting the adjusted target image to the user.

Description

METHOD AND SYSTEM FOR PERSONALIZED IMAGE ENHANCEMENTCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a PCT International Application claiming the benefit of priority of U.S. Provisional Patent Application No. 63 / 442,141, filed January 31, 2023, entitled “PERSONALIZED HISTOGRAM EQUALIZATION SYSTEM AND METHOD” which is hereby incorporated by reference in its entirety.FIELD OF THE INVENTION

[0002] The present invention relates generally to the field of image processing. More specifically, the present invention relates to methods and systems for providing personalized image enhancement.BACKGROUND OF THE INVENTION

[0003] The human visual system responds to luminance differences (i.e., contrast). The sensitivity of this responds (contrast sensitivity) depends on lighting conditions and individual differences.

[0004] The prevalence of eye diseases and thus of vision impairment is increasing globally. Screen watching habits of people with vision impairments are comparable to normally sighted people, however their visual impairment prevents them from fully benefiting from this medium.

[0005] The inverse of the contrast detection threshold is defined as the contrast sensitivity. Individuals with visual impairment require higher levels of contrast than healthy persons to see most stimuli.

[0006] Loss in contrast sensitivity can be thought of as having a low-pass filtering effect, although it is actually a nonlinear, threshold effect. At some critical frequency, no level of presented contrast is sufficient for the stimuli to be seen. This critical frequency and the overall contrast sensitivity vary with the severity of impairment.

[0007] Central field loss, caused by diseases such as macular degeneration and diabetic retinopathy damaging the central retina, has a particularly disabling effect on visual function. As the natural retinal locus of high-resolution vision is damaged, patients learn to use a peripheral part of their retina for detail vision, the preferred retinal locus (PRL). However, contrast sensitivity and acuity decline with retinal eccentricity (the distance of the PRL from the fovea).

[0008] Video image-enhancement techniques designed to assist people with visual impairments, particularly those due to central retinal vision loss were presented. A major difficulty in this endeavor is the lack of evaluation techniques to assess and compare the effectiveness of various enhancement methods.

[0009] Image enhancement techniques have been shown to provide improvements in functional sight in some visually impaired individuals where residual vision is present.

[0010] Image contrast enhancement algorithms play a critical role in visual technologies. One challenge in contrast enhancement is that an algorithm suitable for low contrast images may not be suited for high contrast images. Another challenge is how to optimize contrast enhancement algorithm to the contrast sensitivity profile of the observers and lighting conditions.

[0011] A person's light sensitivity varies depending on lighting conditions and the state of their sensory system.

[0012] Yet, current image enhancement algorithms do not consider individual differences, instead offering a "one size fits all". For example, histogram equalization improves the visibility of nighttime images by modifying the luminance histogram to the full scale between black and white. Thus, there is a need for a new algorithm that enhances images according to individual preferences.

[0013] For example, subjects suffering from Retinitis Pigmentosa (RP) may experience progressive degeneration of the retina, leading to vision loss and, in severe cases, blindness. Individuals with RP often experience difficulty seeing in low-light conditions, such as at night or in dimly lit environments. This is due to the early involvement of rod cells, which are responsible for vision in low-light conditions. It has been experimentally demonstrated by the inventors that such individuals may require personalized image adjustment to overcome visual difficulties.

[0014] The foregoing examples of the related art and limitations related therewith are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent to those of skill in the art upon a reading of the specification and a study of the figures.SUMMARY OF THE INVENTION

[0015] Embodiments of the invention may include a method of providing personalized image enhancement by at least one processor. According to some embodiments, the at leastone processor may be configured to present at least one calibration image to a user via a User Interface (UI); receive from the user, via the UI, one or more modifications to one or more respective visual parameters of the at least one calibration image; define a personalized profile of the user, based on the one or more modifications; receive a target image; adjust the target image based on the personalized profile; and present the adjusted target image to the user.

[0016] Additionally, or alternatively, the at least one processor may be configured to receive at least one context data element, defining a type of the target image; and adjust the target image further based on the context data element.

[0017] Additionally, or alternatively, the at least one processor may be configured to apply an object recognition algorithm on the target image, to obtain a definition of at least one object of a predetermined type, depicted in the target image; and adjusting the target image further based on the definition of the at least one object.

[0018] Additionally, or alternatively, the at least one processor may be configured to modify the visual parameter of the calibration image in an iterative process. Each iteration of the iterative process may include receiving, via the UI, an incremental change in a value of the visual parameter; generating a modified version of the calibration image based on the incremental change; and presenting the modified version via the UI as feedback for the user.

[0019] According to some embodiments, the personalized profile may include: (i) a modified mean value of distribution of brightness levels in a subregion (R) of the calibration image, and (ii) a modified standard deviation value of the distribution of brightness levels in the subregion (R) of the calibration image.

[0020] According to some embodiments, the at least one processor may adjust the target image by changing brightness levels of pixels in a corresponding subregion (R’) of the target image according to the personalized profile, thereby forcing a distribution of brightness levels in the corresponding subregion (R’) of the adjusted target image to have the same modified mean and modified standard deviation values as the sub-region (R) of the calibration image.

[0021] Additionally, or alternatively, the at least one processor may receive, from a gaze detection device, an indication of the user’s gaze direction; and associate the subregion (R) of the calibration image with a section (S) in the user’s Field Of View (FOV) based on the gaze direction.

[0022] The at least one processor may subsequently adjust the target image by changing brightness levels of pixels in a subregion (R’) of the target image, which corresponds to the section (S) of the user’s FOV, according to the personalized profile, thereby forcing a distribution of brightness levels in the corresponding subregion (R’) of the adjusted target image to have the same modified mean and modified standard deviation values as the subregion (R) of the calibration image.

[0023] Additionally, or alternatively, the personalized profile may include a modified, overall brightness level of a sub-region (R) of the calibration image. The at least one processor may adjust the target image by changing brightness levels of pixels in a corresponding sub-region (R’) of the target image according to the personalized profile, so as to have the same modified, overall brightness level as the sub-region (R) of the calibration image.

[0024] Additionally, or alternatively, the personalized profile may include a modified, overall brightness level of a sub-region (R) of the calibration image. The at least one processor may receive, from a gaze detection device, an indication of the user’s gaze direction; associate the subregion (R) of the calibration image with a section (S) in the user’s FOV, based on the gaze direction; and adjust the target image by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the personalized profile, so as to have the same modified, overall brightness level as the sub-region (R) of the calibration image.

[0025] Additionally, or alternatively, the modified visual parameters may include a contrast level of a sub-region (R) of the calibration image. The at least one processor may adjust the target image by changing brightness levels of pixels in a corresponding sub-region (R’) of the target image according to the personalized profile, thereby forcing a contrast level of the corresponding sub-region (R’) to have the same value as the modified contrast level of the sub-region (R) of the calibration image.

[0026] Additionally, or alternatively, the modified visual parameters may include comprise a contrast level of a sub-region (R) of the calibration image. The at least one processor may receive, from a gaze detection device, an indication of the user’s gaze direction; and associate the subregion (R) of the calibration image with a section (S) in the user’s FOV, based on the gaze direction. The at least one processor may subsequently adjust the targetimage by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the personalized profile, thereby forcing a contrast level of the corresponding sub-region (R’) to have the same value as the modified contrast level of the sub-region (R) of the calibration image.

[0027] Additionally, or alternatively, the at least one calibration image may include a plurality of regions (R), each defined by a respective Spatial Frequency (SF), and the personalized profile further may include modified, SF band-specific contrast levels. In such embodiments, the at least one processor may adjust the target image by defining subregions (R’) of the target image; for one or more subregions (R’), applying a set of spatial filters, thereby producing a respective set of patches (P’), each corresponding to a specific SF band; for each patch (P’), changing brightness levels of pixels according to the personalized profile, thereby forcing a contrast level of the patch (P’) to have the same value as the corresponding SF band-specific contrast level of the personalized profile; superimposing patches (P’) pertaining to the same subregions to produce altered subregions of the target image; and combining the altered subregions of the target image to produce the adjusted target image.

[0028] Additionally, or alternatively, the at least one calibration image may include a plurality of regions (R), each defined by a respective SF. In such embodiments, the at least one processor may receive, from a gaze detection device, an indication of the user’s gaze direction; and based on the gaze direction, defining a Contrast Sensitivity Function (CSF) as part of the personalized profile. The CSF function may associate modified contrast levels of at least one subregion (R) with (a) specific SF bands, and (b) specific gaze-directions.

[0029] The at least one processor may subsequently adjust the target image by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the CSF function.

[0030] The at least one processor may change brightness levels of pixels in a sub-region (R’) of the target image by applying a set of spatial filters on sub-region (R’) of the target image, thereby producing a respective set of patches (P’), each corresponding to a specific SF band. For each patch (P’), the at least one processor may change brightness levels of pixels according to the CSF function. The at least one processor may then apply a weighted superposition function on patches (P’) pertaining to the same subregion (R’) of the target image, to produce an altered version of subregion (R’).

[0031] According to some embodiments, the at least one processor may receive, via the UI, a definition of a sub-region (R) of the at least one calibration image, and receiving, via the UI, an indication of a modified visual parameter corresponding to the defined sub-region (R) , thereby determining a region- specific visual parameter value. The at least one processor may then apply at least one adaptation to a corresponding region (R’) of the target image, to adjust the target image.

[0032] Additionally, or alternatively, the at least one processor may adjust the target image by applying a blurring spatial filter on at least one region, based on the personalized profile, thereby exploiting visual characteristics of the user to reduce data redundancy in the target image.

[0033] Embodiments of the invention may include a system for personalized image enhancement. Embodiments of the system may include: at least one first camera; a headmounted projector; a non-transitory memory device, wherein modules of instruction code are stored; and at least one processor associated with the memory device, and configured to execute the modules of instruction code.

[0034] Upon execution of said modules of instruction code, the at least one processor may be configured to: present, via the head-mounted projector, at least one calibration image to a user; receive, from the user, via a User Interface (UI), indication of one or more modifications to one or more respective visual parameters of the at least one calibration image; defining a personalized profile of the user, based on the one or more modifications; receive, via the at least one camera, a target image; adjusting the target image based on the personalized profile; and present the adjusted target image to the user via the projector.

[0035] Additionally, or alternatively, embodiments of the system may include: a non- transitory memory device, wherein modules of instruction code are stored, and at least one processor associated with the memory device, and configured to execute the modules of instruction code. Upon execution of said modules of instruction code, the at least one processor may be configured to: present at least one calibration image to a user via a User Interface (UI); allow the user to apply, via the UI, one or more modifications to one or more respective visual parameters of the at least one calibration image; receive a target image; apply at least one adaptation to the target image, based on the one or more modified visualparameters of the at least one calibration image, thereby producing an adapted version of the target image; and present the adapted version of the target image to the user.BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings in which:

[0037] Fig. 1 is a block diagram, depicting a computing device which may be included in a system for personalized image enhancement according to some embodiments;

[0038] Fig. 2 shows flowchart for a user-specific image enhancement method that may optimize image display based on a user-vision profile, according to some embodiments of the invention;

[0039] Fig. 3 depicts a flowchart and a graph. The top panel (A) is a flowchart showing contrast sensitivity adjustment procedure, according to some embodiments of the invention. The bottom panel (B) is a graph showing different user specific inverse contrast sensitivity functions, according to some embodiments of the invention;

[0040] Fig. 4 is a schematic diagram for a task that a user may perform to determine a userspecific weighting profile, according to some embodiments of the invention;

[0041] Fig. 5A depicts micrographs and graphs showing examples for image adjustment according to histogram equalization, as known in the art, where changes in the intensity of the adjusted image are monotonic, and not necessarily perceived as optimal by specific users;

[0042] Fig. 5B depicts micrographs and graphs showing examples for image adjustment according some embodiments of the invention where changes in the intensity of the adjusted image are not monotonic, and may be adjusted by embodiments of the invention to be perceived as optimal by specific users;

[0043] Figs. 6A and 6B, show the results of a study, that was conducted based on embodiments of the present invention;

[0044] Fig. 7 is a block diagram, depicting a system for personalized image enhancement according to some embodiments of the invention; and

[0045] Fig. 8 is a flow diagram, depicting a method of providing personalized image enhancement according to some embodiments of the invention.

[0046] It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.DETAILED DESCRIPTION OF THE PRESENT INVENTION

[0047] One skilled in the art will realize the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The foregoing embodiments are therefore to be considered in all respects illustrative rather than limiting of the invention described herein. Scope of the invention is thus indicated by the appended claims, rather than by the foregoing description, and all changes that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

[0048] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention. Some features or elements described with respect to one embodiment may be combined with features or elements described with respect to other embodiments. For the sake of clarity, discussion of same or similar features or elements may not be repeated.

[0049] Although embodiments of the invention are not limited in this regard, discussions utilizing terms such as, for example, “processing,” “computing,” “calculating,” “determining,” “establishing”, “analyzing”, “checking”, or the like, may refer to operation(s) and / or process(es) of a computer, a computing platform, a computing system, or other electronic computing device, that manipulates and / or transforms data represented as physical (e.g., electronic) quantities within the computer’s registers and / or memories into other data similarly represented as physical quantities within the computer’s registers and / or memories or other information non-transitory storage medium that may store instructions to perform operations and / or processes.

[0050] Although embodiments of the invention are not limited in this regard, the terms “plurality” and “a plurality” as used herein may include, for example, “multiple” or “two or more”. The terms “plurality” or “a plurality” may be used throughout the specification to describe two or more components, devices, elements, units, parameters, or the like. The term “set” when used herein may include one or more items.

[0051] Unless explicitly stated, the method embodiments described herein are not constrained to a particular order or sequence. Additionally, some of the described method embodiments or elements thereof can occur or be performed simultaneously, at the same point in time, or concurrently.

[0052] Reference is now made to Fig. 1, which is a block diagram depicting a computing device, which may be included within an embodiment of a system for personalized image enhancement, according to some embodiments.

[0053] Computing device 1 may include a processor or controller 2 that may be, for example, a central processing unit (CPU) processor, a chip or any suitable computing or computational device, an operating system 3, a memory 4, executable code 5, a storage system 6, input devices 7 and output devices 8. Processor 2 (or one or more controllers or processors, possibly across multiple units or devices) may be configured to carry out methods described herein, and / or to execute or act as the various modules, units, etc. More than one computing device 1 may be included in, and one or more computing devices 1 may act as the components of, a system according to embodiments of the invention.

[0054] Operating system 3 may be or may include any code segment (e.g., one similar to executable code 5 described herein) designed and / or configured to perform tasks involving coordination, scheduling, arbitration, supervising, controlling or otherwise managing operation of computing device 1, for example, scheduling execution of software programs or tasks or enabling software programs or other modules or units to communicate. Operating system 3 may be a commercial operating system. It will be noted that an operating system 3 may be an optional component, e.g., in some embodiments, a system may include a computing device that does not require or include an operating system 3.

[0055] Memory 4 may be or may include, for example, a Random- Access Memory (RAM), a read only memory (ROM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a double data rate (DDR) memory chip, a Flash memory, a volatile memory, a nonvolatile memory, a cache memory, a buffer, a short term memory unit, a long term memoryunit, or other suitable memory units or storage units. Memory 4 may be or may include a plurality of possibly different memory units. Memory 4 may be a computer or processor non-transitory readable medium, or a computer non-transitory storage medium, e.g., a RAM. In one embodiment, a non-transitory storage medium such as memory 4, a hard disk drive, another storage device, etc. may store instructions or code which when executed by a processor may cause the processor to carry out methods as described herein.

[0056] Executable code 5 may be any executable code, e.g., an application, a program, a process, task, or script. Executable code 5 may be executed by processor or controller 2 possibly under control of operating system 3. For example, executable code 5 may be an application that may provide personalized image enhancement, as further described herein. Although, for the sake of clarity, a single item of executable code 5 is shown in Fig. 1, a system according to some embodiments of the invention may include a plurality of executable code segments similar to executable code 5 that may be loaded into memory 4 and cause processor 2 to carry out methods described herein.

[0057] Storage system 6 may be or may include, for example, a flash memory as known in the art, a memory that is internal to, or embedded in, a micro controller or chip as known in the art, a hard disk drive, a CD-Recordable (CD-R) drive, a Blu-ray disk (BD), a universal serial bus (USB) device or other suitable removable and / or fixed storage unit. Data pertaining to an image may be stored in storage system 6 and may be loaded from storage system 6 into memory 4 where it may be processed by processor or controller 2. In some embodiments, some of the components shown in Fig. 1 may be omitted. For example, memory 4 may be a non-volatile memory having the storage capacity of storage system 6. Accordingly, although shown as a separate component, storage system 6 may be embedded or included in memory 4.

[0058] Input devices 7 may be or may include any suitable input devices, components, or systems, e.g., a detachable keyboard or keypad, a mouse and the like. Output devices 8 may include one or more (possibly detachable) displays or monitors, speakers and / or any other suitable output devices. Any applicable input / output (RO) devices may be connected to Computing device 1 as shown by blocks 7 and 8. For example, a wired or wireless network interface card (NIC), a universal serial bus (USB) device or external hard drive may be included in input devices 7 and / or output devices 8. It will be recognized that any suitablenumber of input devices 7 and output device 8 may be operatively connected to Computing device 1 as shown by blocks 7 and 8.

[0059] A system according to some embodiments of the invention may include components such as, but not limited to, a plurality of central processing units (CPU) or any other suitable multi-purpose or specific processors or controllers (e.g., similar to element 2), a plurality of input units, a plurality of output units, a plurality of memory units, and a plurality of storage units.

[0060] Described herein are a system, method, and computer program product for automatically inducing visual stimuli in a gray background on a calibrated color monitor in a dark room, wherein the stimuli comprise two or more simultaneous light patch stimuli appearing at the center of the display, wherein each stimulus of the two or more simultaneous light patch stimuli, subtend 1° of visual angle, wherein each stimulus vary randomly, wherein a luminance level of a test stimulus will vary according to an adaptive staircase procedure that is based on a user’s response.

[0061] In one embodiment, provided herein is a user-specific image enhancement system, method, and computer program that optimizes image display based on a user-vision profile. In one embodiment, provided herein is a user-specific image enhancement system, method, and computer program for assisting an individual having visual impairment by optimizing the image to the individual luminance sensitivity and contrast sensitivity, hi one embodiment, provided herein is a user-specific image enhancement system, method, and computer program for image optimization. In one embodiment, provided herein is a userspecific image enhancement, system, method, and computer program for image enhancement.

[0062] In one embodiment, a system, method, and computer program of the invention includes a visual display means, hi one embodiment, a system, method, and computer program of the invention includes an image capturing means, hi one embodiment, a system and method of the invention includes closed-circuit television. In one embodiment, a system and method of the invention includes virtual reality goggles connected to a camera. In one embodiment, a system and method of the invention includes a computer. In one embodiment, a system and method of the invention includes a screen or a computer display. In one embodiment, a system and method of the invention includes a mobile phone anddisplay device. hi one embodiment, a system and method of the invention includes augmented reality goggles connected to a camera.

[0063] In one embodiment, the stimuli are presented on a color monitor. In one embodiment, the stimuli are presented on a calibrated monitor. In one embodiment, the stimuli are presented on a monitor with a resolution of 1,280 x 960 pixels. In one embodiment, the stimuli are presented on a color monitor having a refresh rate of at least 50 Hz. In one embodiment, the stimuli are presented on a color monitor ha ving a refresh rate of at least 70 Hz. hi one embodiment, the stimuli are presented on a color monitor having a refresh rate of at least 80 Hz. In one embodiment, the stimuli are presented on a color monitor having a refresh rate of at least 90 Hz. In one embodiment, the stimuli are presented on a color monitor having a refresh rate of at least 100 Hz.

[0064] In one embodiment, a system, method, and computer program of the invention is / are utilized in a dark room. In one embodiment, an individual or a subject treated by the system, method, and computer program of the invention is located 40 to 80 cm from the screen. In one embodiment, an individual or a subject treated by the system, method, and computer program of the invention is located 1cm to 50 cm from the screen. In one embodiment, an individual or a subject treated by the system, method, and computer program of the invention is located 50 to 80 cm from the screen. In one embodiment, an individual or a subject treated by the system, method, and computer program of the invention is located 50 to 70 cm from the screen. In one embodiment, an individual or a subject treated by the system, method, and computer program of the invention is located 55 to 60 cm from the screen. In one embodiment, an individual / user or a subject treated by the system, method, and computer program of the invention is located 57 cm from the screen, hi one embodiment, an indi vidual or a subject treated by the system, method, and computer program of the invention is located 80 cm to 500 cm from the screen. In one embodiment, an individual’s or a subject’s head is supported by a chin-and-forehead rest. In one embodiment, the screen’s background is grayscale (0 cd / m2 to 400 cd / m2). In one embodiment, the screen’s background is gray (50 cd / m2). In one embodiment, the screen’s background is black. In one embodiment, the screen’s background is white. In one embodiment, the screen’s background is gray (50 cd / m2).

[0065] In one embodiment, stimuli comprise two or more light patch stimuli. In one embodiment, two or more light patch stimuli appear close to each other. In one embodiment.close to each other is 8° to 0° (visual angle) of two or more neighboring edges of two or more light patch stimuli, wherein each one of two or more neighboring edges belongs to one light patch stimulus of the light patch stimuli. In one embodiment, close to each other is 8° to 0.0° (visual angle)of two or more neighboring borders of two or more light patch stimuli, wherein each one of two or more neighboring borders belongs to one light patch stimulus of the light patch stimuli.

[0066] In one embodiment, stimuli appear at a central portion of the screen. In one embodiment, central portion of the screen is an area occupying at least 5% of the screen area and located at its center. In one embodiment, central portion of the screen is an area occupying at least 10% of the screen area and located at its center. In one embodiment, central portion of the screen is an area occupying at least 20% of the screen area and located at its center. In one embodiment, central portion of the screen is an area occupying at least 30% of the screen area and located at its center. In one embodiment, central portion of the screen is an area occupying at least 40% of the screen area and located at its center. In one embodiment, central portion of the screen is an area occupying at least 50% of the screen area and located at its center.

[0067] In one embodiment, each stimulus of the stimuli subtending 0.5 to 2° of visual angle. In one embodiment, each stimulus of the stimuli subtending 0.7 to 1.4° of visual angle, In one embodiment, each stimulus of the stimuli subtending 0.8 to 1.2° of visual angle. In one embodiment, each stimulus of the stimuli subtending 1° of visual angle,

[0068] In one embodiment, an individual / user or a subject indicates which stimulus of the stimuli has brighter luminance. In one embodiment, an individual or a subject indicates which stimulus of the stimuli is darker. In one embodiment, luminance level of a standard stimulus will vary randomly across 16 possible levels (e.g. 0, 5, 10, 15, 20, 30, 40, 50, 70, 80, 100, 120, 150, 180, 250, 400 cd / m2).

[0069] In one embodiment, for each standard luminance level, the luminance level of an additional stimulus such as but limited to the test stimulus will vary according to an adaptive staircase procedure that is based on user’s responses (e.g., QUEST or 3 up -1 down staircase). In one embodiment, the standard stimulus and the comparison or test stimulus will appear in separate locations on the screen. In one embodiment, the standard stimulus and the comparison or test stimulus will appear on opposite sides of the screen. In one embodiment, the standard stimulus and the comparison or test stimulus will appear at thesame time and for the same duration on the screen. In one embodiment, the standard stimulus and the comparison or test stimulus will appear separately on the screen. In one embodiment, the standard stimulus will appear first, and the comparison or test stimulus will appear second on the screen. In one embodiment, the comparison or test standard stimulus will appear first and the standard stimulus will appear second on the screen.

[0070] In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for at least 1 second. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for at least 2 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for at least 5 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for at least 10 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for at least 20 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for at least 30 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for up to 5 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for up to 10 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for up to 15 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for up to 20 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for up to 30 seconds. In one embodiment, the standard stimulus, the comparison, or test stimulus or both will appear for up to 60 seconds.

[0071] In one embodiment, estimation of the slopes that could be accomplished from a staircase procedure are designed to track threshold at a particular performance level. In one embodiment, the staircase procedure uses adaptive stimulus such as a test stimulus, then provides a threshold estimate in a variety of ways, such as by averaging the levels at the direction reversals in the adaptive track. In one embodiment, the staircase procedure may include up -down staircases call for a reduction in stimulus luminescence level when the subject’s response is positive to the test stimulus, the standard or both. In one embodiment, the staircase procedure may include up-down staircases call for an increase in stimulus level when the response is negative.

[0072] In one embodiment, the track targets the stimulus luminescence level for which the probability of a correct response equals the probability of an incorrect response or,equivalently, the level at which the track would move up or down on the stimulus axis with equal probability.

[0073] In one embodiment, the present invention provides user Specific histogram adjustment. Existing image contrast (luminance differences) enhancement methods, such as histogram equalization, increase image contrast uniformly, such that the resulted image has maximum contrast levels (without losing image information) in all ranges of the image luminance levels. These methods ignore some fundamental characteristics of visual perception, hi particular, people are more sensitive to contrast levels within the mid-range of luminance than at low' or high levels of luminance.

[0074] The function can adjust and increase image contrast globally. It increases contrast by applying the method of histogram normalization and equalization. Existing methods of histogram equalization use a cumulative uniform (linear) distribution to evenly spread image contrast across luminance values (usual y 0-255).

[0075] In one embodiment, the present invention further adds to cumulative uniform distribution a Gaussian distribution. In one embodiment, the Gaussian distribution component enables the adjustment of contrast levels such that contrast is smaller in midluminance range and larger in high- and low- luminance range.

[0076] In one embodiment, the mean and the standard deviation of the Gaussian distribution is personalized for each individual / subject / user, i.e. based on each user’s individual contrast sensitivity.

[0077] In one embodiment, included herein is an outcome mixture distribution which enables the adjustment of the image contrast (the uniform distribution) while taking into consideration the personalized contrast sensitivity profile of the user (the Gaussian distribution).

[0078] In one embodiment, the present invention includes a method comprising calculation of image histogram. In one embodiment, calculation of image histogram for a grayscale image x , and with “N” number of pixels, the probability for an intensity i is: Where is the number of pixels in intensity / .

[0079] In one embodiment, based on the image histogram the method further includes calculation of a cumulative distribution function CDF:

[0080] In one embodiment, the method further includes a user-specific adjustment step. In one embodiment, the user-specific adjustment step may include a target histogram. In one embodiment, probability mixture distribution for the target histogram T is created by combining a weighted average of a uniform distribution and a gaussian distribution: where m is the number of intensity i:values andis the personalized probability weight of the uniform distribution, and where f(i ) is a Gaussian distribution with a mean and standard deviation

[0081] Idle parameters , and are free and can be adjusted according to the calculation of the parameters for each user.

[0082] In one embodiment, the method further includes cumulative distribution of the pdf calculation comprising

[0083] In one embodiment, the method further includes an algorithm transforming the intensity levels of the original image x histogram according to the intensity of the histogram T by: F(x = z) = min | cdf(T = z) cdf(x = i) \.

[0084] In one embodiment, the invention further includes utilizing a user specific weighting profile. In one embodiment, user specific weight may include stimuli having a set of natural images, hi one embodiment, user specific weight may include stimuli having a set of 50-500 natural images. In one embodiment, user specific weight may include stimuli having a set of 20 to 200 natural images. In one embodiment, user specific weight may include stimuli having a set of 80 to 120 natural images.

[0085] In one embodiment, each image is modified using the personalized histogram equalization according to the user specific parameters mu and sigma. In one embodiment, image is modified according to S number weighting levels uniformly distributed between 0 and 1, leading to a total of “S” number modified images for each original image. In one embodiment, “S” number is 5 to 80. In one embodiment, "‘S” number is 5 to 50. In one embodiment, “S” number is 5 to 40. In one embodiment, “S” number is 5 to 30. In one embodiment, '"S'" number is 10 to 20. In one embodiment, “S” number is 15 to 25. In one embodiment, “S” number is 15.

[0086] In one embodiment, in each trial or test two modified versions of the original image will appear left and right from the fixation cross, counterbalancing sides (Fig. 4). In oneembodiment, each modified version will appear 5-50 times in random order. In one embodiment, the user’s task is to choose which image version is preferable. In one embodiment, for each original image, the optimal beta is determined according to the modified image version selected most often or the lowest beta possible if a few beta levels have identical ranking. In one embodiment, the optimal beta overall will be the average beta across the beta calculated on each of the natural images.

[0087] In one embodiment, a system as described herein is only an exemplary embodiment of the present invention, and in practice may have more or fewer components than shown, may combine two or more of the components, or may have a different configuration or arrangement of the components. The various components of system may be implemented in hardware, software, or a combination of both hardware and software. In various embodiments, system may comprise a dedicated hardware device, or may form an addition to or extension of an existing device. In some embodiments, components of the system may be implemented in the cloud, atty desktop computing device, and / or any mobile computing device.

[0088] In some embodiments, system may comprise a processing unit and memory storage device, hi some embodiments, the system may store in a non-volatile memory thereof, such as a storage device, software instructions or components configured to operate a processing unit (also "hardware processor," "CPU," or simply "processor”), such as a processing unit. In some embodiments, the software components may include an operating system, including various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components.

[0089] In some embodiments, a user interface of the system may include a display monitor for displaying images and a control panel for controlling system. In some variations, the display may be used to display images processed by an image processing module.

[0090] In some embodiments, reference database is provided to store a plurality of reference images.

[0091] In an experiment conducted according to embodiments of the present invention urban scene images in daylight and nighttime, were examined. The experiment included two phases as follows:

[0092] Phase 1 - image adjustment: for each image, participants (n=12) moved the mouse in the x and y directions to simultaneously adjust two parameters, the mean and variance of a gaussian function that determined the luminance and contrast of the image. Images were adjusted when presented alone or when presented near the histogram equalized image.

[0093] Phase 2 - discrimination: the experiment tested whether the selected parameters in phase 1 improve the visibility of the adjusted image. Each adjusted image was compared to the original image or to the histogram equalized image, and the participant selected the image that appeared to have more details.

[0094] Phase 1 : Image parameters varied across image type and participants. Importantly there was a high correlation between the first and second image adjustment both across images and participants. These finding suggests individual difference in image enhancement preferences.

[0095] Phase 2: With daylight images, participants choices did not differ from chance, that is they chose adjusted or comparison images randomly. However, when nighttime images were presented, participants preferred the adjusted images significantly more than the original and the histogram equalized images (88% and 74%, respectively). The study quantifies variation in image enhancement preferences across neurotypical individuals, demonstrate the efficacy of our image enhancement method, and that individual differences should be considered in future applications of image processing.

[0096] These results are presented in Figs. 6A and 6B. Figs. 6 A and 6B shov>' a comparison of preferences of images that were adjusted using the method according to embodiments of the present invention to the original images (Fig. 6A) and to images after histogram equalization (Fig. 6B). Each point represented a participant and their discrimination value (percent times adjusted image was chosen) for N images (daylight) and U images (nighttime).

[0097] WThen accuracy (y axis) is above 50, the method according to embodiments of the present invention was chosen more often, value of 50 is random (participants didn't care about the method used to modify the image), value below 50 the comparison image was preferred. X axis - type of image N images (daylight) and U images (nighttime).

[0098] As may be clearly seen in Figs. 6A and 6B, for nighttime images (U) the method according to embodiment of the present invention is preferable. This was also confirmed using statistical tests.

[0099] Reference is now made to Fig. 7, which is a block diagram depicting a system 100 for providing personalized image enhancement, according to some embodiments.

[0100] According to some embodiments of the invention, system 100 may be implemented as a software module, a hardware module, or any combination thereof. For example, system may be or may include a computing device such as element 1 of Fig. 1 , and may be adapted to execute one or more modules of executable code (e.g., element 5 of Fig. 1) to provide personalized image enhancement, as further described herein.

[0101] As shown in Fig. 7, arrows may represent flow of one or more data elements to and from system 100 and / or among modules or elements of system 100. Some arrows have been omitted in Fig. 7 for the purpose of clarity.

[0102] According to some embodiments, system 100 may include a user interface (UI) module 120, which may be, or may include any combination of an input device (e.g., 7 of Fig. 1) such as a computer mouse or touchscreen, and output device (e.g., 8 of Fig. 1) such as a computer monitor or an extended reality device, adjusted (as known in the art) with head mounted goggles and associated projector(s) for image display.

[0103] System 100 may receive (e.g., from storage 6 of Fig. 1) at least one calibration image 20, and present the at least one calibration image 20 to a user via UI 120. As elaborated herein, system 100 may (e.g., during a calibration stage) allow the user to apply, via UI 120 one or more modifications 120MP to one or more respective visual parameters 20P of the at least one calibration image, resulting in a modified version 120M of the presented calibration image(s).

[0104] According to some embodiments, the modification 120MP of visual parameters 20P of calibration image 20 may be performed in an iterative process. In each iteration, system 100 may receive, via the UI, an incremental change in a value of the visual parameter. For example, a visual parameter of the at least one calibration image may be an overall brightness value, e.g., of a calibration image 20, or in a specified region (R) of calibration image 20. UI 120 may enable the user to incrementally modify this visual parameter (e.g., increase or decrease the brightness) by moving a computer mouse in a specified direction. UI 120 may subsequently generate a modified version 120MP of calibration image(s) 20 based on the incremental change (e.g., having an increased or decreased brightness), and present the modified version 120M of calibration image 20 (e.g., via UI 120) and / or or another display module 190 as feedback for the user.

[0105] System 100 may then discern one or more preferences, or a personalized profile 155 of the user based on, or including these modifications 120MP of parameter values 20P. Pertaining to the example of a modified brightness value, a brightness feature module 130 of system 100 may ascertain the user’s modified, or preferred brightness value 120MP (denoted 130BR). As elaborated herein, preferred brightness 130BR may be an overall brightness value, relating to an entire calibration image 20. Additionally, or alternatively, preferred brightness 130BR may relate to brightness of specific, predetermined regions (R) of calibration image 20. Additionally, or alternatively, preferred brightness 130BR may relate to brightness regions (R) in calibration image 20 having specific spatial frequencies. Additionally, or alternatively, preferred brightness 130BR may relate to brightness in specific gazing directions of the user. Other options for preferred brightness 130BR may also be possible.

[0106] Brightness feature module 130 may thereby adjust the user’s personalized profile 155 to include the preferred brightness 130BR.

[0107] In a subsequent stage, system 100 may receive (e.g., from a camera 110 of Fig. 2) one or more new, incoming images (denoted here as “target image(s) 110”). As elaborated herein, an adjustment module 180 may then apply, or infer the user’s preference or profile 155 on target image(s) 110, to produce one or more personally enhanced, adjusted images 180A.

[0108] Pertaining to the example of the personalized profile 155 representing a modified, or preferred overall brightness level 130BR of a sub-region (R) of a calibration image 20: In this example, adjustment module 180 may adjust the target image HOT by changing brightness levels of pixels in a sub-region (R’) of target image 110T, which corresponds to (e.g., is in the same location as) sub-region (R) of a calibration image 20, according to the personalized profile. In other words, adjustment module 180 may force sub-region (R’) of target image 110T to have the same preferred, or modified, overall brightness level 130BR as the sub-region (R) of the calibration image.

[0109] Adjusted images 180A may then be presented to the user (e.g., in place of target image(s) 110), via UI 120 and / or module 190, as elaborated herein, thereby enhancing the presentation of target image 110 according to the user’s preference.

[0110] System 100 may facilitate enhancement of image presentation over a wide variety of combinations, stemming from selection of different user preferences of diverse visualparameters 20P. It may be appreciated that an exhaustive list of these combinations may not be possible in the scope of this paper. Nevertheless, some non-limiting examples of personalized profile parameters and corresponding enhancements of target images 110T are provided herein.

[0111] For example, system 100 may include, or may be associated with a gaze detection device 160, configured to identify a direction of the user’s gaze 160D, as known in the art. Personalized profile 155 may include a modified, overall brightness level 130BR of a subregion (R) of calibration image 20. Brightness feature module 130 may receive, from gaze detection device 160 an indication of the user’s gaze direction, and thereby associate the subregion (R) of the calibration image with a section (S) in the user’s Field Of View (FOV). Brightness feature module 130 may subsequently update personalized profile 155 to include a modified, overall brightness level 130BR associated with the section (S) of the user’s FOV. Adjustment module 180 may then adjust target image HOT by changing brightness levels of pixels in a sub-region (R’) of target image HOT, which corresponds to the section (S) of the user’s FOV, according to the personalized profile 155.

[0112] In other words, adjustment module 180 may force sub-region (R’) of target image 110T (now adjusted image 180A) to have the same preferred, or modified, overall brightness level 130BR as sub-region (R) of calibration image 20, where both sub-regions (R) and (R’) pertain to the same gaze direction 160D (or section S) in the user’s FOV.

[0113] In another example, user interface 120 may (e.g., during a calibration phase) allow the user to adjust (e.g., by moving a computer mouse) a mean and variance of a gaussian function that determines the luminance and contrast of the image. In other words, a histogram feature module 140 of system 100 may receive from the user, via UI 120: (i) a modified mean value 120MP (denoted 140MU) of distribution of brightness levels in a subregion (R) of calibration image 20, and (ii) a modified standard deviation or variance value 120MP (denoted 140SG) of the distribution of brightness levels in calibration image 20 or in a subregion (R) of calibration image 20. Histogram feature module 140 may update the personalized user profile 155 to include standard deviation and / or mean values 140SG / 140MU.

[0114] In a subsequent inference phase, adjustment module 180 may adjust an incoming target image 110T by changing brightness levels of pixels in a subregion (R’) of target image HOT, which corresponds to (e.g., is in the same location as) subregion (R) of calibrationimage 20, according to personalized profile 155. In other words, adjustment module 180 may force a distribution of brightness levels in corresponding subregion (R’) of the target image HOT (now adjusted target image 180A) to have the same modified mean and modified standard deviation values 140MU / 140SG as in sub-region (R) of calibration image 20.

[0115] Additionally, or alternatively, system 100 may adjust presentation of image 110T according to modified standard deviation and / or mean values 140SG / 140MU, and in relation to the user’s gaze direction 160D.

[0116] In other words, adjustment module 180 may change brightness levels of pixels in a subregion (R’) of target image HOT (now adjusted image 180A), which corresponds to the same direction 160D, or section (S) of the user’s FOV, as that of subregion (R) of calibration image 20, according to personalized profile 155. Adjustment module 180 may thereby force a distribution of brightness levels in the corresponding subregion (R’) of the adjusted target image 180A to have the same modified mean 140MU and modified standard deviation 140SG values as that of sub-region (R) of calibration image 20.

[0117] Additionally, or alternatively, user interface 120 may (e.g., during a calibration phase) allow the user to adjust (e.g., by using keyboard arrow buttons) visual parameters 20P such as a contrast level of one or more sub-regions (R) of calibration image 20. In other words, a contrast feature module 145 of system 100 may receive from the user, via UI 120 of system 100, a modified visual parameter 120MP such as a modified contrast level 145CR of one or more sub-regions (R) of calibration image 20, and update profile 155 to include contrast level 145CR.

[0118] In a subsequent, application phase, adjustment module 180 may adjust target image 110T by changing brightness levels of pixels in a sub-region (R’) of the target image 20 (now adjusted image 180A), that corresponds to (e.g., is in the same location as) subregion (R) of calibration image 110T, according to the personalized profile.

[0119] In other words, adjustment module 180 may force a contrast level of the corresponding sub-region (R’) of adjusted image 180A to have the same value as the modified contrast level 145CR of the sub-region (R) of calibration image 20.

[0120] Additionally, or alternatively, system 100 may adjust presentation of image 110T according to modified or preferred contrast levels 145CR, in relation to the user’s gaze direction 160D.

[0121] For example, profile 155 may include a visual parameter such as a modified, or preferred contrast level 145CR of a sub-region (R) of calibration image. Contrast module 145 may receive, from gaze detection device 160 an indication of the user’s gaze direction, and thereby associate the subregion (R) of the calibration image 20 with a direction 160D or a section (S) in the user’s FOV, based on the gaze direction. During a subsequent inference stage, adjustment module 180 may adjust target image HOT by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the same direction 160D or section (S) of the user’s FOV as sub-region (R) of calibration image 20, according to the personalized profile. Adjustment module 180 may, thereby force a contrast level of the sub-region (R’) of target image 110T (now 180A) to have the same value as the modified contrast level 145CR of sub-region (R) of calibration image 20.

[0122] In another example personalized profile 155 may include, or refer to visual parameters (e.g., overall brightness, brightness distribution, contrast, etc.) that are Spatial Frequency (SF)-specific, and system 100 may adjust target images according to these SF- specific parameters. The term “spatial frequency” may be used herein to refers to a number of cycles of a repetitive pattern in a unit of space in an image.

[0123] For example, the one or more calibration images 20 may include a plurality of regions (R) defined by respective spatial frequencies, e.g., incorporate a repetitive pattern over a predetermined axis in calibration image 20. Contrast module 145 may receive, via UI 120 a modified value 120MP (denoted 145CR) of an SF band-specific contrast level. The SF band-specific contrast level 145CR may represent a preferred contrast within the repetitive pattern of one or more (e.g., each) SF band-specific region (R) of calibration image(s) 20. Contrast module 145 may thereby update personalized profile 155 to include the received, modified, SF band-specific contrast levels 145CR.

[0124] In a subsequent application phase, adjustment module 180 may adjust target image HOT according to the modified SF band-specific contrast levels 145CR of personalized profile 155. In some embodiments, adjustment module 180 may do so by separately applying SF-specific alterations to different components in the spatial and / or frequency domains, and subsequently superimpose the altered components to derive adjusted image 180A.

[0125] For example, contrast module 145 may define one or more subregions (R’) of target image HOT. For one or more (e.g., each) subregion (R’), contrast module 145 mayapply a set of spatial filters 145FLT, thereby producing a respective set of patches (P’), each corresponding to a specific SF band. For one or more (e.g., each) patch (P’), adjustment module 180 may change brightness levels of pixels in that patch (P’) according to the modified, SF band-specific contrast levels 145CR of personalized profile 155.

[0126] In other words, adjustment module 180 may force a contrast level of each patch (P’) in target image 110T (now 180A) to have the same value as the corresponding SF bandspecific contrast level 145CR of personalized profile 155. Adjustment module 180 may then apply a weighted superposition function on patches (P’) pertaining to the same subregions (R’) to superimpose the patches (P’), thereby producing altered subregions (R’) of target image HOT (now 180A). Adjustment module 180 may subsequently combine altered subregions (R’) of the target image 110T to produce adjusted target image 180A.

[0127] As known in the art, the term Contrast Sensitivity may indicate a minimal level of contrast that is distinguishable by a viewer. In this context, a Contrast Sensitivity Function (CSF) is a measure of the visual system's ability to detect and distinguish between different levels of contrast in visual stimuli. In other words, a CSF function may describe how well the human eye can perceive variations in contrast at different spatial frequencies.

[0128] Alternatively, the term CSF function may be used herein to indicate a specific type of visual parameter that may be included in personalized profile 155: Such a CSF function may associate, or map between (a) a modified, or preferred contrast level 145CR in a specific region (R) of calibration image 20, and (b) the SF and gaze direction 160D (or section S in the viewer’s FOV), which characterize region (R).

[0129] As elaborated herein, the at least one calibration image 20 may include a plurality of regions (R), each defined by a respective SF. A CSF feature module 150 of system 100 may obtain (e.g., from UI 120, via contrast feature module 145) modified, or preferred SF- band specific contrast level(s) 145CR. CSF feature module 150 may also receive, from gaze detection device 160, an indication 160D of the user’s gaze direction. Based on gaze direction 160, CSF feature module 150 may calculate a CSF 150C data element (e.g., a lookup table), that may define a CSF function which characterizes the specific viewer (e.g., user of UI 120). CSF function 150C may associate, or map between modified contrast levels 145CR of at least one subregion (R) with (a) specific SF bands of region (R) and (b) specific gaze-directions 160D of region (R). CSF feature module 150 may then update personalized profile 155, to include the user-specific CSF function 150C.

[0130] In a subsequent inference stage, adjustment module 180 may adjust an incoming target image 110T by changing brightness levels of pixels in at least one sub-region (R’) of the target image HOT, which corresponds to the same gaze-direction 160D (or section (S) the user’ s FOV) as sub-region (R) of the calibration image 20, according to the CSF function.

[0131] In other words, adjustment module 180 may force a contrast level of a sub-region (R’) of target image HOT (characterized by specific gaze-direction 160D and SF), to have the same value as modified contrast 145CR of a corresponding (same gaze-direction 160D and SF) sub-region (R) of the calibration image 20.

[0132] For example, during an inference stage (or “application” stage, used herein interchangeably) adjustment module 180 may applying the set of spatial filters 145FLT on sub-region (R’) of the target image, thereby producing a respective set of patches (P’), each corresponding to a specific SF band. For each patch (P’), adjustment module 180 may change brightness levels of pixels according to the CSF function, e.g., to have the same value as modified contrast 145CR of a corresponding (e.g., same gaze-direction 160D and SF) sub-region (R) of the calibration image 20. Adjustment module 180 may then apply a weighted superposition function on patches (P’) pertaining to the same subregion (R’) of the target image, to produce an altered version of subregion (R’), and recombine subregions (R’) to generate adjusted image 180A.

[0133] The superposition function may be weighted in a sense that each patch may be assigned a different weight when combining the patches together to form altered versions of subregion (R’). For example, the weight of each patch may be calculated based on its SF distribution, or power. Additionally, or alternatively, the weight of each patch may be determined so as to provide desired image characteristics, including for example noise filtering (e.g., by assigning higher weights to low SF bands) or edge enhancement (e.g., by assigning higher weights to high SF bands).

[0134] As elaborated herein, embodiments of the invention may improve target image(s) 110T (generate adjusted image(s) 180A) in various manners, to enhance viewing experience of specific users, based on their visual characteristics and / or impairments.

[0135] Additionally, or alternatively, embodiments of the invention may improve target image(s) in a manner that does not necessarily affect the user’s viewing experience, but rather in a sense of improving computer operation. For example, personalized profile 155 may include an indication (145CR / 150C) of at least one region (R) or direction 160D ofcalibration image 20, where a user has impaired visibility (e.g., a low contrast sensitivity). Adjustment module 180 may adjust target image(s) HOT by applying a filter (e.g., a blurring, spatial filter) on at least one corresponding (e.g., same direction 160D or location as) region (R’) of target image HOT, based on the personalized profile 155. Adjustment module 180 may thereby exploit visual characteristics of the user to reduce data redundancy in target image HOT. It may be appreciated that such reduction of data redundancy may translate to smaller storage, networking and processing computer resources.

[0136] Additionally, or alternatively, adjustment module 180 may improve image(s) 110T (produce adjusted image(s) 180A) according to context, or image-type. For example, adjustment module 180 may receive (e.g., via input 7 of Fig. 1) at least one context data element, which may define a type of the target image (e.g., radiology image, text image, night-scene image, etc.). Adjustment module 180 may subsequently adjusting target image 110T to generate adjusted image 180A further based on the context data element.

[0137] For example, adjustment module 180 may be adapted apply a one set of spatial filters 145FLT for radiological images, and another, different set of spatial filters 145FLT for night-time scenery images. In another example, adjustment module 180 may be adapted to avoid applying specific filters 145FLT (e.g., blurring filters) when processing text images.

[0138] Additionally, or alternatively, adjustment module 180 may improve image(s) HOT (produce adjusted image(s) 180A) based on image content. For example, system 100 may include, or may be associated with an object detection module 170, adapted to identify specific objects 170BJ depicted in an image, as known in the art.

[0139] According to some embodiments, adjustment module 180 may employ object detection module 170 to applying an object recognition algorithm on target image(s) HOT, thereby obtaining a definition (e.g., a region (R’), a contour, etc.) of at least one object 170BJ of a predetermined type (e.g., a car, a door, etc.) depicted in target image(s) HOT. Adjustment module 180 may subsequently adjusting target image HOT based on the user’s profile 155 (e.g., CSF 150C), and further based on the definition of the at least one object 170BJ. For example, adjustment module 180 may apply specific spatial filters 145FLT according to CSF (e.g., SF contrast sensitivity and direction 160D), and further based on identified objects of interest (e.g., a car), to accentuate depiction of specific objects in areas of impaired vision.

[0140] Additionally, or alternatively, system 100 may include, or may be integrated into a Virtual Reality (VR) or Extended Reality (ER) device. In such embodiments, display 190 may be, or may include a head-mounted projector, and camera 110 may include one or more head mounted cameras. UI 120 may also be integrated into the input and display options of the VR or ER device, as known in the art.

[0141] As elaborated herein, at least one processor (e.g., processor 2 of Fig. 1) of system 100 may present, via the head-mounted projector 190, at least one calibration image 20 to a user, and receive, from the user (e.g., via UI120), indication of one or more modifications 120MP to one or more respective visual parameters 20P of the at least one calibration image 20. During a calibration stage, the at least one processor 2 may define a personalized profile of the user 155 based on the one or more modifications 120MP. During a subsequent inference stage, the at least one processor 2 may receive, via the at least one camera 110 one or more target images (e.g., an image of a scene). As elaborated herein, the at least one processor 2 may adjust the target image(s) HOT based on the personalized profile, and present the adjusted target image 180A to the user via the projector 190.

[0142] Additionally, or alternatively, at least one processor (e.g., processor 2 of Fig. 1) of system 100 may receive, via UI 120 (a) a definition of a sub-region (R) of at least one calibration image, and (b) an indication of a modified visual parameter corresponding to the defined sub-region (R). Processor 2 may thereby determine a region-specific visual parameter value 120MP. As elaborated herein, processor 2 may subsequently apply at least one adaptation to a corresponding region (R’) of a target image HOT, to adjust the target image. The term “corresponding” may be used in this context to indicate regions (R) and (R') being at the same location, and / or direction of gaze, in respective images 20 and 110T.

[0143] Reference is now made to Fig. 8, which is a flow diagram depicting a method of providing personalized image enhancement by at least one processor (e.g., processor 2 of Fig. 1), according to some embodiments of the invention.

[0144] As shown in step S1005, the at least one processor 2 may present at least one calibration image (e.g., 20 of Fig. 7) to a user, via a UI (e.g., 120 of Fig. 7).

[0145] As shown in step S1010, the at least one processor 2 may receive from the user, via UI 120, one or more modifications (e.g., 120MP of Fig. 7) to one or more respective visual parameters 20P of the at least one calibration image 20.

[0146] As shown in step S1015, the at least one processor 2 may define a personalized profile (e.g., 155 of Fig. 7) of the user, based on the one or more modifications. For example, personalized profile 155 may include one or more elements of modified visual parameters 120MP (e.g., denoted 130BR, 140SG, 140MU, 145CR, 150C) that may be accumulated (e.g., over time, over a plurality of calibration images 20, and the like), and characterize a specific user.

[0147] As shown in step S1020, the at least one processor 2 may (e.g., during an application or inference stage) receive one or more target images 110T. Target images 110T may include, for example single pictures received from another, communicatively connected computing device. Additionally, or alternatively, target images 110T may include frames or snapshots in a video stream, obtained from a camera 110 that may be included in, or communicatively connected to system 100.

[0148] As shown in steps S1025 and S1030, the at least one processor 2 may adjusting the target image 110T based on personalized profile 155, and presenting the adjusted target image to the user e.g., via the UI120 and / or another, dedicated display device (e.g., 190 of Fig. 7).

[0149] It may be appreciated that in some embodiments, e.g., when system 100 is integrated into Virtual Reality (VR) or Extended Reality (ER) technology, processor 2 may be required to perform analysis of the incoming target image(s) 110T in real time, or near real time, to present adjusted, improved images 180A with an adequate sense of user experience.

[0150] Embodiments of the invention may include a practical application for providing personalized improvement, or enhancement of images, in real-time or near real-time. Embodiments of the invention may be included, or integrated into currently available technology, such as devices for augmented or extended reality, to improve different aspects of this technology. Such improvements may include, for example enhancement of assistive aids, e.g., for the population of visually impaired. In another example, embodiments of the invention may improve computer technology, e.g., by omitting redundant image data, thereby reducing data transfer, processing and storage.

[0151] Unless explicitly stated, the method embodiments described herein are not constrained to a particular order or sequence. Furthermore, all formulas described herein are intended as examples only and other or different formulas may be used. Additionally, someof the described method embodiments or elements thereof may occur or be performed at the same point in time.

[0152] While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents may occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.

[0153] Various embodiments have been presented. Each of these embodiments may of course include features from other embodiments presented, and embodiments not specifically described may include various features described herein.

Claims

CLAIMS1. A method of providing personalized image enhancement by at least one processor, the method comprising: presenting at least one calibration image to a user via a User Interface (UI); receiving from the user, via the UI, one or more modifications to one or more respective visual parameters of the at least one calibration image; defining a personalized profile of the user, based on the one or more modifications; receiving a target image; adjusting the target image based on the personalized profile; and presenting the adjusted target image to the user.

2. The method of claim 1, further comprising: receiving at least one context data element, defining a type of the target image; and adjusting the target image further based on the context data element.

3. The method according to any one of claims 1-2, further comprising: applying an object recognition algorithm on the target image, to obtain a definition of at least one object of a predetermined type, depicted in the target image; and adjusting the target image further based on the definition of the at least one object.

4. The method according to any one of claims 1-3, wherein modifying the visual parameter of the calibration image is performed in an iterative process, wherein each iteration comprises: receiving, via the UI, an incremental change in a value of the visual parameter; generating a modified version of the calibration image based on the incremental change; and presenting the modified version via the UI as feedback for the user.

5. The method according to any one of claims 1-4 wherein the personalized profile comprises: (i) a modified mean value of distribution of brightness levels in a subregion (R) of the calibration image, and (ii) a modified standard deviation value of the distribution of brightness levels in the subregion (R) of the calibration image.

6. The method of claim 5, wherein adjusting the target image comprises changing brightness levels of pixels in a corresponding subregion (R’) of the target image according to the personalized profile, thereby forcing a distribution of brightness levels in the corresponding subregion (R’) of the adjusted target image to have the same modified mean and modified standard deviation values as the sub-region (R) of the calibration image.

7. The method according to any one of claims 5-6, further comprising: receiving, from a gaze detection device, an indication of the user’s gaze direction; and associating the subregion (R) of the calibration image with a section (S) in the user’s Field Of View (FOV) based on the gaze direction.

8. The method according to any one of claims 5-7 wherein adjusting the target image comprises changing brightness levels of pixels in a subregion (R’) of the target image, which corresponds to the section (S) of the user’s FOV, according to the personalized profile, thereby forcing a distribution of brightness levels in the corresponding subregion (R’) of the adjusted target image to have the same modified mean and modified standard deviation values as the sub-region (R) of the calibration image.

9. The method according to any one of claims 1-8, wherein the personalized profile comprises a modified, overall brightness level of a sub-region (R) of the calibration image, and wherein adjusting the target image comprises changing brightness levels of pixels in a corresponding sub-region (R’) of the target image according to the personalized profile, so as to have the same modified, overall brightness level as the sub-region (R) of the calibration image.

10. The method according to any one of claims 1-9, wherein the personalized profile comprises a modified, overall brightness level of a sub-region (R) of the calibration image, and wherein the method further comprises: receiving, from a gaze detection device, an indication of the user’s gaze direction; associating the subregion (R) of the calibration image with a section (S) in the user’s FOV, based on the gaze direction; andadjusting the target image by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the personalized profile, so as to have the same modified, overall brightness level as the subregion (R) of the calibration image.

11. The method according to any one of claims 1-10, wherein the modified visual parameters comprise a contrast level of a sub-region (R) of the calibration image, and wherein adjusting the target image comprises changing brightness levels of pixels in a corresponding sub-region (R’) of the target image according to the personalized profile, thereby forcing a contrast level of the corresponding sub-region (R’) to have the same value as the modified contrast level of the sub-region (R) of the calibration image.

12. The method according to any one of claims 1-11, wherein the modified visual parameters comprise a contrast level of a sub-region (R) of the calibration image, and wherein the method further comprises: receiving, from a gaze detection device, an indication of the user’s gaze direction; associating the subregion (R) of the calibration image with a section (S) in the user’s FOV, based on the gaze direction; and adjusting the target image by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the personalized profile, thereby forcing a contrast level of the corresponding sub-region (R’) to have the same value as the modified contrast level of the sub-region (R) of the calibration image.

13. The method according to any one of claims 1-12, wherein the at least one calibration image comprises a plurality of regions (R), each defined by a respective Spatial Frequency (SF), and wherein the personalized profile further comprises modified, SF band-specific contrast levels.

14. The method of claim 13, wherein adjusting the target image comprises: defining subregions (R’) of the target image;for one or more subregions (R’), applying a set of spatial filters, thereby producing a respective set of patches (P’), each corresponding to a specific SF band; for each patch (P’), changing brightness levels of pixels according to the personalized profile, thereby forcing a contrast level of the patch (P’) to have the same value as the corresponding SF band-specific contrast level of the personalized profile; superimposing patches (P’) pertaining to the same subregions to produce altered subregions of the target image; and combining the altered subregions of the target image to produce the adjusted target image.

15. The method according to any one of claims 1-14, wherein the at least one calibration image comprises a plurality of regions (R), each defined by a respective SF, and wherein the method further comprises: receiving, from a gaze detection device, an indication of the user’s gaze direction; and based on the gaze direction, defining a Contrast Sensitivity Function (CSF) as part of the personalized profile, wherein the CSF function associates modified contrast levels of at least one subregion (R) with (a) specific SF bands, and (b) specific gazedirections.

16. The method of claim 15, further comprising adjusting the target image by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the CSF function.

17. The method of claim 16, wherein changing brightness levels of pixels in a sub-region (R’) of the target image comprises: applying a set of spatial filters on sub-region (R’) of the target image, thereby producing a respective set of patches (P’), each corresponding to a specific SF band; for each patch (P’), changing brightness levels of pixels according to the CSF function; and applying a weighted superposition function on patches (P’) pertaining to the same subregion (R’) of the target image, to produce an altered version of subregion (R’).

18. The method according to any one of claims 1-17, further comprising: receiving, via the UI, a definition of a sub-region (R) of the at least one calibration image; receiving, via the UI, an indication of a modified visual parameter corresponding to the defined sub-region (R) , thereby determining a region- specific visual parameter value; and applying at least one adaptation to a corresponding region (R’) of the target image, to adjust the target image.

19. The method according to any one of claims 1-18, wherein adjusting the target image comprises applying a blurring spatial filter on at least one region, based on the personalized profile, thereby exploiting visual characteristics of the user to reduce data redundancy in the target image.

20. A system for personalized image enhancement, the system comprising: at least one first camera; a head-mounted projector; a non-transitory memory device, wherein modules of instruction code are stored; and at least one processor associated with the memory device, and configured to execute the modules of instruction code, whereupon execution of said modules of instruction code, the at least one processor is configured to: present, via the head-mounted projector, at least one calibration image to a user; receive, from the user, via a User Interface (UI), indication of one or more modifications to one or more respective visual parameters of the at least one calibration image; defining a personalized profile of the user, based on the one or more modifications; receive, via the at least one camera, a target image; adjusting the target image based on the personalized profile; and present the adjusted target image to the user via the projector.

21. The system of claim 20, wherein the at least one processor is further configured to: receive at least one context data element, defining a type of the target image; andadjust the target image further based on the context data element.

22. The system according to any one of claims 20-21, wherein the at least one processor is further configured to: applying an object recognition algorithm on the target image, to obtain a definition of at least one object of a predetermined type, depicted in the target image; and adjusting the target image further based on the definition of the at least one object.

23. The system according to any one of claims 20-22, wherein the at least one processor is configured to modify the visual parameter in an iterative process, wherein each iteration comprises: receiving, via the UI, an incremental change in a value of the visual parameter; generating a modified version of the calibration image based on the incremental change; and presenting the modified version via the UI as feedback for the user.

24. The system according to any one of claims 20-23 wherein the personalized profile comprises: (i) a modified mean value of distribution of brightness levels in a subregion (R) of the calibration image, and (ii) a modified standard deviation value of the distribution of brightness levels in the subregion (R) of the calibration image.

25. The system of claim 24, wherein the at least one processor is configured to adjust the target image by changing brightness levels of pixels in a corresponding subregion (R’) of the target image according to the personalized profile, thereby forcing a distribution of brightness levels in the corresponding subregion (R’) of the adjusted target image to have the same modified mean and modified standard deviation values as the sub-region (R) of the calibration image.

26. The system according to any one of claims 24-25 further comprising a gaze detection device, and wherein the at least one processor is configured to: receive, from the gaze detection device, an indication of the user’s gaze direction; andassociate the subregion (R) of the calibration image with a section (S) in the user’s Field Of View (FOV) based on the gaze direction.

27. The system according to any one of claims 24-26 wherein the at least one processor is configured to adjust the target image by changing brightness levels of pixels in a subregion (R’) of the target image, which corresponds to the section (S) of the user’s FOV, according to the personalized profile, thereby forcing a distribution of brightness levels in the corresponding subregion (R’) of the adjusted target image to have the same modified mean and modified standard deviation values as the sub-region (R) of the calibration image.

28. The system according to any one of claims 20-27, wherein the personalized profile comprises a modified, overall brightness level of at least one sub-region (R) of the calibration image, and wherein the at least one processor is configured to adjust the target image by changing brightness levels of pixels in at least one corresponding sub-region (R’) of the target image according to the personalized profile, so as to have the same modified, overall brightness level as the at least one sub-region ® of the calibration image.

29. The system according to any one of claims 20-28, wherein the personalized profile comprises a modified, overall brightness level of a sub-region (R) of the calibration image, and wherein the at least one processor is configured to: receive, from a gaze detection device, an indication of the user’s gaze direction; associate the subregion (R) of the calibration image with a section (S) in the user’s FOV, based on the gaze direction; and adjust the target image by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the personalized profile, so as to have the same modified, overall brightness level as the subregion (R) of the calibration image.

30. The system according to any one of claims 20-29, wherein the modified visual parameters comprise a contrast level of a sub-region (R) of the calibration image, and wherein the at least one processor is configured to adjust the target image by changing brightness levels of pixels in a corresponding sub-region (R’) of the target image accordingto the personalized profile, thereby forcing a contrast level of the corresponding sub-region (R’) to have the same value as the modified contrast level of the sub-region (R) of the calibration image.

31. The system according to any one of claims 20-30, wherein the modified visual parameters comprise a contrast level of a sub-region (R) of the calibration image, and wherein the at least one processor is configured to: receive, from a gaze detection device, an indication of the user’s gaze direction; associate the subregion (R) of the calibration image with a section (S) in the user’s FOV, based on the gaze direction; and adjust the target image by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the personalized profile, thereby forcing a contrast level of the corresponding sub-region (R’) to have the same value as the modified contrast level of the sub-region (R) of the calibration image.

32. The system according to any one of claims 20-31 , wherein the at least one calibration image comprises a plurality of regions (R), each defined by a respective Spatial Frequency (SF), and wherein the personalized profile further comprises modified, SF band-specific contrast levels.

33. The system of claim 32, wherein the at least one processor is configured to adjust the target image by: defining subregions (R’) of the target image; for one or more subregions (R’), applying a set of spatial filters, thereby producing a respective set of patches (P’), each corresponding to a specific SF band; for each patch (P’), changing brightness levels of pixels according to the personalized profile, thereby forcing a contrast level of the patch (P’) to have the same value as the corresponding SF band-specific contrast level of the personalized profile; superimposing patches (P’) pertaining to the same subregions to produce altered subregions of the target image; andcombining the altered subregions of the target image to produce the adjusted target image.

34. The system according to any one of claims 20-33, wherein the at least one calibration image comprises a plurality of regions (R), each defined by a respective SF, and wherein the at least one processor is configured to: receive, from a gaze detection device, an indication of the user’s gaze direction; and based on the gaze direction, defining a Contrast Sensitivity Function (CSF) as part of the personalized profile, wherein the CSF function associates modified contrast levels of at least one subregion (R) with (a) specific SF bands, and (b) specific gazedirections.

35. The system of claim 34, wherein the at least one processor is further configured to adjusting the target image by changing brightness levels of pixels in a sub-region (R’) of the target image which corresponds to the section (S) of the user’s FOV, according to the CSF function.

36. The system of claim 35, wherein the at least one processor is further configured to change brightness levels of pixels in a sub-region (R’) of the target image by: applying a set of spatial filters on sub-region (R’) of the target image, thereby producing a respective set of patches (P’), each corresponding to a specific SF band; for each patch (P’), changing brightness levels of pixels according to the CSF function; and superimposing patches (P’) pertaining to the same subregion (R’) of the target image, to produce an altered version of subregion (R’).

37. The system according to any one of claims 20-36, wherein the at least one processor is configured to: receive, via the UI, a definition of a sub-region (R) of the at least one calibration image; receive, via the UI, an indication of a modified visual parameter corresponding to the defined sub-region (R) , thereby determining a region- specific visual parameter value; andapply at least one adaptation to a corresponding region (R’) of the target image, to adjust the target image.

38. A system for personalized image enhancement, the system comprising: a non- transitory memory device, wherein modules of instruction code are stored, and at least one processor associated with the memory device, and configured to execute the modules of instruction code, whereupon execution of said modules of instruction code, the at least one processor is configured to: present at least one calibration image to a user via a User Interface (UI); allow the user to apply, via the UI, one or more modifications to one or more respective visual parameters of the at least one calibration image; receive a target image; apply at least one adaptation to the target image, based on the one or more modified visual parameters of the at least one calibration image, thereby producing an adapted version of the target image; and present the adapted version of the target image to the user.

Images