Method and system for tagging visual features in video material and neuroimaging recovery of visual evoked potentials

Abstract

Systems and methods are disclosed for assessing visual function of a user or patient. The systems and methods may contain steps, including: extracting data from an electrode site of a sensor; obtaining an average of the data across successive video presentations; applying a fast-Fourier transform to the average; determining a signal-to-noise ratio (SNR) of results of the fast-Fourier transform at tagged frequencies; determining a steady-state visual evoked potential (SSVEP) amplitude based on the SNR of the results of the fast-Fourier transform at the tagged frequencies; and determining a ratio of lower to higher spatial frequency SSVEPs to assess the vision function of the user.

Description

METHOD AND SYSTEM FOR TAGGING VISUAL FEATURES IN VIDEO MATERIAL AND NEUROIMAGING RECOVERY OF VISUAL EVOKED POTENTIALSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 63 / 607,400, filed December ?, 2023, which is incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] The present disclosure relates generally to measurements of visual function, e.g., of a user or patient, and the use of such measurements for diagnosing visual impairments, characterizing the progression of visual impairments, and / or treating visual impairments. The present disclosure further relates generally to the use of neuroimaging techniques, including electroencephalography (EEG), magnetoencephalography (MEG), functional near-infrared spectroscopy (fNIRS), ultrafast ultrasound imaging, and other modalities for assessing visual function and related neural responses.BACKGROUND

[0003] Visual perception, or the ability to see and understand the world around us, is important to the quality of life and independence of most humans. In healthy visual processing, light activates photoreceptive cells in the retina of the eye, which innervates an expansive visual processing network comprising over half of the brain’s cortex. Disease or injury to the eyes, to optic nerves, or to the brain can severely compromise an individual’s “visual function,” i.e., the capacity to process visual stimuli. Such visual disorders are a global problem affecting all demographics, but are especially prevalent amongst older adults. Pertinent examples of visualdisorders include age-related macular degeneration (AMD), diabetic retinopathy, retinitis pigmentosa, optic neuritis, glaucoma, myopia, cortical visual impairment, and visual spatial neglect.

[0004] The ability to treat visual disorders and the ability to investigate novel interventions and treatment pathways depend upon reliable and valid indices of visual processing. For example, it is important for clinicians to understand how a patient’s visual processing capacity is changing as a disease progresses or as an injury recovers. Further, it is important for researchers to be able to objectively measure how different interventions may affect these changes in visual function.

[0005] Clinical approaches for assessing vision function commonly rely on behavioral data and self-reporting. For example, common behavioral measures of visual function include the use of Snellen charts (charts comprised of lines of progressively smaller text to evaluate the smallest print size which patients can reliably report), evaluations of reading speed, Amsler grids (in which patients report distortions or missing areas while viewing a grid pattern), and automated visual field tests (in which patients report whether visual stimuli presented at different parts of their visual field can be seen or not). However, such behavioral assessments typically rely on a “threshold,” i.e. , determining whether a stimulus is visible to a user or not. These assessments are affected by personal biases and confidence, which can vary significantly across populations. For instance, older adults may struggle with consistent reporting due to cognitive decline or motor impairments, while pediatric patients may lack the attention span or communication skills to provide reliable feedback. Furthermore, traditional tests often fail to capture the nuanced visual processing deficits that arise in complex conditions, such as cortical visual impairment or macular degeneration. These limitations underscore the need forneuroimaging-based approaches that directly measure neural responses, bypassing subjective reporting entirely. Thus, these assessments might not be sensitive to subtle changes in visual function. Further, such behavioral assessments require patients to have the capacity to provide a subjective response, which may be impossible or impractical for some patient groups (e.g., infants, young children, traumatic brain injury and stroke patients, and patients with intellectual disabilities). Still further, such techniques do not account for the complexity of real world images. Yet further, such techniques may require patients to gaze at a single central fixation point, which may be impractical for conditions that affect central vision (e.g., age- related macular degeneration (AMD)).

[0006] To address the aforementioned limitations in the known techniques of assessing vision function, methods and a system for obtaining neuroimaging-based measures of disordered visual function are described herein. The methods and the system described herein may utilize full-color and motion videos with natural image statistics or other complex structures of visual features, rather than simplified images. The methods and the system described herein may be used to measure subtle differences in the scale of neural responses, rather than discriminating based on a threshold. Further, using the methods and the system disclosed herein, classification of visual field deficits may be described based on the retinotopic organization of spatial frequency sensitivity, and thus the methods may be performed without a patient having to gaze at a central fixation point. As such, the methods and system described herein overcome many of the aforementioned disadvantages.

[0007] While the methods described herein are applied to visual function, similar principles may be extended to assess auditory processing (e.g., tagging sound frequencies to measure auditory evoked potentials) or motor function (e.g.,tagging motion patterns to assess motor cortex responses). These approaches enable comprehensive evaluations of sensory and cognitive function.

[0008] The background description provided herein is for the purpose of generally presenting the context of the disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art, or suggestions of the prior art, by inclusion in this section.SUMMARY OF THE DISCLOSURE

[0009] According to certain aspects of the disclosure, systems and methods are described for assessing the visual function of a user, and for preparing video material that may be used to assess the visual function of a user.

[0010] In one aspect, a computer-implemented method of preparing video material that is configured to assess visual function of a user is provided herein. The computer-implemented method may include: determining a video having a resolution above a predetermined threshold, the video comprising a plurality of images varying in frequency and orientation over space; determining a wavelet transform to facilitate a separation of spatial frequencies into specified spatial frequency bands based on a predetermined measure of a visual angle of a screen used to view the video from a predetermined viewing distance; determining a method of applying a frequency tag across visual features and temporal frequencies; generating a power filter to determine how the frequency tag is to be applied to the images; applying, for each of the images, the wavelet transform to a value of a color space channel for each of the images; and extracting, for each of the images, a power for each wavelet scale,wherein the power represents a contrast of a tagged feature for each of the images, and wherein the power verifies embedding of the tagged feature across time.

[0011] In another aspect, a computer-implemented method of assessing vision function of a user is provided herein. The method may utilize video material prepared using other methods disclosed herein. The computer-implemented method may include: extracting data from an electrode site of a sensor; obtaining an average of the data across successive video presentations; applying a fast-Fourier transform to the average; determining a signal-to-noise ratio (SNR) of results of the fast-Fourier transform at tagged frequencies; determining a steady-state visual evoked potential (SSVEP) amplitude based on the SNR of the results of the fast-Fourier transform at the tagged frequencies; and determining a ratio of lower to higher spatial frequency SSVEPs to assess the vision function of the user.

[0012] In addition to conventional analysis techniques such as Fourier transforms and signal-to-noise ratio calculations, machine learning models (e.g., neural networks) can be employed to classify neural data and identify complex response profiles. These methods enable the detection of subtle patterns, aiding in diagnosis and response prediction.

[0013] The systems and methods disclosed herein are adaptable to various neuroimaging modalities, including EEG, MEG, fNIRS, functional MRI (fMRI), ultrafast ultrasound imaging, and other technologies. This flexibility ensures applicability across diverse clinical and research settings, accommodating a wide range of patient needs and environmental constraints.

[0014] In another aspect, a system for assessing visual function of a user is provided herein. The system may include: one or more processors; and one or more computer readable media storing instructions that are executable by the one or moreprocessors to perform operations comprising: determining a video having a resolution above a predetermined threshold, the video comprising a plurality of images, each image varying in frequency and orientation over space; determining a wavelet transform to facilitate a separation of spatial frequencies into spatial frequency sub-bands based on a predetermined measure of a visual angle taken up by a viewing screen at a specified viewing distance; determining a method of applying a frequency tag; generating a power filter to determine how the frequency tag is to be applied to the images; applying, for each of the images, the wavelet transform to a value of a color space channel for each of the images; and extracting, for each of the images, a power for each wavelet scale, wherein the power represents a higher spatial frequency in the spatial frequency sub-bands, to determine at least one tagged video.

[0015] The video tagging techniques disclosed herein may utilize a variety of computational methods, including but not limited to wavelet transforms, Fourier filtering, spatial-temporal tagging, or machine learning-based feature tagging. These methods may enable tagging of visual features such as motion, shape, orientation, or color across a wide range of spatial and temporal resolutions.

[0016] The methods disclosed herein are not limited to visual assessments. Tagging techniques may be applied to other sensory modalities, including auditory, tactile, and motor systems, or to cognitive tasks such as memory and attention.

[0017] Additional objects and advantages of the disclosed embodiments will be set forth in part in the description that follows, and in part will be apparent from the description, or may be learned by practice of the disclosed embodiments. The objects and advantages of the disclosed embodiments will be realized and attainedby means of the elements and combinations particularly pointed out in the appended claims.

[0018] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed.BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate several embodiments and together with the description, serve to explain the principles of the disclosure.

[0020] FIG. 1 A depicts an exemplary computer system that may be used to execute embodiments of the methods disclosed herein.

[0021] FIG. 1 B depicts an exemplary software platform that may be used to execute embodiments of the methods disclosed herein.

[0022] FIG. 2 depicts an exemplary workflow for utilizing the embodiments disclosed herein to develop video material that may be used to assess the visual function of a user.

[0023] FIG. 3 depicts several still-frame images from examples of videos that may be used in the workflow described in reference to FIG. 2.

[0024] FIG. 4 depicts an artificial grating image that may be used and / or generated in the workflow described in reference to FIG. 2.

[0025] FIG. 5A depicts a graph showing the parameters of an example power filter that generates flicker by up-regulating and down-regulating the power of oblique angles relative to cardinal angles, with time in seconds being shown on the X axis and power in pV being shown on the y axis.

[0026] FIG. 5B depicts a graph that further shows the parameters of an example power filter that generates flicker by up-regulating and down-regulating the power of oblique angles relative to cardinal angles, with wavelet angle in radians being shown on the X axis and power in pV being shown on the y axis.

[0027] FIG. 6A depicts reproductions of steered wavelet functions, where each equiangular channel shows a wavelet function being steered to rotate a further predetermined amount.

[0028] FIG. 6B depicts an artificial zoneplate image that includes differences in spatial frequency and orientation organized across space.

[0029] FIG. 6C depicts a power recovered for each angular wavelet channel at different scales, with wavelet angle in radians being shown on the gradient key.

[0030] FIG. 6D depicts example power filters across wavelet angles, where one filter suppresses visual information at one oblique angle and a second filter suppresses visual information at the other oblique angle.

[0031] FIG. 6E depicts two filters that may be used to generate a flicker signal by scaling between the two filters.

[0032] FIG. 7 depicts an exemplary workflow for utilizing the embodiments described herein to obtain data from a user.

[0033] FIG. 8 depicts an exemplary workflow for utilizing the embodiments described herein to analyze data that has been obtained from a user to assess the vision function of the user.

[0034] FIG. 9A depicts a cubic spline interpolation of mean steady-state visual evoked potential (SSVEP) amplitude across electrode positions.

[0035] FIG. 9B depicts a graph that shows ratio of amplitudes across spatial frequencies for AMD patients and for control patients.

[0036] FIG. 10 is a simplified functional block diagram of a computer system that may be configured as a computing device for executing the processes described herein.DETAILED DESCRIPTION OF EMBODIMENTS

[0037] The terminology used below may be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific examples of the present disclosure. Indeed, certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section. Both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the features, as claimed.

[0038] In this disclosure, the term “based on” means “based at least in part on.” The singular forms “a,” “an,” and “the” include plural referents unless the context dictates otherwise. The term “exemplary” is used in the sense of “example” rather than “ideal.” The terms “comprises,” “comprising,” “includes,” “including,” or other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, or product that comprises a list of elements does not necessarily include only those elements, but may include other elements not expressly listed or inherent to such a process, method, article, or apparatus. Relative terms, such as “about,” “approximately,” “substantially,” and “generally,” are used to indicate a possible variation of ±10% of a stated or understood value. In addition, the term “between” used in describing ranges of values is intended to include the minimum and maximum values described herein. The use of the term “or” in the claims andspecification is used to mean “and / or” unless explicitly indicated to refer to alternatives only or the alternatives are mutually exclusive, although the disclosure supports a definition that refers to only alternatives and “and / or.” As used herein “another” may mean at least a second or more.

[0039] As used interchangeably herein, the terms “user” and “patient” generally encompass any person or entity, such as a researcher and / or a care provider (e.g., a doctor, etc.), that may desire information, resolution of an issue, or engage in any other type of interaction with a provider of the systems and methods described herein (e.g., via an application interface resident on their electronic device, etc.).

[0040] As used interchangeably herein, the terms “electronic application,” “application,” “program,” or the like generally encompass software that is configured to interact with, modify, override, supplement, and / or operate in conjunction with other software, and / or the terms may refer to software that is configured to be machine-readable.

[0041] As used herein, the term “spatial frequency” refers to the rate at which information changes over space. Thus, it shall be understood that in the context of this disclosure, the more fine and detailed visual information is, the higher the corresponding spatial frequency. The term “spatial frequency” may refer to a set of spatial frequency bands or sub-bands, e.g., that may be characterized in terms of high spatial frequency portions and low spatial frequency portions (e.g., “specified” spatial frequency bands).

[0042] As used herein, the term “steady-state visual evoked potential” (SSVEP) refers to a patient or user’s oscillatory neural response to a periodically flickering visual stimulus.

[0043] Several types of visual impairments are prevalent across the global population. For example, age-related macular degeneration (AMD), which is a leading cause of vision loss and blindness among people aged 65 and older, is characterized by the deterioration of a small area in the center of the retina that is responsible for sharp, central vision. As another example, glaucoma typically begins with visual changes and the development of scotomas in peripheral vision (as opposed to central vision). As yet another example, patients who experience a stroke or a brain injury may experience visual spatial neglect, which is a profound attentional and processing deficit in one side of space that can be body-centered (i.e. , a deficit in space relative to the viewer) or object-centered (i.e. , deficit in space relative to objects the view perceives). Several other causes of visual impairment exist as well.

[0044] To measure indices of visual impairment in patients (and thus to make determinations regarding diagnosis and treatment), a variety of conventional methods that are based on behavioral measures are commonly utilized. These conventional methods often involve determining whether a stimulus that is presented to a patient is seen (i.e., making a “threshold”-based determination). However, these methods have several drawbacks in terms of accuracy, precision, and applicability across patient groups.

[0045] To circumvent the limitations of tests that are based on behavioral measures, visual testing may be performed that utilizes neuroimaging and visual stimuli. These techniques may involve recording neural responses to visual stimuli, e.g., using electroencephalography (EEG), which is a relatively affordable and widely available neuroimaging technique that measures the electrical activity generated byneurons in the brain’s cortex using electrodes placed at the scalp, or by using magnetoencephalography (MEG).

[0046] While EEG and MEG are described as primary modalities for neuroimaging, other approaches such as functional near-infrared spectroscopy (fNIRS), functional MRI (fMRI), and ultrafast ultrasound can be used to detect neural activity with varying spatial and temporal resolutions. These alternative modalities offer the flexibility to adapt the tagging and analysis methods for specific patient needs, including deeper cortical imaging or portable bedside applications.

[0047] In one sub-set of neuroimaging-based techniques known as visual evoked potential (VEP) measures of visual acuity utilize neural responses to a presentation of a visual stimulus. The extension of this technique is steady-state visual evoked potentials (SSVEPs). Steady-state visual evoked potentials are oscillatory neural responses to a periodically flickering visual stimulus. In particular, when a visual stimulus is flickered at a set temporal frequency and is observed by a patient, the visual cortical neurons involved in processing that stimulus respond at the same frequency, and a response may be detected in a resulting EEG (or MEG) signal. By converting the EEG signal from the time-domain to the frequency domain (e.g., using methods such as fast Fourier transforms (FFTs) or wavelet decomposition), SSVEP amplitudes may be used to evaluate the threshold spatial frequency for which SSVEPs can no longer be detected by a user. SSVEPs may be elicited by reversing the black and white elements of a textured grating at a set rate, and thus when the spatial frequency becomes too high to resolve, the pattern may appear to be a uniform gray and will not evoke an SSVEP.

[0048] A drawback of VEP measures of visual acuity is that, much like many behavioral tests, the techniques may utilize a threshold and thus may not allow forany exploration of subtle changes in visual function or response profiles. Indeed, while visual acuity scores derived with this SSVEP threshold-based approach align well with objective behavioral measurements in general, the correlation may be weaker for patients with visual deficits in specific parts of their visual fields (e.g., patients with macula pathologies, glaucoma, or cerebral visual impairment). Glaucoma patients, for example, typically lose peripheral vision before central vision. As peripheral vision is largely sensitive to low spatial frequency information, the threshold for high spatial frequency detection may remain intact until the disease is very advanced.

[0049] Another sub-set of neuroimaging-based techniques rely on principles similar to VEP visual acuity protocols, but rather than presenting a single flickering grating pattern to a patient, a dartboard-like checkerboard including concentric rings of black and white gratings may be used. Visual field deficits may be probed by measuring the neural response to pattern reversals in specific regions of the checkerboard stimulus, which may be called “multifocal visual steady-state evoked potential protocols” (MFSSVEPs). In some techniques, checkerboard segments may employ more periodic flickering to evoke SSVEPs at more specific frequencies, and these may be referred to as “steady-state multifocal visual evoked potential protocols” (SSMFVEPs). The checkerboard patterns that may be used in VEP-based visual field mapping protocols may increase in spatial frequency as the concentric segments approach a central point, i.e. , the point at which the patient is asked to maintain fixation. Retinal ganglion cells, the neurons that respond to photoreceptor cells in the retina, are most densely packed in the eye’s fovea, the part of the retina at the center of the visual field. In the fovea, two neighboring photoreceptive cells may innervate two separate retinal ganglion cells, and thus a person may detectdifferences in light between these two neighboring retinal locations. In contrast, in peripheral vision, a single retinal ganglion cell may be innervated by thousands of photoreceptors. The area of the visual field in which visual stimulation can be detected by a neuron is referred to as that neuron’s “receptive field.” Thus, neurons mapped to central vision typically have smaller receptive fields, and are sensitive to higher spatial frequencies. Neurons mapped more peripherally typically have larger receptive fields and thus are sensitive to lower spatial frequencies. Exploiting this organization of the visual field with the dartboard-like organization of the checkerboard stimulus used to evoke MFVEPs may be used to ensure that evoked responses are maximal for all stimulated visual field locations.

[0050] One drawback of MFVEP-based techniques is that these techniques may require patients to maintain fixation at the center of the display, such that the patients’ visual field lines up correctly with the checkerboard stimulus. This may pose a barrier for many of the patient groups the technique is designed to study (e.g., AMD patients, who experience blind spots at central vision and / or have trouble maintaining fixation).

[0051] Notably, both aforementioned sub-sets of neuroimaging techniques (for measuring visual function) rely on simplified visual stimuli, such as black and white gratings and plaid textures. The visual information that patients must parse in real life may share little resemblance to these simplified visual stimuli. Indeed, where VEP acuity and MFVEP tests may present a single spatial frequency across the whole screen or evaluated region, the natural scenes the visual system encounters in real-life have much more complicated mathematical properties with overlapping visual features throughout the visual field.

[0052] Despite the heterogeneity across the natural visual scenes commonly encountered by humans every day, natural scenes share a well-defined mathematical structure, such that the luminance of different spatial and temporal frequencies follows a stereotyped 1 / / pattern, where lower frequencies tend to have larger variations in luminance and higher frequencies have proportionally smaller variations. The human brain is sensitive to this structure, and is more sensitive to discriminating visual information when spatial frequency is structured according to the temporal and spatial statistics found in natural scenes. Indeed, there may be marked differences in how natural scenes are processed compared with simplified textured patterns. Lower spatial frequency information from retinal ganglion cells in peripheral vision is conveyed to the visual cortex through a faster cellular pathway than higher spatial frequency information from central vision. In turn, when processing natural scenes, lower spatial frequency information may be processed earlier to extract context. This information may be fed back down the visual hierarchy and influences how processing resources are distributed amongst high spatial frequency information. This fast and automatic feedforward / feedback process may impact neural responses to visual information of distinct spatial frequencies, and may be taken into account when evaluating a patient's visual function.

[0053] In addition to lacking the mathematical structure of features in natural scenes, typical stimuli used in the aforementioned neuroimaging-based vision assessments lack both motion and color. Visual cortical areas responsible for motion discrimination are preferentially innervated by the faster magnocellular pathway that primarily carries low-spatial frequency information from peripheral vision. Thus moving stimuli evoke larger responses outside of central vision, and responses to moving stimuli are less sensitive to light and contrast conditions. Indeed, techniquesthat may be called “motion perimetry” may be used specifically to take advantage of this phenomenon, as tests involving moving stimuli are often more sensitive to subtle changes in visual function. There are also distinct retinotopic differences in color perception. The retina is composed of two different types of photoreceptor cells; cone cells, which are sensitive to distinct wavelengths of light (i.e. , different colors), are largely contained to the fovea. Rod cells, which are more sensitive in low-light conditions and cannot differentiate between colors, are absent in central vision and become more and more prevalent throughout peripheral vision. This is why, when star-gazing, it is often possible to see faint stars in peripheral vision which seem to disappear when you look directly at them. As such, we would expect to find distinct patterns of retinotopically mapped responses for moving and colorful stimuli.

[0054] To address these limitations, methods and systems are disclosed herein that may be used to take neuroimaging-based measurements of disordered visual function for a user.

[0055] The methods and the system described herein involve tagging low- level visual features in video material to allow a neural response to these features to be recovered from corresponding neuroimaging data. The methods may apply to both the encoding of video frames to induce tags and the analysis of neuroimaging data recovered from patients watching the tagged videos.

[0056] The concept summarized above, and further elaborated upon herein, overcomes several issues faced by conventional techniques. For instance, the methods and the system described herein may utilize full-color and motion videos with natural image statistics or other complex structures of visual features, which more accurately accounts for the complexities of real world images than many known techniques do. Further, the methods and the system described herein may beused to measure subtle differences in the scale of neural responses, rather than discriminating based on a threshold. Further, using the methods and the system disclosed herein, classification of visual field deficits may be described based on the retinotopic organization of spatial frequency sensitivity, and thus the methods may be performed without a patient having to gaze at a central fixation point.

[0057] In an aspect, the collective concepts presented in this disclosure offer concrete and tangible applications in neuroscience and neuroengineering. These concepts represent improvements in computer technology by introducing innovative applications at the intersection of neuroscience and computing. For instance, the novel concepts utilize video tagging as a tool for neurostimulation, a novel application of software in healthcare, and the novel concepts further utilize the detection and analysis of signals resulting from neurostimulation. The methods disclosed herein may be used to assess patients’ visual function, to dynamically track visual function for neurotherapies, and to study visual perceptual processing under naturalistic viewing conditions.

[0058] The novel processes described herein also advance the technical field of neurostimulation and cognitive therapy by making the ability to measure vision function more accurate and more applicable across patient groups. Existing methods of measuring visual function may be impractical for patient groups or for patient conditions, and existing methods may not account for subtle changes in visual impairment or for the complexity of real world images. The processes discussed here, by contrast, may be employed across many patient groups and may be used to more accurately and precisely detect indices of patient visual function. Additionally, the system’s ability to derive information from a neural response to the tagged neurostimulatory video without the need for subjective feedback furtherenhances accessibility, making the technology scalable and practicable for widespread use.

[0059] The concepts described in this neurostimulatory approach cannot practically be performed in the human mind due in part to the complexity, scale, and real-time nature of the computational tasks involved. For instance, the use of EEG signals, eye-tracking data, and tagged video material allows for the accurate measurement of indices of visual impairment (e.g., signals) that cannot be performed in the human mind. Data is gathered for a patient in real time based on complex data sets that may include neural signals, video frame data, patient-specific responses, a condition being investigated, etc. Human cognition is not equipped to process and interpret this type of data instantaneously or with such a high level of precision to determine a level of a patient’s visual impairment.

[0060] The described tagging methodology may also be applied to other sensory systems. For instance, auditory stimuli with frequency-specific tags can be used to assess auditory evoked potentials for diagnosing hearing impairments. Similarly, tactile stimuli with spatial frequency tags may measure somatosensory processing. Beyond sensory systems, tagged cognitive tasks may evaluate executive function or attention.

[0061] The subject matter of the present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific exemplary embodiments. An embodiment or implementation described herein as “exemplary” is not to be construed as preferred or advantageous, for example, over other embodiments or implementations; rather, it is intended to reflect or indicate that the embodiment(s) is / are “example” embodiment(s). Subject matter may be embodied in a variety ofdifferent forms and, therefore, covered or claimed subject matter is intended to be construed as not being limited to any exemplary embodiments set forth herein; exemplary embodiments are provided merely to be illustrative. Likewise, a reasonably broad scope for claimed or covered subject matter is intended. Among other things, for example, subject matter may be embodied as methods, devices, components, or systems. Accordingly, embodiments may, for example, take the form of hardware, software, firmware, or any combination thereof. The following detailed description is, therefore, not intended to be taken in a limiting sense.

[0062] Throughout the specification and claims, terms may have nuanced meanings suggested or implied in context beyond an explicitly stated meaning. Likewise, the phrase “in one embodiment” or “in some embodiments,” or “in one aspect” or “in some aspects” as used herein does not necessarily refer to the same embodiment or aspect, and the phrase “in another embodiment” or “in another aspect” as used herein does not necessarily refer to a different embodiment or aspect. It is intended, for example, that claimed subject matter include combinations of exemplary embodiments in whole or in part.Systems

[0063] FIG. 1 A depicts an exemplary system by which the methods described herein may be executed. Exemplary system 100 includes a data collection component 10, a database 20, and a data intelligence component 30, each operably connected to each other via network 40. Alternatively, or additionally, one or more of the components may be connected with another component locally without reliance on network connection; e.g., through a wired connection.

[0064] As disclosed herein, data collection component 10 of system 100 may include a device or machine with which electrical activity in the brain may bemeasured. In some embodiments, data collection component 10 may be an electroencephalograph (EEG) machine that contains, or that is configured to support, one or more electrodes, amplifiers, filters, analog to digital converters, etc., by which to conduct an EEG test. Recent advancements in EEG technology have enabled the development of portable, consumer-grade devices, such as headsets with dry electrodes, which require minimal setup and are well-suited for at-home diagnostics. Additionally, alternative neuroimaging modalities like functional near-infrared spectroscopy (fNI S) and ultrafast ultrasound may be employed to gather complementary data, particularly in scenarios requiring higher spatial resolution or portability. These systems provide flexibility for applications across diverse clinical and non-clinical environments. Other devices, such as a magnetoencephalography (MEG) machine, may be used. In some aspects, data collection component 10 may be a database that receives EEG test data from one or more other sources. In other aspects, data collection component 10 may be any other brain recording device or modality that may convey information about neural activity. Consumer-grade virtual and augmented reality headsets that integrate data collection components into the headsets may be another form of device that can be utilized for data collection component 10. Data collected by data collection component 10 may be data that is elicited and collected from a patient in response to visual stimuli from tagged video material, as will be described in more detail in reference to the various workflows disclosed herein.

[0065] Database 20 used to store data obtained from data collection component 10 may be, for example, a remote cloud server (e.g., one or more large storage buckets such as simple storage service “S3” buckets, etc.) from which data may be retrieved on demand. As further examples, data processing, analysis, andclassification may be performed in cloud-based environments using services like cloud-based data processing platforms, serverless computing, cloud-based machine learning platforms, and the like.

[0066] In some embodiments, database 20 is a local database that stores data retrieved from another device (e.g., a user device or a server). In some embodiments, database 20 may store data retrieved in real-time from internet searches. In some embodiments, database 20 may send data to and receive data from one or more of the other functional modules, including, but not limited to, data collection component 10, data intelligence component 30, and network 40. In some embodiments, some or all real-sample data and / or synthetic sample data may be stored on database 20. In some embodiments, database 20 may be a database local to the other functional modules. In some embodiments, database 20 may be a remote database that may be accessed by the other functional modules via wired or wireless network connection. In some embodiments, database 20 may include a local portion and a remote portion.

[0067] Data acquired by the data collection component 10 may be transferred to database 20 via network 40 or a direct, local or network connection. In some embodiments, the collected data may be analyzed by data intelligence component 30, via network 40 or a local or network connection. FIG. 1 B depicts exemplary functional modules that may be implemented to perform tasks of data intelligence component 30.

[0068] FIG. 1 B depicts an exemplary computer system 110 for using techniques discussed herein, for example, during a process of measuring patient visual function using tagged video material. Exemplary system 110 may practice the techniques discussed herein by implementing, on one or more computer devices, auser input and output (I / O) module 120, a memory or a database 130, a data processing module 140, a data analysis module 150, a classification module 160, a network communication module 170, and any other functional modules that may be needed for carrying out a particular task (e.g., an error correction or compensation module, a data compression module, etc.). These modules may correspond to the modules of FIG. 1A. For example, database 130 may correspond to database 20, modules 140, 150, 160, and 170 may correspond to data intelligence component 30, and the input aspect of module 120 may correspond to data collection component 10.

[0069] User I / O module 120 may further include an input sub-module, such as a keyboard, MEG, EEG, eye tracking data, and an output sub-module, such as a display (e.g., a printer, a television, a smartphone, a monitor, a virtual reality (VR) device, and / or a touchpad). In some embodiments, all functionalities may be performed by one computer system. In some embodiments, the functionalities are performed by more than one computer system. The various modules (e.g., for data processing, analysis, classification, communication, etc.) may be one or more processes executing in a distributed computing environment. For instance, in some embodiments, one or more components of the computer system 110 may be network accessible via cloud infrastructure. For example, the database 130 used to store data may be stored in one or more remote cloud servers. In this regard, the database may be one or more large storage buckets (e.g., cloud-based storage buckets such as simple storage service “S3” buckets, etc.) from which data may be retrieved on demand. As another example, data processing, analysis, and classification may be performed in cloud-based environments using services likecloud-based data processing platforms, serverless computing, cloud-based machine learning platforms, and the like.

[0070] Also disclosed herein, a particular task may be performed by implementing one or more functional modules. In particular, each of the enumerated modules itself may, in turn, include multiple sub-modules implementing one or more techniques discussed herein. For example, data processing module 140 may include a sub-module for data quality evaluation (e.g., for performing iterative refinement and validation), a sub-module for normalizing any assigned weights to ensure that the weights contribute proportionally to the overall response, a sub-module for performing interpolation or extrapolation, and the like.

[0071] In some embodiments, a user may use I / O module 120 to manipulate data that is available either on a local device or can be obtained via a network connection from a remote service device or another user device. For example, I / O module 120 may allow a user, e.g., via a keyboard, a mouse, or a touchpad, to perform data analysis via a graphical user interface (GUI). In some embodiments, a user may manipulate data via voice control. In some embodiments, user authentication may be required before a user is granted access to the data being requested. In some embodiments, user I / O module 120 may be used to manage various functional modules. For example, a user may request via user I / O module 120 input data while an existing data processing session is in process. A user may do so by selecting a menu option or type in a command discretely without interrupting the existing process. In another example, a user may utilize user I / O module 120 to set various thresholds, configure sample matching settings, and / or provide other instructions to computer system 110 that dictate how electrical signalsin the brain are captured and / or monitored. As disclosed herein, a user may use any type of input to direct and control data processing and analysis via I / O module 120.

[0072] In some embodiments, system 110 further comprises database 130.In some embodiments, database 130 includes a local database that may be accessed via user I / O module 120. In some embodiments, database 130 includes a remote database that may be accessed by user I / O module 120 via a network connection. In some embodiments, database 130 is a local database that stores data retrieved from another device (e.g., a user device or a server). In some embodiments, memory or database 130 may store data retrieved in real-time from internet searches. In some embodiments, database 130 may send data to and receive data from one or more of the other functional modules, including, but not limited to, a data collection component (not shown), data processing module 140, data analysis module 150, classification module 160, network communication module 170, and etc. In some embodiments, some or all real-sample data and / or synthetic sample data may be stored on database 130.

[0073] As discussed in reference to database 20, database 130 may be a database local to the other functional modules. In some embodiments, database 130 may be a remote database that may be accessed by the other functional modules via wired or wireless network connection (e.g., via network communication module 170). In some embodiments, database 130 may include a local portion and a remote portion.

[0074] In some embodiments, system 110 comprises a data processing module 140. Data processing module 140 may receive the real-time data from I / O module 120 or database 130. In some embodiments, data processing module 140 may perform standard data processing algorithms, such as one or more of noisereduction, signal enhancement, normalization, interpolation and / or extrapolation, etc. In some embodiments, data processing module 140 may be configured to process received and / or collected neural activity data associated with one or more subjects. In various embodiments, data processing module 140 may additionally create a training data set, on which one or more machine-learning models (e.g., for classification, clustering, scoring, etc.) may be trained.

[0075] In some embodiments, system 110 comprises a data analysis module 150. In some embodiments, data analysis module 150 includes identifying brain activity patterns associated with particular medical conditions, as described in connection with data processing module 140.

[0076] In some embodiments, system 110 comprises a classification module 160, which may embody a “machine-learning model” or “trained classifier.” As used herein, a “machine-learning model” or “trained classifier” generally encompasses instructions, data, and / or a model configured to receive input, and apply one or more of a weight, bias, classification, and / or analysis on the input to generate an output. The output may include, for example, a classification of the input, an analysis based on the input, a design, process, prediction, and / or recommendation associated with the input, or any other suitable type of output. A machine-learning model is generally trained using training data, e.g., experiential data and / or samples of input data, which are fed into the model in order to establish, tune, or modify one or more aspects of the model, e.g., the weights, biases, criteria for forming classifications or clusters, or the like. Aspects of a machine-learning model may operate on an input linearly, in parallel, via a network (e.g., a neural network), or via any suitable configuration.

[0077] The execution of the machine-learning model(s) may include deployment of one or more machine-learning techniques, such as k-nearest neighbors, linear regression, logistic regression, random forest, gradient boosted machine (GBM), deep learning, a deep neural network (e.g., recurrent neural network (RNN), convolutional neural network (CNN), transformers) and / or any other suitable machine-learning technique. Supervised, semi-supervised, and / or unsupervised training may be employed. For example, supervised learning may include providing training data and labels corresponding to the training data, e.g., as ground truth. Unsupervised approaches may include clustering, classification or the like. K-means clustering or K-Nearest Neighbors may also be used, which may be supervised or unsupervised. Combinations of K-Nearest Neighbors and an unsupervised cluster technique may also be used. Any suitable type of training may be used, e.g., stochastic, gradient boosted, random seeded, recursive, epoch or batch-based, etc.

[0078] Techniques discussed herein may also be implemented using multiple machine learning models, which may be executed in series and / or in parallel. For example, a first machine learning model may detect and / or interpret a neurological signal from the patient, and a second machine learning model may generate neurostimulatory imagery to present to the patient to achieve the desired result.

[0079] Neurostimulatory imagery may be generated using deep learning models. The deep learning models may have pre-trained weights or the weights may be learned from training on collected datasets which may combine visual stimulation, neural recordings, and behavioral recordings. The visual stimuli may be generated from a deep learning model that generates a group of video frames simultaneously from the stimulation parameters in a closed-loop fashion, and / or they may begenerated frame-by-frame, conditioned on the changing neural data being recorded in real-time. Visual stimulation may also be generated from pre-specified visual features, e.g., gratings or white noise, orfrom combinations of pre-specified visual features and features generated from a deep learning model.

[0080] In an exemplary use case, a machine-learning model may be trained to analyze test data from a test subject whose specific neural activity with respect to a medical condition may be unknown and then subsequently identifying portions or characteristics of the test subject’s brain that may be responsible for or may be resultant of the medical condition. In some embodiments, the one or more parameters may include a score (e.g., a binomial probability score that may be calculated based on logistic regression analysis). As disclosed herein, the binomial probability score may correspond to the likelihood of a subject having a certain medical condition, the likelihood of a portion of the subject’s brain being active or inactive, the likelihood of a particular stimuli affecting a desired portion of the brain, etc. For example, a score of over a predefined threshold may indicate that a specific stimulus or sequence or set of stimuli has effectively stimulated a non-sensory region of the brain.

[0081] As disclosed herein, network communication module 170 may be used to facilitate communications between a user device, one or more databases, and any other suitable system or device through a wired or wireless network connection. Any communication protocol / device may be used, including, without limitation, a modem, an Ethernet connection, a network card (wireless or wired), an infrared communication device, a wireless communication device, and / or a chipset (such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, cellular communication facilities, etc.), a near-field communication (NFC), a Zigbeecommunication, a radio frequency (RF) or radio-frequency identification (RFID) communication, a PLC protocol, a 3G / 4G / 5G / LTE based communication, and / or the like. For example, a user device having a user interface platform for processing / analyzing tumor fraction data may communicate with another user device with the same platform, a regular user device without the same platform (e.g., a regular smartphone), a remote server, a physical device of a remote loT (internet-of- Things) local network, a wearable device, a user device communicably connected to a remote server, and etc.

[0082] Techniques disclosed herein may be used in combination with those discussed in U.S. Pat. No. 10,736,526 and U.S. App. No. 18 / 044,054, each of which are incorporated by reference herein in their entireties.

[0083] The functional modules described herein are provided by way of example. It will be understood that different functional modules may be combined to create different utilities. It will also be understood that additional functional modules or sub-modules may be created to implement a certain utility.Methods of Preparing Tagged Video Material

[0084] Referring now to FIG. 2, an exemplary workflow 200 is provided for preparing video material that may be used to assess the visual function of a user. Preparation of the video material may involve incorporating tags into the video material, where the tags are selected to evoke a particular neurophysiological response. Aspects of the exemplary workflow 200 may be performed in accordance with some or all components described in FIGS. 1 A and 1 B.

[0085] Workflow 200 may be used to prepare video material that is suitable for measuring, detecting, or otherwise assessing the presence (or progression of)age-related macular degeneration (AMD) in a user. AMD patients often develop scotomas (blind spots) in central vision - the part of vision sensitive to high spatial frequencies. Such scotomas have been shown to lead to reorganization of the receptive fields of neurons retinotopically mapped to the peri-scotoma visual field. Specifically, these neuron’s receptive fields typically grow larger and shift outward. As a result, visual cortical neurons grow more sensitive to lower spatial frequency information as they grow less responsive to higher spatial frequency information. Notably, this consideration may not be currently accounted for in any current neuroimaging-based method for evaluating visual function in AMD patients. Thus, in an embodiment, workflow 200 may be used to prepare video material that is suitable for evaluating visual function in AMD patients via neurophysiological response.

[0086] As shown at step 205, workflow 200 involves determining a video having a resolution that is above a predetermined threshold (e.g., >720p), where the video includes a plurality of images, each image varying in frequency and orientation over space. The video may be processed during workflow 200 and ultimately presented to a user, e.g., to elicit signals from the user that can be used to assess vision function of the user. In embodiments, the video may include a full-color video of natural scenes (e.g., visual displays of aspects of nature). The video may include a diverse, dynamic video that features motion and salient subject matter spanning a full display. Exemplary still-frames from videos that may be used in step 205 are shown in FIG. 3. As can be seen in FIG. 3, six sets of six individual still frames are included, where visual information may fill a display for each still frame (e.g., no empty or blank spaces). While workflow 200 describes the preparation of a plurality of videos, it shall be understood that workflow 200 may also be suitable for preparing a single video for presentation to a user (e.g., to assess vision function).

[0087] Returning now to FIG. 2, as shown at step 210, individual frames within each video in the plurality of videos may be “zero-padded” to create square images. The images may have dimensions of a predetermined multiple. “Zeropadding” involves adding extra zeros to the edges of a still image to create an image having a specific shape or size, such that the still image is suitably modified for further processing. As an example, the width and height of each frame may be adjusted to create square images having dimensions that are a multiple of 256, or having dimensions of another predetermined multiple. Zero-padding of the individual frames may be used to enhance computational efficiency, e.g., when applying a wavelet transform.

[0088] As shown at step 215, each square image generated at 210 may be converted from RGB (red, green, blue) color space to HSV (hue, saturation, value) color space. The conversion of the square images may be performed using any suitable technique known to a person of skill in the art.

[0089] As shown at step 220, two-dimensional (2D) steerable wavelet decomposition parameters may be defined that allow for a separation of spatial frequencies greater than and less than a predetermined measure of visual angle (e.g., 2, 3, or 4 cycles per degree, or variations thereof). “Visual angle” may refer to the angular portion, in degrees, of a visual field that an object subtends, and the term may further refer to a visual angle of a screen that is used to view a video at a set viewing distance. For example, “high” spatial frequencies may refer to those that are of >3 cycles per degree of visual angle, and “low” spatial frequencies may refer to those that are of <3 cycles per degree of visual angle, though the threshold may vary. The 2D steerable wavelet decomposition parameters may be set to separate an approximate border between higher spatial frequencies that, typically, can only beresolved in a central 8 degrees of visual angle of a user’s visual field (paracentral vision), and lower spatial frequencies may be more strongly represented in peripheral vision. The 2D steerable wavelet decomposition parameters may not unresolvable by peripheral cells. As one example, for a 720p video scaled to fill a 60x34 cm monitor, e.g., at a viewing distance of 64 cm, a 5thorder Simoncelli isotropic mother wavelet may be applied using 5 scales and steered using 16 equiangular when the two smallest scales are designed for tagging high spatial frequencies, and the two largest scales are designated for tagging low spatial frequencies. A middle-most scale may be left unaltered.

[0090] Steerable wavelets are particularly well-suited for this application as they enable precise decomposition of images into orientation-specific components, facilitating targeted tagging of features such as oblique angles or cardinal edges. Alternatively, other decomposition methods, such as Gabor filtering or Fourier-based techniques, could be employed to achieve similar results. These methods may offer computational advantages in scenarios requiring real-time processing or broader feature coverage. The choice of decomposition method may depend on the specific neural features being targeted and the computational resources available.

[0091] In embodiments, at step 220, an artificial grating image may be generated, and the artificial grating image may be the same size as a video frame. The artificial grating image may systematically shift in spatial frequency from left to right (e.g., using a series of sinusoidal signals of increasing frequency). An example of such an image is depicted in FIG. 4. The artificial image shown in FIG. 4 has been generated such that a spatial frequency of each pixel of the image is known. Users may be able to resolve from approximately 0 cycles per degree to 60 cycles per degree of a visual angle. Pixel density on a display monitor being used to display theplurality of videos may limit the maximum spatial frequency able to be displayed to approximately 10 cycles per degree to 20 cycles per degree at normal viewing distances. The spatial frequency in cycles per degrees of visual angle for an image may be calculated based on viewing distance, display size, and based on whether the video will be presented to users in a format that is scaled-up from an original size. For example, one pixel in a base image may be presented as being spread across two pixels when displayed to users, thereby halving the spatial frequency. Translation of an image size in pixels to degrees of visual angle may be performed using a suitable method known to those known to those skilled in the art. In embodiments, a 2D wavelet transform may be applied with the proposed parameters to assess which spatial frequency is captured by each wavelet scale. Step 220 may be performed once or repeated as many times as desired, e.g., until parameters are identified that provide suitable separation at a desired value (e.g., 3 degrees of visual angle, etc ). In embodiments, the artificial grating image may be used to verify spatial frequencies (e.g., in a set of spatial frequency bands) that are tagged, e.g., with a tagging feature of a stimulus set.

[0092] Returning now to FIG. 2, as shown at step 225, a tagging method may be selected. The method may be a method of applying a frequency tag across visual features and temporal frequencies. For example, for AMD visual function assessment of a user, frequency-tagging may be used to evoke SSVEPs at, e.g., 6 Hz and 7.5 Hz using a suitable instrument, such as a sinusoidal flicker. In deciding on flicker parameters, a monitor refresh rate of monitors being used to display the plurality of videos to a user may be controlled to provide SSVEP evocations at frequencies that are desirable, such as frequencies that are divisible by the monitor refresh rate. Once flicker frequency and display frequency have been determined,two flicker signals spanning from 0 to 1 may be generated, e.g., sinusoidal signals with amplitude 0.5, offset +0.5, frequency 6 and 7.5 Hz, sampling rate at the monitor refresh rate, and a duration of 8 seconds.

[0093] As shown at step 230, a power filter may be generated to determine how a frequency tag will be applied across different orientations. Leaving the cardinal orientations un-tagged may result in smaller SSVEPs, while leaving the semantic content video material perceptually undisturbed. It may be possible to cycle between up- and down- regulating each oblique angle (+45°, -45°). Graphical representations of a power filter that may be used to generate flicker by up- regulating and down-regulating the power of oblique angles relative to cardinal angles are shown in FIGS. 5A and FIG. 5B. These figures are discussed in more detail herein in reference to step 240 of workflow 200.

[0094] In some embodiments, after step 230 in workflow 200, an original framerate of video material being used is compared to a planned monitor refresh rate (e.g., a refresh rate of a monitor on which the video material will be displayed). If the values do not match, a new video refresh rate matching the monitor refresh rate may be interpolated by duplicating or removing frames. As an example, if original video material images are at 30 Hz, but will be used to evoke SSVEPs from a user on a 60 Hz monitor, the videos may be interpolated such that each frame is presented twice. Additional interpolation methods are contemplated herein, e.g., averaging frame content to interpolate.

[0095] As shown at step 235, squared, interpolated video frames of each video in the plurality of videos are cycled, and wavelet transform is applied to the value of the HSV color space for each image. Wavelet transforms may be used in time-frequency analyses and may involve convolving an input signal with a “motherwavelet” function. The mother wavelet function can be scaled (e.g., stretched wider or narrower) to evaluate a power of an input signal at different frequencies. Many options for different mother wavelet functions, which respectively allow for image processing to focus on highly specific features (e.g., circular or textured luminance changes).

[0096] One example of a mother wavelet function is a Simoncelli isotropic wavelet function, which is shown in FIG. 6A. This function may provide an estimate for early visual processing and may facilitate an analysis of change in luminance between neighboring visual locations at specific angles. Each equiangular channel included in the figure shows the wavelet function “steered” to rotate a further 22.5°. FIG. 6B shows an example of an artificial zoneplate image that varies in both spatial frequency and orientation over space for illustrative purposes. When convolved with the Simoncelli wavelet functions illustrated in FIG. 6A, a strength with which different orientations are represented at different parts of the image may be visualized. As shown in FIG. 6C, mother wavelet functions may be scaled across different spatial frequencies to visualize how strongly different orientations may be represented across such spatial frequencies. In embodiments, scaling of mother wavelet functions facilitates reconstruction of an original image after editing a power of specific orientation and scale channels of the image. In embodiments, a power recovered for each angular wavelet varies at different scales (e.g., scale 1 recovers the highest spatial frequencies, whereas scale 5 recovers the lowest spatial frequencies). FIG. 6D shows a graphical representation of the result of systematically altering a power of each of the orientation channels to suppress each of the oblique angles (45°, -45°) in an input image. In particular, example power filters across different wavelet angles are shown. A first filter may suppress visualinformation at one oblique angle (-45°), and a second filter may suppress visual information at the other oblique angle (45°). In embodiments, this systematic alteration may be applied to generate a flicker signal and induce an SSVEP. A power of a tagged feature (e.g., oblique angles at high spatial frequencies) may be systematically up-regulated and down-regulated across successive frames, a graphical representation of the result of which is shown in FIG. 6E. Applying tagging to oblique angles may mitigate a visual system’s sensitivity to cardinal angles. In other words, videos with distorted or flickering content limited to oblique angles may look less distorted and may be more pleasant to view than those where a filter is applied to the cardinal angles. Tags may be stronger when applied to cardinal angles, which may be suitable for patients with very poor vision.

[0097] Returning now to FIG. 2, as shown at step 240 and after wavelet transform has been applied, for each frame, a power may be extracted for each wavelet scale that is designated as representing a predetermined higher spatial frequency (e.g., >3 cycles per degree of visual angle). The power may represent a contrast of a tagged feature and may be used to verify embedding of the tagged feature across time (e.g., in the images). For each angle channel, linear interpolation between two power filters may be performed, where the interpolation is based on the amplitude of the flicker signal for a frame of video. For example, as shown in FIGS. 5A and 5B, if the flicker signal is in the center of its cycle (e.g., approximately amplitude 0.5), then the filter for all channels will be 0.5. If the flicker signal is at its peak, then the filter at the oblique angles will be maximal (1 ) and at baseline (0.5) for the cardinal angles. The extracted power for each angular channel may be multiplied by the corresponding filter value for that channel. In embodiments, step 240 may be repeated for wavelet scales representing lower spatial frequency (e.g., <3 cycles perdegree of visual angle) using the flicker signal for a second flicker frequency. In embodiments, each video in the plurality of videos may be edited twice to counterbalance spatial frequency with flicker frequency. As an example, in a first edited video, high spatial frequencies may be tagged at 6 Hz and low spatial frequencies may be tagged at 7.5 Hz, while in a second edited video, high spatial frequencies may be tagged at 7.5 Hz and low spatial frequencies may be tagged at 6 Hz.

[0098] In embodiments, after step 240 and once power filters have been applied for each frame of video in the plurality of videos, the original frame image (for each video in the plurality of videos) may be reconstructed using the altered power values.

[0099] Although the present disclosure describes the use of 2D steerable wavelets for tagging spatial frequency bands, other methods may be used. For instance, Fourier-based filtering could allow for simpler and computationally efficient decomposition of visual features. Alternatively, spatial-temporal tagging could be employed to target motion-specific visual features, integrating data from optical flow analyses. Additionally, machine learning models (e.g., convolutional neural networks) may be trained to tag complex patterns, shapes, or textures dynamically based on real-time neural or behavioral feedback.

[0100] In embodiments, altered power values for each frame of video in the plurality of videos may be stored for a value channel of the full HSV color space image together with original hue and saturation values. In embodiments, zero- padded values may be cropped out to return each video in the plurality of videos to its respective original resolution.

[0101] As shown at step 245, each frame of video in the plurality of videos may be converted back to RGB color space, and edited video files may be saved. In embodiments, twice as many video files may remain after workflow 200, two for each original video in which high and low spatial frequencies are tagged with each of the flicker frequencies.

[0102] While workflow 200 has been described in the exemplary context of preparing a single video for presentation to a user, e.g., to assess vision function of the user, it shall be understood that workflow 200 may also be suitable for preparing video material that is suitable for assessing vision function of other user groups and / or for preparing a plurality of videos. For example, workflow 200 may be used to prepare video material that is suitable for assessing visual function in patients with glaucoma, for assessing visual function in patients with visual spatial neglect, for assessing visual function of infants and toddlers, and for evaluating several other groups of users as well. It shall be further understood, however, that workflow 200 may be adjusted based on the patient group for which the tagged video material is intended. In embodiments, when video material is being prepared to assess visual function of glaucoma patients, a threshold between spatial frequencies may be adjusted to separate peripheral vision from central vision (e.g., approximately 2 cycles per degree). In embodiments, when video material is being prepared to assess visual function of visual spatial neglect patients, video material may be segmented such that different regions of space are tagged with different flicker frequencies, thereby facilitating tracking of a relative response to visual information in both a view-centric and object-centric framework.

[0103] As described, workflow 200 may be used to prepare tagged video material that is suitable for assessing visual function of users / patients. An exampleworkflow (e.g., that may utilize tagged video material produced using workflow 200) will now be described.Assessing Visual Function to Obtain Data from a User

[0104] An example workflow 700 for assessing visual function to obtain data from a user is shown in FIG. 7. Example workflow 700 may utilize video material that has been prepared using an example of workflow 200, as described herein.

[0105] As shown at step 705, a user may be fitted with an EEG cap (as described herein). Depending upon a desired depth of analysis, the EEG cap that is used may include any number of electrodes ranging from a single electrode to 256 electrodes. Additional electrodes may be included in the EEG cap, if desired. While workflow 700 is described in the exemplary context of using EEG, other forms of data collection may be used (such as those described herein, e.g., MEG).

[0106] While EEG caps are used in the present workflow, alternative neuroimaging devices such as portable fNIRS headsets or ultrafast ultrasound systems may be employed depending on the specific use case. For example, fNIRS may be suitable for bedside applications, while ultrasound imaging can provide deeper cortical insights in cases requiring finer spatial resolution.

[0107] As shown at step 710, a user may be directed to sit in front of a presentation monitor, e.g., using a chin-rest to fix a viewing distance used to determine a spatial frequency cutoff. In examples, the viewing distance ranges from 1 cm to about 200 cm. In a specific example, the viewing distance is 64 cm. The viewing distance may be controlled and maintained, such that processing of accurate spatial frequencies may be assessed. During step 710, the user may be asked to freely view a stimulus video (e.g., video material prepared using workflow 200) whileminimizing unnecessary blinking and movement to reduce artifacts in the EEG recording. In embodiments, the user may view the video material without maintaining fixation on a single point. In other embodiments, the system may process neural responses in real-time, adjusting the tagged stimuli dynamically to optimize neural activation. For instance, if neural signals indicate insufficient engagement with a specific spatial frequency, the system may increase the tagging intensity or shift to a different frequency range. Similarly, adaptive algorithms may alter the video content, such as introducing motion or color changes, to enhance neural responses and improve data quality.

[0108] As shown at step 715, EEG data may be recorded. Step 715 may be performed consecutively or concurrently with step 710. The data may be in the form of signals that are representative of visual cortex activity of a user.

[0109] As shown at step 720, video material (e.g., that has been prepared using workflow 200) may be presented to the user. The number of videos and repetitions may vary and may be controlled based upon a desired vision assessment. In embodiments, onset of each video may be electrically triggered in an EEG amplifier trigger channel with a unique voltage, such that the exact onset time of each video and each condition can be identified.

[0110] As shown at step 725, EEG data recording is stopped and the EEG cap is removed from the user.

[0111] The data obtained during workflow 700 may be stored on a suitable database, such as any of the databases described in reference to the systems depicted in FIGS. 1 A and 1 B. EEG data may be post-processed to reduce noise and signal drift using processing protocols (e.g., filtering, average-reference, IGA blink artifact correction, interpolation of poor channels, etc.).

[0112] In embodiments, video material prepared using workflow 700 includes a predetermined stimulus set that is configured to analyze visual function of a user.

[0113] While workflow 700 focuses on visual function, the analysis pipeline may be adapted for data from other domains, such as auditory evoked responses to tagged sound frequencies or motor cortex responses to tagged motion patterns. Machine learning models may also classify responses across these sensory or cognitive domains.

[0114] The data obtained during workflow 700 may be analyzed to make determinations regarding the tested user’s visual function. An example workflow that may utilize data from workflow 700 will now be described.Analyzing Data Obtained During Visual Function Assessment

[0115] The data obtained during workflow 700 may be analyzed using a method represented by workflow 800 in FIG. 8.

[0116] Prior to the outset of workflow 800, EEG data for a user (e g., that has been obtained using workflow 700) may be pre-processed to reduce noise and signal drift using processing methods using the techniques described herein.Further, data representing two different tagging conditions may be separated out (e.g., high spatial frequency: 6 Hz, low spatial frequency 7.5 Hz, or high spatial frequency 7 Hz, low spatial frequency 6 Hz, though predetermined thresholds may vary). In embodiments, 6 videos are presented to a user and 5 repetitions are performed prior to the onset of workflow 800.

[0117] As shown at step 805, data may be extracted from desired electrode sites on the EEG cap worn by a user (and that was used to obtain the data being analyzed). These electrode sites may be central occipitoparietal electrode-sites. Anytrials that have undesired excessive noise in any channels after preprocessing may be discarded (e.g., those with an amplitude beyond a predetermined level, such as > 300 pV).

[0118] As shown at step 810, an average across all epochs for each condition is taken. Further, a fast-Fourier transform (FFT) is applied to the averaged data.

[0119] As shown at step 815, a signal-to-noise ratio (SNR) of the FFT from step 810 is calculated (e.g., by dividing each point by its four nearest neighbors). The SSVEP amplitude at two flicker frequencies may be extracted from this SNR spectrum. The SSVEP may be computed using two occipitoparietal electrodes, such as POz, 01 , Oz, 02, Iz (e.g., where the SSVEP signal is maximal).

[0120] While this traditional approach provides a reliable measure of neural response strength, machine learning methods offer a complementary strategy for uncovering more complex patterns in the data. For example, supervised learning models, such as support vector machines or neural networks, can classify neural responses based on tagged video features.

[0121] As shown at step 820, an average may be taken across the two conditions to derive the SSVEP amplitude for each spatial frequency. In embodiments, a low spatial frequency response may be taken as the mean of the 6 Hz and 7.5 Hz low-spatial frequency tags, and the high spatial frequency may be taken as the mean of the 6 Hz and 7.5 Hz high spatial frequency tags. In a real-time adaptive framework, the system may monitor these SSVEP amplitudes continuously, dynamically adjusting the stimuli based on the user’s neural responses. For example, if high spatial frequency responses are weak, the system may increase the tagging duration or introduce additional tagged stimuli at finer scales to enhance theneural signal. This approach allows for personalized testing that adapts in real-time to the user’s neural profile.

[0122] As shown at step 825, a ratio of low to high spatial frequency SSVEPs may be computed based on the average across the two conditions taken at step 820. This ratio may correspond to visual function (e.g., presence of AMD scotomas) of a user from which the analyzed data was obtained. Other mathematical operations, aside from the ratio of SSVEPs, may also be used to assess visual function. An example of mean SSVEP amplitudes (in pV) analyzed during workflow 800, e.g., at step 825 are shown in FIG. 9A, where the mean SSVEP amplitudes analyzed during workflow 800 may be used to predict relative location(s) of AMD scotomas in a user from which the analyzed data was obtained. Data obtained during workflow 800 may be used to compare visual function for a given user across treatment days.

[0123] Data obtained during workflow 800 may be used as a benchmark, e.g., for comparison to other users. Data obtained during workflow 800 may be used to derive measures of scotoma position and shape, due in part to the fact that scotomas influence the topographic distribution of SSVEPS to higher and lower spatial frequency information (example shown in FIG. 9A, as discussed). Both behavior field visual field mapping data and SSVEP ratio scores from a cohort of AMD patients may be collected and used together.

[0124] To demonstrate the concept of deriving visual function from SSVEP ratios, 13 users were tested, where 6 of the users did not have AMD and 7 of the users had AMD. The 13 users were shown tagged video material (e.g., prepared using an example of workflow 200 and collected using an example of workflow 700). The data obtained from the 13 users was analyzed using an example of workflow800, and a graphical representation of the results of the analysis is shown in FIG. 9B. As shown in FIG. 9B, the users with AMD exhibited a higher mean SSVEP ratio relative to the control group without AMD. These results are indicative of the fact that workflows 200, 700, and 800 may separately (or collectively) be used to assess vision function of users.Further Applications of Visual Function Assessment

[0125] Further to workflows 200, 700, 800, which may comprise a first aspect disclosed herein, another aspect disclosed herein involves dynamic visual function tracking for neurotherapies.

[0126] In the first aspect disclosed herein, video feature tagging may be used to measure and benchmark visual function (of users) using the neural response or an average across multiple repetitions of presentation. However, according to the second aspect, the method may further be applied to evaluate dynamic changes in visual processing from moment to moment, due in part to shifts in attention or learned changes in neuronal response profiles. The second aspect may thus pertain to dynamic tracking of visual processing for neurotherapeutic applications.

[0127] In embodiments of the second aspect, neurological feedback protocols may embed participants in a closed-loop framework and neuroimaging data may be analyzed in real-time to provide sensory feedback and encourage a targeted change in neural activity. In embodiments, pilot data may be recorded to define a neural target. The neural target may be, as examples, based on data from healthy subjects, patients with the same visual disorder but with stronger performance, and / or on data from the trained subject when visual function was known to be better. SSVEP responses that are associated with a desired neuralspace may be pre-defined. In embodiments, machine learning is used to determine the SSVEP responses that are associated with a desired neural space. In embodiments, algorithms, latent-space mapping, or first principles and theoretical knowledge of visual processing are used to determine the SSVEP responses that are associated with a desired neural space. SSVEP signals may be analyzed in realtime while users observe a stimulus set that is included in tagged video material.

[0128] Machine learning models may also be applied to the processed data. For instance, a recurrent neural network could analyze temporal patterns in EEG responses to classify patient-specific visual deficits. It could be used to predict visual field impairments based on extracted spatial frequency features.

[0129] In further embodiments of the second aspect, entertainment-based approaches may utilize the video tagging methods described herein. Such approaches may utilize flickering visual stimuli. Video-tagging may be used to provide more specific visual stimulation that is purposed to entrain subpopulations of neurons tuned toward tagging features (e g., horizontally oriented lines, central versus peripheral vision, red-colored items, etc ). Such approaches may further be adapted to target complex cortical dynamics using videos generated to contain specific combinations of spatiotemporal features.

[0130] In still further embodiments of the second aspect, video feature tagging may be used to derive cognitive prosthetics to assist patients in processing visual information. In such embodiments, neural responses to tagged visual information (text or videos) may be measured and dynamically altered to boost neural responses. Further in such embodiments, a frame of a video may be shifted to boost perception, e.g., by enlarging subject matter, increasing contrast, etc.

[0131] According to a third aspect disclosed herein, vision research applications may utilize the video tagging methods described thus far. Such applications may involve the study of visual perceptual processing under naturalistic viewing conditions. The use of workflows 200, 700, 800 may allow researchers and practitioners to extract natural response to specific low-level visual features typically used to compose simplified visual stimuli (color, orientation, spatial frequency) in the context of natural scene viewing. Such applications may allow researchers to study cognitive phenomena such as visual attention, reward, perceptual decision-making, and prediction in an ecologically valid context, as well as how these low-level visual features influence the processing of visual scenes. According to the third aspect, tagging may be applied to a feature of study, rather than, e.g., broad spatial frequency bands. For example, red versus green color channels may be tagged at different frequencies or visual information at different orientations.

[0132] According to a fourth aspect described herein, a video-based tagging approach may be utilized in which tracking of visual processing of dynamic visual scenes is involved. The fourth aspect may be useful for, e.g., neurofeedback applications for performance enhancement in healthy users (e.g., users without AMD, glaucoma, or another condition that affects vision function). For example, video game play benefits from incremental performance gains that result from subtle shifts in visual processing, and such gains may be achieved by tracking allocation of attention during play rather than by extensive practice. In accordance with the fourth aspect, neurofeedback protocols may be developed that expedite practice effects by rewarding beneficial shifts in visual processing toward a pattern found for experts.

[0133] Future applications of these methods include adaptive cognitive therapies, real-time feedback for auditory training, and motor function recoveryfollowing injury. By leveraging tagged sensory and cognitive stimuli, the system can be used across a broad range of diagnostic and therapeutic contexts.

[0134] It shall be understood that various features of the aforementioned aspects may be altered across all of the embodiments described herein. Several examples of further variations of the aforementioned embodiments are provided below.

[0135] As one example, image filtering may be controlled when using the methods and systems described herein. Though embodiments have been described that involve filtering via the use of 2D wavelet transforms, image processing speed may benefit from tagging specific orientation features or preserving semantic content instead. In these cases, filtering may involve using a 2D-FFT approach.

[0136] As another example, though the embodiments described herein may involve tagging obliquely oriented luminance edges, other variations are possible. Tags may be applied to a color channel, saturation channel, to a specific color channel in RGB or CYMK color spaces, or in any other color spaces. Tags may be applied at oblique angles to perceptually preserve semantic content of video material. In embodiments, the power filter may take multiple different shapes, tagging cardinal angles or any other specific angles. Though mother wavelets described herein may be oriented grating to simulate and target early visual processing, neurons may be sensitive to highly specific patterns, and thus a different choice of mother wavelet may be used to induce a tag specific to a group of neurons or level of visual processing. In embodiments, users may be asked to hold fixation at a center of a screen or to integrate a closed-loop eye tracking system and dynamically blend between two or more tagged video frames in real-time, such that the correct information is presented at the correct place in the visual field. Inembodiments, spatial frequencies are separated into finer portions (relative to the high and low frequency ranges described herein) to facilitate precise and detailed measures of visual function.

[0137] As yet another example, a tag may be controlled or modified depending on the particular user being tested (or the condition being tested for). For designs that are configured to evoke SSVEPs, flicker frequency may be controlled (e.g., between a range of 3 Hz to 100 Hz) and flicker type may be controlled (e.g., square-wave, sinusoidal, interpolated sinusoidal, filter-shape). Tagging may involve aperiodic methods. Such aperiodic methods may involve a random sequence or a design similar to those employed by MFVEP protocols in which a tag employs a switch between filters at regular intervals. At each interval, the filter may switch or remain the same. Neural responses may be assessed, e.g., in terms of the degree of difference in the signal when the filter was switched versus when the filter was not switched, to analyze the visual feature of interest. In embodiments, a number of repeats per tagged video is controlled. In embodiments, a number of tagged videos is controlled.

[0138] As still another example, the stimulus set included in a tagged video (or in a plurality of tagged videos) may be delivered via a computer monitor, via a virtual reality (VR) headset, via an experimental binocular rivalry setup (in which each eye views a separate display through a combination of screens and mirrors), or via a purpose build full-view screen.

[0139] As still another example, video content of videos (that are to be tagged and presented to users) may be controlled, such that the tagged videos used are suitable for a particular test. In embodiments, natural scene videos may be chosen, such as those involving people, animals, objects, orthographic stimuli, andplaces in the real world with natural image statistics. In embodiments, patient relevant videos may be chosen. For example, for young children, cartoons with natural image statistics may be selected for tagging. As another example, a user’s own personal home videos (or videos of a favorite animal or sport) may be selected for tagging. In embodiments, generative video may be used, which may be generated to probe specific visual dynamics.

[0140] As still another example, tags encoded in a video signal may be recovered with any suitable neuroimaging technology with the temporal resolution to resolve the signal. Though EEG and MEG readings have been described thus far, in embodiments, SEEG electrodes or ECOG arrays may be used (e.g., for epilepsy patients who are to view the tagged videos). In embodiments, functional nearinfrared spectroscopy (fNIRS) may be used to measure tags. In embodiments, ultrafast ultrasound, which measures cerebral blood volume with a high degree of spatial-temporal resolution, may be used.

[0141] FIG. 10 is a simplified functional block diagram of a computer system 1000 that may be configured as a computing device for executing the processes described herein, according to exemplary embodiments of the present disclosure. In various embodiments, any of the systems herein may be an assembly of hardware including, for example, a data communication interface 1020 for packet data communication. The platform also may include a central processing unit (“CPU”) 1002, in the form of one or more processors, for executing program instructions. The platform may include an internal communication bus 1008, and a storage unit 1006 (such as ROM, HDD, SDD, etc.) that may store data on a computer readable medium 1022, although the system 1000 may receive programming and data via network communications via electronic network 1025, which may correspond tonetwork 40 (e.g., voice, video, audio, images, or any other data over the electronic network 1025). The system 1000 may also have a memory 1004 (such as RAM) storing instructions 1024 for executing techniques presented herein, although the instructions 1024 may be stored temporarily or permanently within other modules of system 1000 (e.g., processor 1002 and / or computer readable medium 1022). The system 1000 also may include input and output ports 1012 and / or a display 1010 to connect with input and output devices such as keyboards, mice, touchscreens, monitors, displays, etc. The various system functions may be implemented in a distributed fashion on a number of similar platforms, to distribute the processing load. Alternatively, the systems may be implemented by appropriate programming of one computer hardware platform.

[0142] In this disclosure, the term “based on” means “based at least in part on.” The singular forms “a,” “an,” and “the” include plural referents unless the context dictates otherwise. The term “exemplary” is used in the sense of “example” rather than “ideal.” The terms “comprises,” “comprising,” “includes,” “including,” or other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, or product that comprises a list of elements does not necessarily include only those elements, but may include other elements not expressly listed or inherent to such a process, method, article, or apparatus. Relative terms, such as “about,” “approximately,” “substantially,” and “generally,” are used to indicate a possible variation of ±10% of a stated or understood value. In addition, the term “between” used in describing ranges of values is intended to include the minimum and maximum values described herein. The use of the term “or” in the claims and specification is used to mean “and / or” unless explicitly indicated to refer to alternatives only if the alternatives are mutually exclusive, although the disclosuresupports a definition that refers to only alternatives and “and / or.” As used herein “another” may mean at least a second or more.

[0143] As used herein, the term “user” generally encompasses any person or entity, such as a researcher and / or a care provider (e.g., a doctor, etc.), that may desire information, resolution of an issue, or engage in any other type of interaction with a provider of the systems and methods described herein (e.g., via an application interface resident on their electronic device, etc.). The term “electronic application” or “application” may be used interchangeably with other terms like “program,” or the like, and generally encompasses software that is configured to interact with, modify, override, supplement, or operate in conjunction with other software.

[0144] Program aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of executable code and / or associated data that is carried on or embodied in a type of machine-readable medium. “Storage” type media include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer of the mobile communication network into the computer platform of a server and / or from a server to the mobile device. Thus, another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elementsthat carry such waves, such as wired or wireless links, optical links, or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine “readable medium” refer to any medium that participates in providing instructions to a processor for execution.

[0145] Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention, and form different embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0146] Thus, while certain embodiments have been described, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such changes and modifications as falling within the scope of the invention. For example, functionality may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added or deleted to methods described within the scope of the present invention.

[0147] The above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other implementations, which fall within the true spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the following claims and their equivalents, and shall not be restricted or limited by the foregoing detailed description. While variousimplementations of the disclosure have been described, it will be apparent to those of ordinary skill in the art that many more implementations are possible within the scope of the disclosure. Accordingly, the disclosure is not to be restricted except in light of the attached claims and their equivalents.

Claims

WHAT IS CLAIMED IS:1 . A computer-implemented method of preparing video material that is configured to assess visual function of a user, the computer-implemented method comprising: determining a video having a resolution above a predetermined threshold, the video comprising a plurality of images, each image varying in frequency and orientation over space; determining a wavelet transform to facilitate a separation of spatial frequencies into specified spatial frequency bands based on a predetermined measure of a visual angle of a screen used to view the video from a predetermined viewing distance; determining a method of applying a frequency tag across visual features and temporal frequencies; generating a power filter to determine how the frequency tag is to be applied to the images; applying, for each of the images, the wavelet transform to a value of a color space channel for each of the images; and extracting, for each of the images, a power for each wavelet scale, wherein the power represents a contrast of a tagged feature for each of the images, and wherein the power verifies embedding of the tagged feature across time.

2. The computer-implemented method of claim 1 , wherein the wavelet transform is a two-dimensional (2D) steerable wavelet transform.

3. The computer-implemented method of claim 1 , wherein, during the determining of the wavelet transform, the method further involves generating an artificial grating image to verify spatial frequencies that are altered in each of a plurality of wavelet bands.

4. The computer-implemented method of claim 1 , wherein the video material is suitable for evaluating visual function in age-related macular degeneration (AMD) patients via neurophysiological response.

5. The computer-implemented method of claim 4, wherein the video material includes a stimulus set that is configured to evoke a neurophysiological response in a user.

6. The computer-implemented method of claim 5, wherein the stimulus set is detectable via electroencephalography (EEG) or magnetoencephalography (MEG).

7. The computer-implemented method of claim 1 , further comprising: zero-padding individual frames within the video to create square images for computational efficiency in applying the wavelet transform.

8. The computer-implemented method of claim 1 , wherein applying the wavelet transform comprises: converting the plurality of images from RGB color space to HSV color space; andconverting tagged images back to the RGB color space after applying the wavelet transform.

9. The computer-implemented method of claim 8, wherein prior to converting the images back to the RGB color space, the extracting of the power of each wavelet scale is performed for a combination of sub-bands of the specified spatial frequency bands.

10. A computer-implemented method of assessing vision function of a user, the method comprising: extracting data from an electrode site of a sensor; obtaining an average of the data across successive video presentations; applying a fast-Fourier transform to the average; determining a signal-to-noise ratio (SNR) of results of the fast-Fourier transform at tagged frequencies; determining a steady-state visual evoked potential (SSVEP) amplitude based on the SNR of the results of the fast-Fourier transform at the tagged frequencies; and determining a ratio of lower to higher spatial frequency SSVEPs to assess the vision function of the user.11 . The computer-implemented method of claim 10, wherein the data is obtained by presenting the user with a video set tagged with a stimulus set.

12. The computer-implemented method of claim 11 , wherein the stimulus set includes a tag that is tailored to a specified frequency or to a feature sensitivity of the user.

13. The computer-implemented method of claim 12, wherein the sensor utilizes electroencephalography (EEG) or magnetoencephalography (MEG) to obtain the stimulus set in real time.

14. The computer-implemented method of claim 11 , wherein the stimulus set includes a plurality of tags.

15. A system for assessing visual function of a user, the system comprising: one or more processors; and one or more computer readable media storing instructions that are executable by the one or more processors to perform operations comprising: determining a video having a resolution above a predetermined threshold, the video comprising a plurality of images, each image varying in frequency and orientation over space; determining a wavelet transform to facilitate a separation of spatial frequencies into spatial frequency sub-bands based on a predetermined measure of a visual angle taken up by a viewing screen at a specified viewing distance; determining a method of applying a frequency tag;generating a power filter to determine how the frequency tag is to be applied to the images; applying, for each of the images, the wavelet transform to a value of a color space channel for each of the images; and extracting, for each of the images, a power for each wavelet scale, wherein the power represents a higher spatial frequency in the spatial frequency sub-bands, to determine at least one tagged video.

16. The system of claim 15, wherein the wavelet transform is a two- dimensional (2D) steerable wavelet transform.

17. The system of claim 15, wherein, during the determining of the wavelet transform, the method further involves generating an artificial grating image to verify spatial frequencies that are tagged.

18. The system of claim 15, wherein the tagged video comprises video material suitable for evaluating visual function in age-related macular degeneration (AMD) patients via neurophysiological response.

19. The system of claim 18, wherein the video material includes a stimulus set that is configured to evoke a neurophysiological response in a user in real time.

20. The system of claim 19, wherein the stimulus set is detectable via electroencephalography (EEG) or magnetoencephalography (MEG).

Images