Method for assessing cognitive performance under environmental light stimulation based on multi-modal physiological monitoring

Abstract

The application provides an environmental light stimulation cognitive performance evaluation method based on multi-modal physiological monitoring, and belongs to the field of visual perception analysis. The evaluation method comprises the following steps: collecting electroencephalogram signals, cardiovascular signals and eye movement / pupil signals of a target object; in response to a triggered light stimulation event, collecting environmental light to obtain a measured light output waveform; determining a light-specific reflection window and a stable cognitive window from the measured light output waveform; based on the light-specific reflection window and the stable cognitive window, extracting light-specific response features and cognitive-related features from the electroencephalogram signals, cardiovascular signals and eye movement / pupil signals of the target object; and based on the light parameters of the environmental light, the summary features of the measured light output waveform, the light-specific response features and the cognitive-related features, performing conditional fusion to output a quantitative score, a change index or a classification level of the target object in at least one cognitive performance dimension, and obtaining an evaluation result of the cognitive performance of the target object to the environmental light stimulation.

Description

Technical Field

[0001] This invention relates to the field of visual perception technology, and more specifically, to a method for evaluating cognitive performance in response to ambient light stimuli based on multimodal physiological monitoring. Background Technology

[0002] Ambient light is one of the important environmental factors affecting human visual perception, work performance, and cognitive function. Related studies have shown that parameters such as illuminance, color temperature, spectral distribution, flicker characteristics, and dynamic changes in the light field can significantly affect an individual's alertness, visual attention, short-term memory, and task performance efficiency, and can induce physiological changes through visual pathways, circadian rhythm regulation, neuroendocrine responses, and central nervous system activation.

[0003] Therefore, how to objectively assess the changing patterns of individual cognitive performance under different ambient light conditions has become an important research topic in the fields of human factors engineering, healthy lighting, and intelligent environmental control. Summary of the Invention

[0004] In view of this, the present invention provides a method for evaluating cognitive performance under environmental light stimulation based on multimodal physiological monitoring.

[0005] One aspect of the present invention provides a method for evaluating cognitive performance in response to ambient light stimulation based on multimodal physiological monitoring, comprising: controlling the output of ambient light from a light source under a unified time reference, and acquiring electroencephalogram (EEG), cardiovascular signals, and eye movement / pupil signals of a target subject; in response to a triggered light stimulation event, acquiring the ambient light to obtain a measured light output waveform, wherein the triggering of the light stimulation event indicates a change in the light parameters of the ambient light; extracting event feature points corresponding to the light stimulation event from the measured light output waveform to obtain the measured light event time; performing optical link phase correction on the measured light event time using the light stimulation trigger time to obtain a corrected light stimulation event time; and performing optical link phase correction based on the above correction. The specific light reflection window is determined at the moment of the subsequent light stimulus event, and the stable cognitive window is determined based on the stable start time of the measured light output waveform that satisfies the stability criterion. Based on the specific light reflection window and the stable cognitive window, specific light response features and cognitive-related features are extracted from the target object's EEG signals, cardiovascular signals, and eye movement / pupil signals, respectively. Based on the ambient light parameters, the summary features of the measured light output waveform, the specific light response features, and the cognitive-related features, conditional fusion is performed to output the target object's quantitative score, change index, or classification level in at least one cognitive performance dimension, thereby obtaining the target object's cognitive performance evaluation result for ambient light stimulation.

[0006] According to an embodiment of the present invention, the determination of the stable cognitive window based on the stable start time of the measured light output waveform satisfying the stability criterion includes: taking the corrected light stimulation event time as the starting point, performing a signal stability scan based on a time window of a preset length to determine the start time of the measured light output waveform entering a stable state, thereby obtaining the stable start time; and determining the stable cognitive window based on the stable start time.

[0007] According to an embodiment of the present invention, the extraction of light-specific response features and cognitive-related features from the electroencephalogram (EEG), cardiovascular signals, and eye movement / pupil signals of the target object based on the aforementioned light-specific reflection window and the aforementioned stable cognitive window includes: extracting features based on a first EEG signal segment corresponding to the aforementioned light-specific reflection window in the aforementioned EEG signal and a first eye movement / pupil signal segment corresponding to the aforementioned light-specific reflection window in the aforementioned eye movement / pupil signals to obtain the aforementioned light-specific response features; and extracting features based on a second EEG signal segment corresponding to the aforementioned stable cognitive window in the aforementioned EEG signal, a cardiovascular signal segment corresponding to the aforementioned stable cognitive window in the aforementioned cardiovascular signal, and a second eye movement / pupil signal segment corresponding to the aforementioned stable cognitive window in the aforementioned eye movement / pupil signals to obtain the aforementioned cognitive-related features.

[0008] According to an embodiment of the present invention, the above-mentioned light-specific response features are obtained by extracting features from the first EEG signal segment corresponding to the light-specific reflection window in the above-mentioned EEG signal and the first eye movement / pupil signal segment corresponding to the light-specific reflection window in the above-mentioned eye movement / pupil signal, including: extracting the pupillary constriction latency, pupillary constriction amplitude, pupillary constriction speed, pupillary constriction recovery time and pupillary change index of the target object from the first eye movement / pupil signal segment to obtain pupil light reflection sub-features; performing time-domain feature extraction and frequency-domain feature extraction on the first EEG signal segment to obtain visual evoked response sub-features; and obtaining the above-mentioned light-specific response features based on the pupil light reflection sub-features and the visual evoked response sub-features.

[0009] According to an embodiment of the present invention, the above-described extraction of time-domain features and frequency-domain features from the first EEG signal segment to obtain visual evoked response sub-features includes: extracting the amplitude and latency of event-related potentials from the time-domain signal of the first EEG signal segment to obtain time-domain response feature parameters; extracting signal segments within the effective analysis bandwidth from the frequency-domain signal of the first EEG signal segment to obtain frequency-domain signal segments; performing feature extraction based on the frequency-domain signal segments to obtain frequency-domain response feature parameters related to light stimulation; and obtaining the visual evoked response sub-features based on the time-domain response feature parameters and the frequency-domain response feature parameters.

[0010] According to an embodiment of the present invention, the above-mentioned feature extraction based on the second EEG signal segment corresponding to the stable cognitive window in the above-mentioned EEG signal, the cardiovascular signal segment corresponding to the stable cognitive window in the above-mentioned cardiovascular signal, and the second eye movement / pupil signal segment corresponding to the stable cognitive window in the above-mentioned eye movement / pupil signal to obtain the above-mentioned cognitive-related features includes: determining the power and ratio characteristics of the second EEG signal segment based on the absolute power and relative power of each of the multiple feature bands included in the second EEG signal segment, and obtaining EEG signal sub-features; performing time-domain analysis based on the second cardiovascular signal segment to determine the heart rate and heart rate variability of the target object, and obtaining cardiovascular signal sub-features; parsing the second eye movement / pupil signal segment to determine the eye movement behavior characteristics and pupil dilation statistical characteristics of the target object, and obtaining eye movement / pupil signal sub-features; and obtaining the above-mentioned cognitive-related features based on the above-mentioned EEG signal sub-features, the above-mentioned cardiovascular signal sub-features, and the above-mentioned eye movement / pupil signal sub-features.

[0011] According to an embodiment of the present invention, the above-mentioned conditional fusion of the above-mentioned ambient light light parameters, the above-mentioned measured light output waveform summary features, the above-mentioned light-specific response features, and the above-mentioned cognitive-related features, outputting the quantitative score, change index, or classification level of the target object in at least one cognitive performance dimension, and obtaining the cognitive performance evaluation result of the target object to ambient light stimulation, includes: aggregating the above-mentioned light-specific response features using a light-specific feature branch network to form a branch representation characterizing the visual or neural transient response directly caused by changes in light parameters, obtaining a first branch feature; aggregating the above-mentioned cognitive-related features using a cognitive feature branch network to form a branch representation characterizing continuous cognitive state changes, obtaining a second branch feature; based on the above-mentioned ambient light light parameters and / or the above-mentioned measured light output waveform summary features, the above-mentioned first branch feature and the above-mentioned second branch feature are weighted and fused using a conditional gating weighting method through a fusion processing network to obtain a fused feature; and inputting the above-mentioned fused feature into a dimension-level multi-head output network to output the quantitative score, change index, or classification level of the target object in at least one cognitive performance dimension, obtaining the above-mentioned cognitive performance evaluation result.

[0012] According to an embodiment of the present invention, the above-mentioned summary features based on the optical parameters of the ambient light and / or the measured optical output waveform are weighted and fused using a conditional gating weighting method through a fusion processing network to obtain fused features. This includes: determining weighting coefficients based on the summary features of the optical parameters of the ambient light and / or the measured optical output waveform, and the first branch statistics related to the first branch features and the second branch statistics related to the second branch features; and weighting and fusion using a conditional gating weighting method based on the weighting coefficients to obtain the fused features.

[0013] According to an embodiment of the present invention, the method further includes: obtaining auxiliary input features based on the light parameters of the ambient light and / or the summary features of the measured light output waveform; and concatenating the auxiliary input features with the fusion features to obtain new fusion features; wherein, the step of inputting the fusion features into the dimensional multi-head output network to output the quantitative scores, change indices and classification levels of multiple cognitive performance dimensions to obtain the cognitive performance evaluation results includes: inputting the new fusion features into the dimensional multi-head output network to obtain the cognitive performance evaluation results.

[0014] Another aspect of the present invention provides an environmental light stimulation cognitive performance evaluation system based on multimodal physiological monitoring, comprising: a light source; a light source controller configured to control the light source to output ambient light at a preset time reference; a measured light detection module configured to detect the ambient light and obtain a measured light output waveform; an electroencephalogram (EEG) acquisition module configured to acquire EEG signals of a target object at the preset time reference; an electrocardiogram (ECG) acquisition module configured to acquire cardiovascular signals of the target object at the preset time reference; an image acquisition module configured to acquire eye movement / pupil signals of the target object at the preset time reference; and a processing module configured to process the measured light output waveform, the EEG signals, the cardiovascular signals, and the eye movement / pupil signals using the methods described above to obtain the cognitive performance evaluation result of the target object in response to environmental light stimulation.

[0015] According to an embodiment of the present invention, by obtaining a measured light output waveform in response to a triggered light stimulus event, phase correction can be performed on the light stimulus triggering time based on the measured light output waveform under the actual eye conditions of the target object. Based on the characteristic moments in the measured light output waveform, a light-specific reflection window and a stable cognitive window are determined. Furthermore, based on the light-specific reflection window and the stable cognitive window, light-specific response features and cognitive-related features are obtained. Thus, a dual-window decoupling and segmentation mechanism using the light-specific reflection window and the stable cognitive window can be employed to improve stimulus-response alignment accuracy and feature stability. Finally, the light-specific response features and cognitive-related features are input into the evaluation... The estimation model obtains the cognitive performance evaluation results of the target object in response to ambient light stimuli. By constructing a complete technical link for the generation and regulation of ambient light parameters, simultaneous acquisition of multimodal visual / physiological information, alignment and segmentation of light stimulus process, analysis of light-specific physiological characteristics, and mapping and evaluation of cognitive performance, it overcomes the problems of strong subjectivity, insufficient dynamic process characterization, limited ability to interpret single physiological modalities, and insufficient utilization of light-specific responses in existing light environment evaluation technologies. It achieves an objective, continuous, and quantitative evaluation of individual alertness level, attention state, and short-term memory performance under different illuminance, color temperature, spectral composition, and dynamic light change conditions. Attached Figure Description

[0016] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings.

[0017] Figure 1 A flowchart of an environmental light stimulus cognitive performance assessment method based on multimodal physiological monitoring according to an embodiment of the present invention is shown.

[0018] Figure 2 A schematic diagram illustrating the phase correction principle of an environmental light stimulus cognitive performance assessment method based on multimodal physiological monitoring according to an embodiment of the present invention is shown.

[0019] Figure 3 A schematic diagram of the architecture of an environmental light stimulus cognitive performance assessment method based on multimodal physiological monitoring according to an embodiment of the present invention is shown.

[0020] Figure 4 A schematic diagram of the multimodal physiological data processing flow of the environmental light stimulation cognitive performance assessment method based on multimodal physiological monitoring according to an embodiment of the present invention is shown.

[0021] Figure 5 A block diagram of an environmental light stimulus cognitive performance assessment system based on multimodal physiological monitoring according to an embodiment of the present invention is shown. Detailed Implementation

[0022] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms "comprising," "including," etc., as used herein indicate the presence of the above-described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0026] In the embodiments of this invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of data (e.g., including but not limited to user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0027] In the embodiments of the present invention, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0028] Currently, the assessment of the effects of light environment still has several shortcomings. First, existing methods rely heavily on questionnaires, subjective ratings, or interviews, which are easily influenced by factors such as individual preferences, emotional states, and recall biases of the target subjects, resulting in insufficient objectivity and repeatability, and making it difficult to form a stable quantitative evaluation basis. Second, existing assessment methods usually focus on comparing overall effects over a longer time scale, while paying insufficient attention to short-term physiological responses and cognitive fluctuations during changes in light stimulation. In fact, ambient light often affects the human body in the form of scene switching, gradual changes in brightness, color temperature adjustment, or sudden changes in screen brightness. Its effects are often clearly dynamic and phased. Without precise temporal alignment and continuous monitoring, it is difficult to accurately reveal the correspondence between changes in different light parameters and changes in cognitive performance.

[0029] Furthermore, while existing technologies have begun to incorporate physiological signals to assess the impact of ambient light, most analyses rely on a single modality, such as using only EEG, heart rate, or eye movement data. These methods struggle to fully reflect the multi-layered physiological and cognitive changes under ambient light. For example, while heart rate reflects overall arousal levels, it doesn't directly characterize attentional states or short-term memory performance; EEG signals reflect cognitive processing, but are susceptible to artifacts and individual differences when used alone; and while eye movement and pupillary data can reflect visual attention allocation and cognitive load changes, under ambient light stimulation, pupillary light reflexes and task-related pupillary changes are superimposed, limiting their explanatory power when analyzed alone. Moreover, ambient light possesses unique physiological response channels distinct from other environmental stimuli, such as pupillary light reflexes and event-related potentials (ERPs), but current methods haven't fully integrated these light-specific physiological indicators with EEG and heart rate signals for joint analysis, thus failing to reveal the overall mechanism by which ambient light parameters affect cognitive performance.

[0030] Therefore, we propose a technical solution for assessing cognitive performance under ambient light stimulation conditions. This solution can simultaneously collect multimodal physiological information within a unified time frame and, in conjunction with the changes in light stimulation parameters, objectively, continuously, and quantitatively assess an individual's cognitive performance, such as alertness, attention, and short-term memory. This will provide technical support for healthy lighting design, intelligent light environment control, and related human factors research.

[0031] Figure 1 A flowchart of a method for assessing cognitive performance in response to ambient light stimulation based on multimodal physiological monitoring, according to an embodiment of the present invention, is shown.

[0032] like Figure 1 As shown, the environmental light stimulation cognitive performance assessment method based on multimodal physiological monitoring according to an embodiment of the present invention includes operations S110~S170.

[0033] In operation S110, under a unified time reference, the ambient light output of the light source is controlled to collect the target object's electroencephalogram (EEG) signals, cardiovascular signals, and eye movement / pupil signals.

[0034] In operation S120, in response to the triggering of a light stimulation event, the ambient light output by the light source is collected to obtain the measured light output waveform. The triggering of the light stimulation event indicates that the light parameters of the ambient light have changed.

[0035] In embodiments of the present invention, ambient light can be set by a light source controller according to experimental design or application requirements. The light source controller uses preset combinations of ambient light parameters to place the target object under corresponding ambient light conditions. Simultaneously, it can record the start time, end time, change pattern, and corresponding configuration data of various ambient light parameters. Ambient light parameters include at least one or more of the following: vertical illuminance at the eye, average brightness of the field of view, correlated color temperature, spectral composition, proportion of short-wavelength components, modulation depth, or scene change curve. The recorded information is marked using a unified timestamp or hardware triggering method.

[0036] In embodiments of the present invention, multimodal physiological data of the target object can be collected. This multimodal physiological data includes, but is not limited to, electroencephalogram (EEG) signals, cardiovascular signals, eye movement / pupil signals, and pupil signals. EEG signals reflect cognitive processing activities and visual evoked responses; cardiovascular signals characterize arousal levels and autonomic nervous system regulation; and eye movement / pupil signals and pupil signals reflect visual attention allocation, pupillary light reflex, and changes in cognitive load. The aforementioned multimodal physiological data can be synchronized using a unified clock, a synchronous trigger interface, or a time alignment mechanism to ensure that it is within the same timeframe as the changes in ambient light parameters.

[0037] In embodiments of the present invention, multimodal physiological data can be preprocessed, including filtering and denoising, baseline correction, smoothing, artifact detection, and outlier correction. Specifically, EEG signals can be bandpass filtered, preferably with a filtering frequency band of 0.5–50 Hz, and power frequency suppression (50 Hz notch filtering) and artifact removal can be performed; cardiovascular signals can be filtered, smoothed, and waveform corrected; eye movement / pupil signals and pupil signals can be coordinate calibrated, drift corrected, blink interpolated, and outlier removed. The preprocessed multimodal physiological data retains effective information related to changes in ambient light and cognitive performance, providing a unified and reliable data foundation for subsequent evaluation and analysis.

[0038] Information about the photostimulation process of ambient light can be encapsulated and stored in structured data format to support subsequent alignment correction and effect analysis. Structured data may include: event ID (identification), event type, and control command timing. Light stimulation trigger time Measured time of light event Light stimulation event time Optical output link phase offset Ambient light optical parameter vector, summary characteristics of measured light output waveform, and stable start time. And stability criteria parameters, etc. Among them, event types can include: fixed, step, gradual, flicker, PWM (Pulse Width Modulation) and scene switching; ambient light parameters can include: illuminance, brightness, correlated color temperature, spectral composition, proportion of short-wavelength components, flicker frequency, modulation depth, PWM duty cycle, spatial distribution, incident direction, duration and change curve parameters, etc.; the summary characteristics of the measured light output waveform can include: peak and mean values, rise / fall time, maximum slope, half-point time (50% point), start time of the stable segment, modulation phase reference point, period / phase statistics, etc.

[0039] When there is a task presentation alignment requirement, structured data may include records of the master clock reference time. The actual time of task presentation and task link time offset To be used to map task events to... Under a consistent and unified timeline, structured data can also optionally record the event time calculation method identifier (threshold method / cross-correlation method / trigger difference method) and data quality / confidence identifier (e.g., signal-to-noise ratio, missing data marker) to support alignment reliability determination and outlier removal.

[0040] In operation S130, event feature points corresponding to the light stimulation event are extracted from the measured light output waveform to obtain the measured light event time.

[0041] In operation S140, the optical link phase is corrected by using the optical stimulus triggering time to obtain the corrected optical stimulus event time.

[0042] In embodiments of the present invention, to improve the authenticity and reproducibility of the recorded photostimulation process, the event moment can be defined and corrected in conjunction with the measured light output waveform. When the ambient light parameters undergo a step change, the event moment can preferably be defined as the moment when the measured light output waveform first crosses the target amplitude threshold. The target amplitude threshold can be a preset proportion of the target change (e.g., 10% or 20%), or optionally defined as the moment when the first derivative of the measured light output waveform reaches the maximum slope point (maximum rise / fall rate). When the ambient light parameters undergo a gradual change, the event moment can preferably be defined as the moment when the measured light output waveform reaches 50% of the cumulative change (halfway point), or optionally defined as the peak moment of the slope during the gradual change, to characterize the gradual change. Representative alignment points for the change process; when the ambient light parameters include flickering or PWM modulation, the event moment can preferably be defined as the phase reference point of the modulation waveform (e.g., zero crossing at the rising edge / duty cycle start edge) or the start point of the stable cycle after entering stable oscillation, and the modulation period, duty cycle and phase information can be recorded as event metadata; when the ambient light stimulus is a composite scene switching (multiple parameters change simultaneously or in stages), the event moment can be recorded for each change in the ambient light parameters, and a composite event record containing sub-event sequences can be formed to support subsequent stage alignment and analysis.

[0043] The specific calibration method for realizing photostimulation events based on measured light output waveform response and light source control triggering is as follows: Assume the light stimulation triggering time output by the light source controller (or synchronization clock and triggering unit) used to control the light source is... The measured light output event time corresponding to the event feature points (threshold crossing point / maximum slope point / half-range point / stable period start point, etc.) in the measured light output waveform is: Then the phase offset of the optical output link is defined as:

[0044] (1);

[0045] Based on this, the corrected light-stimulated event time is defined as:

[0046] (2);

[0047] Among them, under the same clock Equivalent to .

[0048] The above operations are used to correct the timing deviation of "command - light emission - exposure".

[0049] Furthermore, to ensure that task events are aligned with the main timeline, it is preferable to use the main clock reference time. Based on this, the time base of the task display system is synchronized and calibrated, and the time offset of the task display link is defined as:

[0050] (3);

[0051] in, The actual time when the task stimulus is presented on the display (which can be measured by the display system timestamp, vertical synchronization signal, or screen photoelectric sensor) is used to map the task event to... On a consistent, unified timeline.

[0052] To avoid the task interface itself becoming an additional source of light stimulation, the task interface preferably adopts a fixed background brightness and controlled color scheme, and locks the screen brightness / contrast parameters to keep the screen light output consistent across different light schemes.

[0053] In embodiments of the present invention, the phase difference between the control command time and the measured light output time can also be calculated and written as an event correction quantity into the photostimulation process information. The method for calculating the phase difference includes at least one of the following: (1) Threshold method: taking the control command issuance time as a reference, the time difference at which the measured light output waveform first crosses a preset threshold (e.g., a target change of 10% or 50%) is calculated as... (2) Cross-correlation method: Cross-correlation matching is performed between the target waveform corresponding to the control command and the measured optical output waveform, and the time shift corresponding to the cross-correlation peak is taken as the cross-correlation peak value. To adapt to situations with noise, delay and waveform distortion; (3) Trigger pulse-sensor response difference method: When the light source controller or synchronous clock outputs a hardware trigger pulse with the trigger unit, the time difference of the measured light output waveform response characteristic points (such as threshold crossing point, maximum slope point or stable period start point) is calculated with reference to the trigger pulse time. By recording the above-mentioned event timing definition rules, measured optical output waveform characteristic points, and... The calibration parameters enable precise control and complete recording of ambient light stimulation parameters, providing standardized input conditions for subsequent cognitive performance assessment.

[0054] In operation S150, the light-specific reflection window is determined based on the corrected light stimulation event time, and the stable cognitive window is determined based on the stable start time of the measured light output waveform that satisfies the stability criterion.

[0055] In the embodiments of this application, the stability criterion may be that the fluctuation of the measured light output waveform is lower than a preset threshold within 5 consecutive seconds, for example, the peak-to-peak value of the measured light output waveform is lower than the preset threshold within 5 consecutive seconds.

[0056] Light-specific reflection window This is used to capture direct light responses such as the pupillary light reflex (PLR) and the visual evoked potential (VEP). VEP is a weak electrophysiological signal generated by the visual cortex of the brain when it receives visual stimuli (such as flashes of light, checkerboard flips, graphics, or text). Furthermore, it can be based on light-specific reflection windows. This invention enables event-related potential (ERP) analysis, where ERP refers to the electroencephalographic response associated with a specific psychological event (such as seeing a picture, hearing a sound, or making a decision). In embodiments of this invention, the timing of the light stimulation event can be analyzed. For reference, The 0 to 2 seconds following this are defined as the light specific reflection window. Preferably, ERP analysis is based on The ERP time window is locked for the zero-point interception event. The ERP time window is -200 ms to +800 ms, which is used for subsequent ERP waveform construction and feature extraction.

[0057] Stable cognitive window This method is used to statistically analyze cognitive-related indicators such as electroencephalogram (EEG) band power, heart rate (HR), heart rate variability (HRV), eye movement behavior, and baseline-corrected pupillary dilation during the steady-state light phase, which are related to alertness, attention, and memory load. This allows for the analysis of physiological and behavioral changes related to alertness levels, attentional states, and short-term memory.

[0058] In operation S160, based on the optical specific reflection window and the stable cognitive window, optical specific response features and cognitive related features are extracted from the target object's EEG signals, cardiovascular signals and eye movement / pupil signals, respectively.

[0059] In embodiments of the present invention, feature parameters for cognitive performance assessment can be extracted from the aforementioned multimodal physiological data within a response window that includes a light-specific reflection window and a stable cognitive window. The feature parameters include light-specific response features under ambient light stimulation and cognitive-related features.

[0060] In operation S170, the light parameters of ambient light, the summary features of the measured light output waveform, the light specific response features and the cognitive related features are conditionally fused to output the quantitative score, change index or classification level of the target object in at least one cognitive performance dimension, thereby obtaining the cognitive performance evaluation result of the target object in response to ambient light stimulation.

[0061] In embodiments of the present invention, an evaluation model can be used to obtain the cognitive performance assessment results of a target object in response to ambient light stimuli. The evaluation model can output results for dimensions such as alertness, attention, and short-term memory, as well as a comprehensive cognitive performance index, during the inference phase. Optionally, the output features of the evaluation model can be sorted according to importance / contribution to aid in interpreting the evaluation results, and relevant information can be displayed or stored as needed. As an optional implementation, the evaluation model can also be replaced with models such as support vector machines, random forests, multimodal neural networks, or temporal neural networks. The evaluation results of the evaluation model include at least quantitative scores, change indices, relative baseline change magnitudes, or level determination results for dimensions such as alertness level, attention state, and short-term memory, and can further generate a comprehensive evaluation conclusion; optionally, it can output a ranking of key influencing factors and generate a visual report. The evaluation results can be displayed in real time, stored, or used for comparative analysis and optimization decisions between different ambient light schemes.

[0062] In embodiments of the present invention, the evaluation model can be trained based on supervised output variables. The supervised output variables are calculated from the behavioral performance data of the target object when performing a preset cognitive task, and the behavioral performance data includes at least task accuracy and average reaction time. The dispersion of mean reaction time (e.g., standard deviation of mean reaction time) or the coefficient of variation of the mean reaction time This includes statistics on task error types (such as false negative rate, false positive rate, or incorrect answer rate). For different cognitive dimensions, the supervision output variables can be constructed according to rules such as alertness level labels, attention state labels, short-term memory labels, or grading labels.

[0063] Alertness level labels are constructed based on reaction time and slow-event statistics, with the preferred method being the normalization of average reaction time. and define alertness level as Or normalize the retardation rate (such as the proportion of average reaction time exceeding a threshold) and then... Weighted combination formation .

[0064] Attention status labels are constructed based on attention stability and error control, with the mean reaction time variation coefficient being the preferred factor. Normalize the false response rate / false negative rate statistics separately, and define the attention state as follows: (For example, taking the stability term) (and weighted and merged with error items) to reflect the stability and interruption of sustained attention.

[0065] Short-term memory labels are constructed based on the accuracy, d′, or incorrect answer structure of short-term memory tasks and n-back tasks. Preferably, the labels are defined as the normalized representation of memory task performance. It is used to characterize the ability of short-term memory to be retained and updated.

[0066] The grading labels are segmented based on the variation of the continuous labels relative to the individual baseline or reference conditions, resulting in grading outputs such as "improved / unchanged / decreased" or "high / medium / low", which are used to train the classification output head or for displaying results.

[0067] Through embodiments of the present invention, by obtaining a measured light output waveform in response to a triggered light stimulus event, phase correction can be performed on the light stimulus triggering time based on the measured light output waveform under the actual eye conditions of the target object. Based on the characteristic moments in the measured light output waveform, a light-specific reflection window and a stable cognitive window are determined. Furthermore, based on the light-specific reflection window and the stable cognitive window, light-specific response features and cognitive-related features are obtained. Thus, a dual-window decoupling and segmentation mechanism using the light-specific reflection window and the stable cognitive window can be employed to improve stimulus-response alignment accuracy and feature stability. Finally, the light-specific response features and cognitive-related features are input into the evaluation... The estimation model obtains the cognitive performance evaluation results of the target object in response to ambient light stimuli. By constructing a complete technical link for the generation and regulation of ambient light parameters, simultaneous acquisition of multimodal visual / physiological information, alignment and segmentation of light stimulus process, analysis of light-specific physiological characteristics, and mapping and evaluation of cognitive performance, it overcomes the problems of strong subjectivity, insufficient dynamic process characterization, limited ability to interpret single physiological modalities, and insufficient utilization of light-specific responses in existing light environment evaluation technologies. It achieves an objective, continuous, and quantitative evaluation of individual alertness level, attention state, and short-term memory performance under different illuminance, color temperature, spectral composition, and dynamic light change conditions.

[0068] In a specific embodiment of the present invention, the stable cognitive window can be obtained by the following method: taking the corrected light stimulation event time as the starting point, performing a signal stability scan based on a time window of a preset length to determine the start time of the measured light output waveform entering a stable state, thereby obtaining the stable start time; and taking the stable start time as the starting point to determine the stable cognitive window.

[0069] Stable cognitive window This can be obtained based on the stable light phase, i.e. Not from the moment of the light stimulus event Instead of starting immediately, the process begins when the measured light output waveform reaches a stable state. A stable state is determined by the measured light output waveform's fluctuation falling below a preset threshold for five consecutive seconds; for example, the peak-to-peak value or standard deviation is below the threshold, or the deviation from the target value is below the threshold. The moment the measured light output waveform begins to reach a stable state is recorded, thus obtaining the stable start time. Stable start time The significance lies in guaranteeing The statistics focus on cognitive-related physiological states under "stable light exposure conditions," rather than fluctuations that occur during the light output transition process. Preferably, the data can be collected from the initial moment after the measured light output waveform has reached a stable state. To achieve a stable cognitive window, a 10-second sliding window was initially used for continuous updates. ,Right now It can be defined as A sliding window, wherein the step size of the sliding window can be selected as 1 second or 2 seconds. Optionally, it can be... Based on the sliding statistical results, summary statistics are performed according to the task structure (such as per question, per task segment, or per minute). The length of the above summary statistical window can be from several seconds to tens of seconds or longer, and is used to generate cognitive indicator comparison results at the task segment level.

[0070] Figure 2 A schematic diagram illustrating the phase correction principle of an environmental light stimulus cognitive performance assessment method based on multimodal physiological monitoring according to an embodiment of the present invention is shown.

[0071] like Figure 2 As shown, control command timing Used to characterize control command events, it is based on the software trigger time when the control program issues the command. The synchronous trigger pulse is used to characterize the hardware trigger time of the hardware module that applies the pulse in real time. In traditional schemes, it is generally based on the timing of the control command. Or the timing of light stimulation triggering As the moment of light stimulation However, in the embodiments of the present invention, the actual light output time of the measured light output waveform is first determined. With the triggering time of light stimulation Obtain the phase shift of the optical output link Then use light stimulation to trigger the timing. Phase shift of optical output link The values ​​are added together to obtain the corrected light stimulation time, where the measured light stimulation time is... The measured light detection time is used as the reference. After unifying the measured light detection time reference and the hardware trigger time reference to the same time reference, i.e., under the same clock, Equivalent to .

[0072] Through the aforementioned process of "triggering—measurement—phase correction—stability criterion—dual-window segmentation," a one-to-one correspondence can be established between visual / physiological responses at different time scales and the specific ambient light parameters. Subsequently, the corrected event markers and window segmentation information are synchronously written into multimodal physiological data streams such as EEG (Electrocardiogram), PPG (Photoplethysmography), eye movement, and pupillary data. This enables precise synchronization and segmented analysis of "light stimulation—task presentation—physiological response" within a unified time frame.

[0073] In a specific embodiment of the present invention, the optical specific response features can be obtained by the following method: extracting the pupillary constriction latency, pupillary constriction amplitude, pupillary constriction speed, pupillary constriction recovery time, and pupillary change index of the target object from the first eye movement / pupil signal segment to obtain pupillary light reflex sub-features; extracting the amplitude and latency of event-related potentials from the time-domain signal of the first EEG signal segment to obtain time-domain response feature parameters; extracting signal segments within the effective analysis bandwidth from the frequency-domain signal of the first EEG signal segment to obtain frequency-domain signal segments; performing feature extraction based on the frequency-domain signal segments to obtain frequency-domain response feature parameters related to optical stimulation; obtaining visual evoked response sub-features based on the time-domain response feature parameters and the frequency-domain response feature parameters; and obtaining the optical specific response features based on the pupillary light reflex sub-features and the visual evoked response sub-features.

[0074] In the special reflection window of light Within the context of ambient light stimulation, the specific light response characteristics include at least pupillary light reflex features and visually evoked response features. Pupillary light reflex features include pupillary constriction latency, pupillary constriction amplitude, pupillary constriction velocity, recovery time, and baseline-corrected pupillary change indices. Visually evoked response features include the amplitude and latency of event-related potentials (ERPs), as well as frequency domain response parameters related to light modulation. These frequency domain response parameters are preferably extracted within the effective EEG analysis bandwidth (0.5–50 Hz) to characterize changes in the electroencephalogram (EEG) spectrum induced by light stimulation. The light modulation frequency and waveform parameters can be provided by the measured light output detection unit and used as auxiliary input features. Furthermore, for ERP feature extraction, a short-time analysis window is preferably set within the event-locked time window, preferably 0–300 ms, to extract characteristic parameters such as peak amplitude, latency, and energy of early components.

[0075] In a specific embodiment of the present invention, cognitive-related features can be obtained by the following methods: determining the power and ratio characteristics of the second EEG signal segment based on the absolute and relative power of each of the multiple characteristic bands included in the second EEG signal segment, thereby obtaining EEG signal sub-features; performing time-domain analysis based on the second cardiovascular signal segment to determine the target object's heart rate and heart rate variability, thereby obtaining cardiovascular signal sub-features; analyzing the second eye movement / pupil signal segment to determine the target object's eye movement behavior characteristics and pupil dilation statistical characteristics, thereby obtaining eye movement / pupil signal sub-features; and obtaining cognitive-related features based on the EEG signal sub-features, cardiovascular signal sub-features, and eye movement / pupil signal sub-features.

[0076] In embodiments of the present invention, cognitive-related features are preferably based on a stable cognitive window. Extraction includes at least EEG band power and ratio features, heart rate and HRV, eye movement behavior features (including fixation distribution, gaze duration, scan rate and blink rate, etc.), and baseline-corrected pupil dilation statistics. Optional features can be combined with task performance characteristics to characterize persistent changes in alertness level, attention state, and short-term memory load. Furthermore, these features can be normalized and combined according to dimensions such as alertness level, attention state, and short-term memory to form a multidimensional feature vector for subsequent model input evaluation.

[0077] In a specific embodiment of the present invention, the cognitive performance evaluation result of the target object to ambient light stimulation can be obtained by the following method: Aggregating light-specific response features using a light-specific feature branch network to form a branch representation characterizing the visual or neural transient response directly triggered by changes in light parameters, resulting in a first branch feature; aggregating cognitive-related features using a cognitive feature branch network to form a branch representation characterizing continuous changes in cognitive state, resulting in a second branch feature; determining weighting coefficients based on the summary features of ambient light parameters and / or measured light output waveforms, and the first branch statistics related to the first branch feature and the second branch statistics related to the second branch feature; and weighting and fusing the first branch feature and the second branch feature using a conditional gating weighting method based on the weighting coefficients, resulting in a fused feature; and inputting the fused feature into a dimensional multi-head output network to output the target object's quantitative score, change index, or classification level in at least one cognitive performance dimension, resulting in a cognitive performance evaluation result.

[0078] The optical specific feature branch network receives optical specific reflections from the optical specific reflection window. The extracted PLR features, VEP / ERP features, and auxiliary features aligned with the measured light output waveform form the first branch of features characterizing the "visual / neural transient response directly triggered by changes in light parameters". .

[0079] Cognitive feature branch networks receive data from stable cognitive windows. The extracted EEG band power and ratio, heart rate and HRV, eye movement behavior features, and optional task performance features form a second branch of features characterizing "persistent changes in cognitive states such as alertness, attention, and memory load". .

[0080] In embodiments of the present invention, the first branch features and the second branch features can be subjected to standardization processing and individual baseline correction consistent with the training phase, respectively, and feature aggregation / summarization can be completed to obtain branch-level feature representations. Furthermore, the first branch features and the second branch features can be concatenated at the feature level through a fusion layer to form fused features. Optionally, to achieve adaptive adjustment of the contributions of the first and second branch features under different light stimulation types and individual conditions, a conditional gating weighting method can be used to weight and converge the first and second branch features. The weighting coefficients are jointly determined by the ambient light light parameter vector / waveform summary features and branch statistics, and can be learned during training by a subsequent Gradient Boosting Decision Tree (GBDT) or Light Gradient Boosting Machine (LightGBM) evaluator, thereby strengthening the conditional mapping of "light parameters - physiological response - cognitive performance".

[0081] In embodiments of the present invention, a dimensional multi-head structure can be used in the output layer to obtain a dimensional multi-head output network, which can output regression scores, change indices, or classification levels for dimensions such as alertness level, attention state, and short-term memory, while simultaneously outputting a comprehensive cognitive performance index (accuracy × (1- The output head of the multi-head output network can be implemented using an independent regression / classifier (preferably GBDT / LightGBM or an equivalent model) to ensure controllable error and output stability for different cognitive dimensions. That is, independent GBDT sub-models can be trained separately for different cognitive dimensions to construct output heads for alertness level, attention state, short-term memory, and comprehensive cognitive performance, respectively. As input, the corresponding dimension's regression score, change index, or classification level is output. The optimal definition for the comprehensive cognitive performance index is: accuracy × (1- ), where the proportion of accuracy rates in the 0-1 range, It is a dimensionless quantity in the interval 0 to 1 (which can be obtained by linear mapping and truncation to the interval 0 to 1).

[0082] In embodiments of the present invention, auxiliary input features can also be obtained, and these auxiliary input features can be concatenated with fusion features obtained based on first branch features and second branch features to obtain new fusion features, thereby using the new fusion features to obtain cognitive performance evaluation results. The specific method is as follows: auxiliary input features are obtained based on summary features of ambient light parameters and / or measured light output waveforms; and the auxiliary input features are concatenated with fusion features to obtain new fusion features; wherein, the fusion features are input into a dimension-level multi-head output network to output the quantitative scores, change indices, and classification levels of multiple cognitive performance dimensions to obtain cognitive performance evaluation results, including: inputting the new fusion features into the dimension-level multi-head output network to obtain cognitive performance evaluation results.

[0083] In embodiments of the present invention, auxiliary input features can be obtained based on the summarization features of ambient light parameters and / or measured light output waveforms. Furthermore, by optionally constructing cross-branch interaction terms (such as element-wise product, difference, ratio, or related statistics), the auxiliary input features are concatenated with the first and second branch features to perform feature fusion, resulting in new fused features that enhance the coupling expression between transient optical response information and continuous cognitive information.

[0084] Figure 3 A schematic diagram of the architecture of an environmental light stimulus cognitive performance assessment method based on multimodal physiological monitoring according to an embodiment of the present invention is shown.

[0085] like Figure 3 As shown, the environmental light stimulus cognitive performance evaluation method based on multimodal physiological monitoring includes five parts: input data, preprocessing, windowing and alignment, feature output and feature vector.

[0086] Input data includes various acquired multimodal physiological data, including EEG, ECG, eye movement and pupil-related data. Preprocessing involves filtering, denoising, baseline correction, and other data processing operations on the input data to obtain more effective input data. Windowing and alignment refer to determining the light-specific reflection window and stable cognitive window based on the characteristic moments in the measured light output waveform, using the light stimulus triggering time as a reference. Feature output refers to extracting abstract light stimulus / measured light features, light-specific physiological features, cognitively related multimodal features, and auxiliary features from the target object based on the light-specific reflection window and stable cognitive window, obtaining specific feature vectors. Feature vectors include light-specific response features, cognitively related features, auxiliary input features, and fusion features, and are used for cognitive performance mapping and evaluation.

[0087] Figure 4 A schematic diagram of the multimodal physiological data processing flow of the environmental light stimulation cognitive performance assessment method based on multimodal physiological monitoring according to an embodiment of the present invention is shown.

[0088] like Figure 4 As shown, the multimodal physiological data processing flow, in chronological order, includes the generation of ambient light parameters, joint signal acquisition, synchronization of light stimulation and response segmentation, analysis of light-specific physiological characteristics, and mapping and output of cognitive performance evaluation results. Simultaneously, the relevant data in these five processing flows interact with memory or databases, and the data results related to the mapping and output of cognitive performance evaluation results can also be input to the output interface / display.

[0089] In embodiments of the present invention, the environmental light stimulation cognitive performance evaluation method based on multimodal physiological monitoring can be implemented through the control flow and data flow described below.

[0090] The control flow can include three main lines: "light scheme issuance - light source execution - trigger synchronization": (1) The processor or ambient light parameter generation module issues the ambient light stimulation scheme (including illuminance / brightness, correlated color temperature, spectral / short-wavelength component ratio, flicker parameters, spatial distribution / incident direction, duration and change curve, etc.) to the light source controller; (2) The light source controller drives the dimmable and color temperature adjustable LED lamps, programmable light strips, display light sources or multi-light source arrays to execute the corresponding output; (3) At the start, end or parameter change of the light stimulation, the light source controller or synchronization clock and trigger unit output hardware trigger pulse / synchronization signal, and send the trigger signal to the multi-modal acquisition device and processor for unified alignment and timing correction.

[0091] The data stream can include five categories: "optical parameter data stream - optical measurement data stream - physiological data stream - task data stream - evaluation result stream". (1) Optical parameter data stream: The optical parameter recording unit records the ambient light parameter configuration data, the start and end times of the change, the change mode and process curve, and writes it into the memory as the metadata of the light stimulation event; (2) Optical measurement data stream: The actual light output waveform (which may include the illuminance change curve over time, flicker waveform characteristics, etc.) is collected by the actual light detection module (preferably an illuminance / brightness / spectral sensor deployed close to the line of sight of the target object), and the measured waveform is transmitted to the processor as an alignment anchor point to correct the time phase difference between "instruction time - actual light output time"; (3) Physiological data stream: The EEG acquisition module outputs the raw EEG data stream, and the ECG acquisition module outputs ECG data stream. The G / PPG raw data stream, the image acquisition module outputs the gaze position, gaze / saccade events and pupil diameter data stream, all data streams are attached with a unified timestamp or receive a trigger synchronization signal and are transmitted to the processor and memory; (4) Task data stream: the task presentation terminal outputs the task stimulus presentation timestamp, key response timestamp, accuracy and reaction time and other behavioral data, and synchronizes them with the light stimulus events and physiological data on a unified time axis; (5) Evaluation result stream: after the processor preprocesses the above data, performs event alignment, window segmentation, feature extraction and fusion evaluation, it outputs the evaluation results such as alertness level, attention state, short-term memory performance and comprehensive cognitive performance, and writes the score, change index and level judgment into the memory or sends them to the display / external interface; optionally, it outputs key influencing factor information for interpretation.

[0092] Figure 5 A schematic diagram of an environmental light stimulation cognitive performance assessment system based on multimodal physiological monitoring according to an embodiment of the present invention is shown.

[0093] like Figure 5 As shown, the environmental light stimulation cognitive performance assessment system based on multimodal physiological monitoring includes a light source 510, a light source controller 520, a measured light detection module 530, an electroencephalogram (EEG) acquisition module 540, an electrocardiogram (ECG) acquisition module 550, an image acquisition module 560, and a processing module 570.

[0094] The light source controller 520 is configured to control the output of ambient light from the light source 510 under a preset time reference. The light source controller 520 can generate and regulate combinations of light parameters of the ambient light according to a preset scheme, and simultaneously record the start and end times, change methods, and corresponding configuration data of various light parameter changes to form traceable light stimulation process information. The light source controller 520 may include three parts: a light source control unit, a light parameter configuration unit, and a process recording unit.

[0095] The light source 510 can be composed of dimmable and color-tunable LED (Light Emitting Diode) lamps, programmable light strips, display screen light sources or multi-light source arrays, and the parameters such as illuminance, brightness, correlated color temperature, spectral composition, short-wavelength component ratio, spatial distribution and incident direction can be adjusted through the light source control unit.

[0096] The light parameter configuration unit is used to set the ambient light stimulation scheme, which includes, but is not limited to, fixed light conditions, step changes, gradual changes, periodic changes, and composite scene switching. The relevant parameters include illuminance range, correlated color temperature range, flicker frequency, modulation depth, PWM duty cycle, scene duration, and change curve.

[0097] The process recording unit is used to record the changes in various optical parameters, the timing of these changes, and the corresponding configuration data, and encapsulates this information into photostimulation process information required for subsequent analysis. Furthermore, the process recording unit preferably encapsulates and stores the photostimulation process information in structured data format to support subsequent alignment correction and effect analysis.

[0098] The measured light detection module 530 is configured to detect ambient light and obtain the measured light output waveform.

[0099] The EEG acquisition module 540 is configured to acquire EEG signals from a target subject at a preset time reference to characterize cognitive brain processing activity under changing ambient light conditions and to support event-related potential analysis. The EEG acquisition module 540 and... Figure 5 The corresponding EEG electrodes / EEG cap.

[0100] The ECG acquisition module 550 is configured to acquire cardiovascular signals from a target subject at a preset time reference to extract indicators reflecting arousal level and autonomic nervous system regulation, such as heart rate and heart rate variability. The cardiovascular signals include ECG or PPG signals. The ECG acquisition module 550 and... Figure 5 The corresponding ECG / PPG sensor.

[0101] Image acquisition module 560 is configured to acquire eye movement / pupil signals of the target object at a preset time reference. This is used to obtain gaze position, fixation behavior, saccade trajectory, and pupil diameter changes, thereby characterizing changes in visual attention allocation, pupillary light reflex, and cognitive load. Image acquisition module 560 and... Figure 5 The corresponding eye movement / pupil acquisition unit in the middle.

[0102] The measured light detection module 530, EEG acquisition module 540, ECG acquisition module 550, and image acquisition module 560 can be implemented based on a visual / physiological joint acquisition module. This module is used to simultaneously acquire multimodal visual and physiological signals of the target object during changes in ambient light parameters, ensuring that all data and the light stimulation process are on a unified timeline. The visual / physiological joint acquisition module may also include a synchronization management unit. This unit is used to achieve unified time synchronization and cross-device drift correction during multimodal physiological data acquisition through a common clock, hardware synchronization interface, or network time synchronization. This module can continuously record visual and physiological information during changes in ambient light, providing a unified and complete data foundation for subsequent light stimulation response analysis.

[0103] The processing module 570 is configured to process the measured light output waveform, electroencephalogram (EEG) signal, cardiovascular signal and eye movement / pupil signal using the methods described above, and obtain the cognitive performance evaluation results of the target object in response to ambient light stimulation.

[0104] In embodiments of the present invention, the processing module 570 may include a light stimulation synchronization and response segmentation submodule, a light-specific physiological feature analysis submodule, and a cognitive performance mapping and evaluation output submodule. The light stimulation synchronization and response segmentation submodule is used to time-align the raw multimodal physiological data and to event-based label and segment the collected data according to the changes in ambient light parameters, forming structured response data fragments that can be used for subsequent analysis. The light-specific physiological feature analysis submodule is used to extract response features specific to ambient light stimuli from multimodal visual and physiological data and to construct an interpretable evaluation index system by combining cognitively relevant features. The cognitive performance mapping and evaluation output submodule is used to fuse and model multimodal features, establish a mapping relationship between changes in ambient light parameters and changes in cognitive performance, and output the individual's cognitive performance evaluation results under different light conditions.

[0105] The photostimulation synchronization and response segmentation submodule can be divided into three parts: a time alignment unit, a data cleaning unit, and a response segmentation unit. The time alignment unit maps ambient light parameter changes and multimodal physiological data onto the same time axis. The data cleaning unit performs filtering, smoothing, baseline correction, outlier identification, and artifact suppression on EEG, ECG / PPG, eye movement, and pupillary data. Specifically, EEG data can be processed using independent component analysis and adaptive filtering to remove electrooculography and motion artifacts, while pupillary data can undergo blink interpolation and outlier correction. The response segmentation unit is used to segment data based on the corrected photostimulation event timestamps. and the measured start time of stable light output The collected data is segmented into two windows, specifically into windows representing specific light reflections. and stable cognitive window The data is updated by sliding in 1-second or 2-second increments to form structured data segments corresponding to the light stimulation phase. Optionally, it can be... Based on the sliding statistical results, longer-term (e.g., minute-level) summary statistics are performed to analyze long-term trends such as rhythm / fatigue. The photostimulation synchronization and response segmentation submodule can transform continuously acquired visual / physiological data into response units suitable for analysis, thereby supporting the evaluation of photostimulation effects at different time scales.

[0106] The optical specific physiological feature analysis submodule can be divided into three parts: a pupillary light reflex feature extraction unit, a visual evoked response feature extraction unit, and a cognitive-related feature construction unit. The pupillary light reflex feature extraction unit extracts pupillary light reflex-related features, including pupillary constriction latency, maximum pupillary constriction amplitude, pupillary constriction speed, and recovery time. It preferably corrects these features against an individual baseline, generating relative baseline change indices such as difference, relative change rate, or Z-score to reduce the impact of individual pupillary differences and acquisition condition variations on feature stability. The visual evoked response feature extraction unit extracts event-related brain response features, including peak amplitude, latency, and waveform energy of specific waveform components. It preferably outputs standardized change in latency and amplitude / energy values ​​obtained based on baseline window statistics to enhance comparability across target objects and device conditions. The cognitive-related feature construction unit further extracts EEG band power and ratio features, heart rate and HRV features, eye movement behavior features, and auxiliary indicators related to task execution, and normalizes and integrates them around dimensions such as alertness level, attention state, and short-term memory. Specifically, EEG, heart rate / HRV, and eye-tracking related features are preferably converted into relative baseline change indicators and their dimensions are standardized to facilitate subsequent multimodal fusion modeling. The photo-specific physiological feature analysis submodule can form a multidimensional feature set that takes into account both photo-specific response and cognitive performance representation, providing interpretable input for subsequent model evaluation; at the same time, it can retain the original absolute values ​​and hierarchical labels of the features as auxiliary outputs for visualization and engineering threshold constraints.

[0107] The cognitive performance mapping and evaluation output submodule is preferably deployed in the processor of the ambient light stimulation cognitive performance evaluation system. It establishes data and control connections with the light source controller, the measured light detection module, the synchronization clock and trigger unit, the EEG acquisition module, the ECG acquisition module, or the image acquisition module. This is used to complete the alignment, segmentation, feature construction, and fusion evaluation output of "light stimulation—task presentation—multimodal physiological response" under a unified time reference. Specifically, the light source controller drives the light source output according to the stimulation scheme issued by the light parameter configuration unit; the synchronization clock and trigger unit outputs trigger markers at the start, end, or parameter change of light stimulation, providing a unified time reference for the light source controller, the actual light detection module, and each physiological acquisition device. Cross-device time synchronization and drift correction can be achieved through a common clock, hardware trigger interface, or network time synchronization; the measured light detection module acquires the measured light output waveform (which may include illuminance / brightness change curves and flicker / PWM modulation waveforms) at a position close to the target object's eye conditions, and uses it together with the control command time and trigger pulse timestamp to calculate the time phase difference between "control command—actual light output—individual exposure". This allows for the correction of the timing of the light stimulation event and the recording of the light stimulation process information. The above... The event time can be obtained using the threshold method, cross-correlation method, or trigger pulse-sensor response difference method. The event time can also be determined based on the threshold crossing point, maximum slope point, half-stroke point, or stable period start point of the measured waveform. The corrected event time is recorded as... After completing event correction, the cognitive performance mapping and evaluation output submodule synchronously writes the corrected light stimulation event labels into the multimodal physiological data stream, and performs windowing segmentation based on the corrected events to obtain the light-specific reflection window. and stable cognitive window Subsequently, the cognitive performance mapping and evaluation output submodule completes feature extraction and assembly within the corresponding window.

[0108] In embodiments of the present invention, the ambient light stimulus cognitive performance evaluation system may further include a storage module for storing ambient light light parameter configurations, event markers, raw / preprocessed multimodal physiological data, feature vectors, and evaluation results and visualization reports.

[0109] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the steps of the environmental light stimulation cognitive performance evaluation method based on multimodal physiological monitoring according to the embodiments of the present invention. These steps include at least setting and recording the ambient light parameters, joint acquisition of multimodal visual and physiological signals, preprocessing of multimodal physiological data, alignment and response segmentation of the light stimulation process, extraction of light-specific responses and cognitive-related features, and mapping and evaluation of cognitive performance and output of results, thereby completing an objective evaluation process of individual cognitive performance under different ambient light parameter conditions.

[0110] The following describes specific embodiments of the present invention in the context of an open-plan office setting, but the scope of protection of the present invention is not limited to these embodiments. This embodiment deploys an ambient light stimulus cognitive performance assessment system in a typical open-plan office environment. Using a task-driven model (the target subject performs a preset cognitive task), it assesses the changes in individual alertness levels, attentional states, and short-term memory performance under different lighting environments, and based on this, generates lighting optimization suggestions.

[0111] The first step is to set up the scene and configure the lighting scheme.

[0112] First, dimmable and color-temperature adjustable LED luminaires were installed in the open-plan office to construct a programmable ambient light control system. Based on the needs of the office environment, four ambient light parameter schemes were pre-set, including: Scheme A: a baseline office lighting environment with an illuminance of 500 lux and a correlated color temperature of 4000K; Scheme B: an alertness-enhancing lighting environment with an illuminance of 800 lux and a correlated color temperature of 6500K, and an increased proportion of short-wavelength components; Scheme C: a soothing and relaxing lighting environment with an illuminance of 300 lux and a correlated color temperature of 3000K; and Scheme D: a flicker-intervention lighting environment, where flicker intervention was set under an average illuminance of 500 lux, with a flicker frequency of 120 Hz, a modulation depth of 30%, and a PWM duty cycle of 50%, to evaluate the impact of flicker parameters on attentional stability and fatigue-related performance.

[0113] The ambient light parameter generation module switches between light schemes in two ways: a step switch lasting 2 seconds and a gradual switch lasting 30 seconds, to cover abrupt and gradual light stimuli in typical office scenarios. The system generates an event ID for each light scheme switch and records the event type, scheme number, parameter vector, and change curve. Simultaneously, the light source controller writes the control command timing. Output the trigger timestamp Illuminance sensors were deployed near the line of sight of the target object to collect measured light output waveforms. Extract the measured light event time Calculated based on this And obtain the time of the light stimulus event. The initial time of stabilization was recorded when the measured light output met the stabilization criterion for 5 consecutive seconds. The above information is written into the structured fields of the photostimulation process information, including at least: event ID, event type, and protocol number. , , , , , Ambient light parameter vector and measured light waveform summary characteristics; the illuminance sensor and data platform share a unified clock and complete time calibration.

[0114] The second step is to select target object samples and design experimental rounds.

[0115] Twenty-four office workers (12 men and 12 women), aged 22–35 years, with normal or corrected vision, were selected, excluding individuals with a history of neurological diseases or severe sleep disorders. A within-subjects design was used, with each participant experiencing both options A and D to minimize individual variability. Each participant completed two rounds of the experiment, 24 hours apart; each round included one exposure to each of the four light options. Each light option lasted 8 minutes, with the first 2 minutes as an adaptation period and the last 6 minutes as a task period; there was a 2-minute rest period between options. Eight valid light condition data points were obtained from each participant (4 options × 2 rounds), forming a structured dataset suitable for modeling and statistical analysis.

[0116] The third step is to conduct task-driven data acquisition and baseline establishment.

[0117] Multimodal data acquisition devices were fitted to the target subjects: an EEG acquisition cap, ECG electrode patches or PPG acquisition devices, and wearable eye-tracking and pupil-tracking devices. Sampling rates were set as follows: EEG 500 Hz, ECG / PPG 256 Hz, eye-tracking / pupil tracking 120 Hz, and illuminance sensor 200 Hz. All devices were connected to a unified data platform and used a unified time reference.

[0118] Resting state and simplified cognitive task data were collected under the baseline lighting environment of Scheme A to establish individual baselines (for subsequent Z-score / relative change correction). Subsequently, under different lighting schemes, the target subjects completed cognitive tasks via computer: a color visual short-term memory task and a 2-back task. The task system recorded stimulus presentation time, key response time, accuracy, and reaction time.

[0119] To avoid the task interface itself becoming an additional source of light stimulation, the monitor brightness is locked at 120 cd / m², the background grayscale is fixed (RGB=128,128,128), and the interface colors use a controlled color palette and remain consistent throughout.

[0120] The fourth step involves optical stimulus alignment, preprocessing, and response segmentation.

[0121] After the experiment, the collected data were preprocessed: EEG bandpass filtering was performed from 0.5 to 50 Hz and notch filtering was applied at 50 Hz to suppress artifacts; ECG / PPG smoothing and peak detection were performed and abnormal pulsations were removed; eye movement and pupil were subjected to coordinate calibration, drift correction, blink interpolation and outlier correction.

[0122] The light-stimulated event correction adopts the "trigger-measurement" phase difference correction as described above.

[0123] The window segmentation adopts a fixed rule: (1) Light-specific reflection window ERP event lock window (2) Stable cognitive window First determine the stable start time. The steady state is defined as the measured illuminance. Within any consecutive 5 seconds, the peak-to-peak fluctuation must not exceed 2% of the target illuminance. Then define It is updated in a step of 2 seconds.

[0124] The fifth step involves feature extraction, dataset construction, and obtaining comprehensive cognitive performance dependent variables.

[0125] exist Internal extraction of specific light response features: (1) PLR features: pupillary latency, pupillary amplitude, pupillary speed, recovery time, pupillary changes after baseline correction; (2) VEP / ERP features: peak amplitude, latency, waveform energy; early component features of ERP are extracted within a short window of 0 to 300 ms.

[0126] exist Internally extracted cognitive-related auxiliary features include: EEG band power and ratio, heart rate and HRV, fixation distribution, gaze duration, scan rate, blink rate, and baseline-corrected pupil dilation statistics. These features are then fused with task performance features (accuracy and reaction time). All features are converted into relative baseline change indices (difference, relative rate of change, or Z-score) and, after dimensional unification, form a multidimensional feature vector.

[0127] The dataset outputs single-dimensional scores / ranks for alertness level, attention state, and short-term memory performance, as well as overall cognitive performance, as dependent variables.

[0128] (4);

[0129] Where Acc represents the task accuracy (0 to 1). Normalization of the reaction time yields:

[0130] (5);

[0131] in, Take the 5th percentile of the baseline task reaction time distribution. Taking the 95th percentile, clip truncates the value to [0,1].

[0132] The sixth step is to perform fusion modeling to obtain evaluation results on model performance and statistical significance.

[0133] Fusion modeling adopts a structured process of "optical specific branch + cognitive branch + fusion layer + dimensional multi-head output": (1) The input of the optical specific branch is (2) Cognitive branch input is the summarization features of PLR / VEP / ERP and measured optical waveforms; (3) The fusion layer uses feature splicing and adds interaction terms to construct the fusion representation: .in For optical parameter vectors and waveform summarization features, For element-wise interaction items. Dimensional multi-head output is implemented using multiple models: alertness, attention, memory, and comprehensive cognitive performance are each trained with a separate GBDT / LightGBM regression / classification model, to achieve... The input and output are the results corresponding to the dimensions.

[0134] Training and validation employ leave-one-out cross-validation. The overall cognitive performance regression output is obtained as follows: MAE=0.05; overall accuracy of graded output is 0.84, and macro-mean F1 is 0.81. Statistical analysis using repeated measures ANOVA revealed a main effect of the scheme on overall cognitive performance. Pairwise comparison: Option B is an improvement over Option A. Option C is lower than Option A. Option D is lower than Option A. .

[0135] Step 7: Conduct a control experiment.

[0136] To verify the effectiveness of the key mechanism of this invention, two control experiments can be set up. Control 1: No experiment performed. Phase correction, directly using As the event moment; Comparison 2: Remove the light-specific branch (not using) PLR / VEP / ERP features), only using Feature modeling.

[0137] Comparison 1 Results: Regression of Overall Cognitive Performance MAE=0.06; graded output accuracy 0.78, F1 score 0.75. Control 2 results: Comprehensive cognitive performance regression. MAE=0.06; graded output accuracy 0.79, F1 is 0.76. This verifies that the measured optical output phase correction and dual-window decoupling segmentation jointly improve alignment accuracy and feature stability.

[0138] Step 8: Output results and lighting optimization suggestions.

[0139] The system outputs the target object's alertness, attention, short-term memory, and overall cognitive performance under schemes A through D, and provides lighting optimization suggestions: Scheme B should be used during periods of high-intensity focused tasks, and Scheme C should be used during periods of low-stress recovery; Scheme D is not recommended under this combination of flicker parameters. The system writes the evaluation results, parameter schemes, and suggestions into the database and generates a report for archiving.

[0140] This embodiment illustrates that the method and system described above can be used in actual open office environments to objectively, continuously, and multidimensionally evaluate changes in individual cognitive performance under different ambient light schemes in a task-driven mode. Furthermore, it can combine multimodal visual and physiological information to reveal the influence of changes in ambient light parameters on alertness level, attention state, and short-term memory performance, thus possessing good practicality and promotional value.

[0141] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0142] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A method for assessing cognitive performance in response to ambient light stimulation based on multimodal physiological monitoring, characterized in that, include: Under a unified time reference, the ambient light output of the light source is controlled to collect the target object's electroencephalogram (EEG) signals, cardiovascular signals, and eye movement / pupil signals. In response to a triggered light stimulation event, the ambient light is collected to obtain a measured light output waveform, wherein the triggering of the light stimulation event indicates a change in the light parameters of the ambient light; The event feature points corresponding to the light stimulation event are extracted from the measured light output waveform to obtain the measured light event time. The optical link phase is corrected by using the optical stimulus triggering time to obtain the corrected optical stimulus event time; The optical specific reflection window is determined based on the corrected optical stimulation event time, and the stable cognitive window is determined based on the stable start time of the measured optical output waveform that satisfies the stability criterion; based on the optical specific reflection window and the stable cognitive window, optical specific response features and cognitive related features are extracted from the target object's EEG signals, cardiovascular signals and eye movement / pupil signals, respectively. Based on the light parameters of the ambient light, the summary features of the measured light output waveform, the light specific response features, and the cognitive-related features, conditional fusion is performed to output the quantitative score, change index, or classification level of the target object in at least one cognitive performance dimension, thereby obtaining the cognitive performance evaluation result of the target object in response to ambient light stimuli.

2. The method according to claim 1, characterized in that, The determination of the stable cognitive window based on the stable start time of the measured optical output waveform satisfying the stability criterion includes: Starting from the corrected time of the light stimulation event, a signal stability scan is performed based on a time window of a preset length to determine the start time of the measured light output waveform reaching a stable state, thus obtaining the stable start time. The stable cognitive window is determined starting from the stable initial time.

3. The method according to claim 1, characterized in that, Based on the optical specific reflection window and the stable cognitive window, optical specific response features and cognitive-related features are extracted from the target object's electroencephalogram (EEG), cardiovascular signals, and eye movement / pupil signals, respectively, including: Based on the first EEG signal segment corresponding to the light-specific reflection window in the EEG signal and the first eye movement / pupil signal segment corresponding to the light-specific reflection window in the eye movement / pupil signal, feature extraction is performed to obtain the light-specific response feature; The cognitive-related features are obtained by extracting features from the second EEG signal segment corresponding to the stable cognitive window in the EEG signal, the cardiovascular signal segment corresponding to the stable cognitive window in the cardiovascular signal, and the second eye movement / pupil signal segment corresponding to the stable cognitive window in the eye movement / pupil signal.

4. The method according to claim 3, characterized in that, The light-specific response features are obtained by extracting features from the first EEG signal segment corresponding to the light-specific reflection window in the EEG signal and the first eye movement / pupil signal segment corresponding to the light-specific reflection window in the eye movement / pupil signal, including: Extract the pupillary constriction latency, pupillary constriction amplitude, pupillary constriction speed, pupillary constriction recovery time, and pupillary change index of the target object from the first eye movement / pupil signal segment to obtain pupillary light reflectance sub-features; Temporal and frequency domain features were extracted from the first EEG signal segment to obtain visual evoked response sub-features. The light-specific response features are obtained based on the pupil light reflector features and the visual evoked response features.

5. The method according to claim 4, characterized in that, The step of extracting time-domain features and frequency-domain features from the first EEG signal segment to obtain visually evoked response sub-features includes: The amplitude and latency of event-related potentials are extracted from the time-domain signal of the first EEG signal segment to obtain time-domain response characteristic parameters; Extract the signal segment within the effective analysis bandwidth from the frequency domain signal of the first EEG signal segment to obtain the frequency domain signal segment; Based on the frequency domain signal segment, feature extraction is performed to obtain frequency domain response feature parameters related to optical stimulation; Based on the time-domain response feature parameters and the frequency-domain response feature parameters, the visual evoked response sub-features are obtained.

6. The method according to claim 3, characterized in that, The cognitive-related features are obtained by extracting features from the second EEG signal segment corresponding to the stable cognitive window in the EEG signal, the cardiovascular signal segment corresponding to the stable cognitive window in the cardiovascular signal, and the second eye movement / pupil signal segment corresponding to the stable cognitive window in the eye movement / pupil signal, including: Based on the absolute and relative power of each of the multiple characteristic bands included in the second EEG signal segment, the power and ratio characteristics of the second EEG signal segment are determined, and the EEG signal sub-features are obtained. Based on the cardiovascular signal fragments, time-domain analysis is performed to determine the heart rate and heart rate variability of the target object, and cardiovascular signal sub-features are obtained. The second eye movement / pupil signal segment is analyzed to determine the eye movement behavior characteristics and pupil dilation statistical characteristics of the target object, thereby obtaining eye movement / pupil signal sub-features; The cognitive-related features are obtained based on the electroencephalogram (EEG) signal features, the cardiovascular signal features, and the eye movement / pupil signal features.

7. The method according to claim 1, characterized in that, The system conditionally fuses the light parameters of the ambient light, the summary features of the measured light output waveform, the light-specific response features, and the cognitive-related features to output a quantitative score, change index, or classification level of the target object in at least one cognitive performance dimension, thereby obtaining the cognitive performance evaluation result of the target object in response to ambient light stimuli, including: The optical specific response features are aggregated using an optical specific feature branch network to form a branch representation that characterizes the visual or neural transient response directly caused by changes in optical parameters, thus obtaining the first branch feature; The cognitive-related features are aggregated using a cognitive feature branch network to form a branch representation that characterizes continuous changes in cognitive state, thus obtaining the second branch feature; Based on the summarizing features of the ambient light parameters and / or the measured light output waveform, the first branch features and the second branch features are weighted and fused using a conditional gating weighting method through a fusion processing network to obtain fused features; The fused features are input into a dimensional multi-head output network, which outputs the quantitative score, change index, or classification level of the target object in at least one cognitive performance dimension, thereby obtaining the cognitive performance evaluation result.

8. The method according to claim 7, characterized in that, The summary features based on the ambient light parameters and / or the measured light output waveform are processed by a fusion network. The first branch features and the second branch features are weighted and fused using a conditional gating weighting method to obtain fused features, including: Based on the light parameters of the ambient light and / or the summary features of the measured light output waveform, as well as the first branch statistics related to the first branch features and the second branch statistics related to the second branch features, the weighting coefficients are determined. Based on the weighting coefficients, the first branch features and the second branch features are weighted and fused using a conditional gating weighting method to obtain the fused features.

9. The method according to claim 8, characterized in that, Also includes: Based on the summarization features of the ambient light parameters and / or the measured light output waveform, auxiliary input features are obtained; The auxiliary input features are concatenated with the fused features to obtain new fused features; Specifically, the process of inputting the fused features into the dimensional multi-head output network to output the quantitative scores, change indices, and classification levels of multiple cognitive performance dimensions, thereby obtaining the cognitive performance evaluation results, includes: The new fusion features are input into the dimensional multi-head output network to obtain the cognitive performance evaluation results.

10. A cognitive performance assessment system for environmental light stimulation based on multimodal physiological monitoring, characterized in that, include: light source; The light source controller is configured to control the output of ambient light from the light source based on a preset time reference. The measured light detection module is configured to detect the ambient light and obtain the measured light output waveform. The EEG acquisition module is configured to acquire the EEG signals of the target object at the preset time reference. An electrocardiogram (ECG) acquisition module is configured to acquire cardiovascular signals of the target object at the preset time reference. The image acquisition module is configured to acquire the eye movement / pupil signals of the target object at the preset time reference. The processing module is configured to process the measured light output waveform, the electroencephalogram signal, the cardiovascular signal, and the eye movement / pupil signal using the method described in any one of claims 1 to 9, to obtain the cognitive performance evaluation results of the target object in response to ambient light stimulation.

Images