Intelligent dimming method and system based on multi-modal physiological feedback

Abstract

The application provides a kind of intelligent dimming method and system based on multi-modal physiological feedback, it is related to intelligent lighting and lighting control technical field.The application synchronously acquires eye movement data and near-infrared functional imaging brain oxygen signal under task scene, with eye movement event as anchor point alignment and compensate brain oxygen time delay, obtain time alignment multi-modal data segment;State index reflecting attention, cognitive load and fatigue degree is extracted, individualized baseline is established and dynamically updated, and normalized state feature vector is generated;The feature vector is input into neural network light regulation model, and the target light intensity, correlated color temperature and spatial distribution ratio are output;Based on the target parameter, a joint control instruction is generated, and a short physiological response is collected to form a closed-loop feedback.The application realizes real-time adaptive matching of lighting and individual physiological state, significantly improves the accuracy and human comfort of intelligent lighting.

Description

Technical Field

[0001] This invention relates to the field of intelligent lighting and lighting control technology, and in particular to an intelligent dimming method and system based on multimodal physiological feedback. Background Technology

[0002] In practice, intelligent dimming often combines environmental information with preset strategies for control. Meanwhile, domestic publications have already incorporated physiological signals into lighting feedback. For example, a scheme that uses eye images and vital signs such as heart rate to establish a mapping between light and attention, and adjusts light intensity and color temperature accordingly (application number CN201910263082.1), has been disclosed in "Lighting Control Device Based on Attention Factors." Control systems for improving sleep efficiency use vital signs such as eye opening change rate, heart rate change rate, and body movement frequency change rate as inputs, and achieve dimming through neural networks and multi-objective optimization. Earlier, there was also "Smart Scene Lighting Based on Bio-Information Feedback," which uses information such as electrocardiogram and blood oxygenation for control. These publications provide feasible implementation paths for physiological feedback lighting.

[0003] Human-centered dynamic lighting is evolving from single-signal to multi-source biometric fusion, focusing on combining behavioral and physiological indicators for online assessment while ensuring safety and energy efficiency, and coordinating the adjustment of light intensity, correlated color temperature, and spatial distribution. Domestically, there are also publicly available ideas on using wearable devices to collect information such as body temperature, pulse, and heart rate for indoor feedback lighting control, reflecting a technological trend towards personalization and self-adaptation.

[0004] Existing technologies primarily rely on eye images and vital signs such as heart rate for lighting control. A complete technical solution exists that synchronizes and compensates for delays between near-infrared brain oxygenation signals and eye movement signals on the same timeframe before using them for real-time closed-loop dimming. Simultaneous acquisition of near-infrared brain functional imaging and eye movements is currently used more for cognitive assessments than directly for lighting control closed-loop systems. This creates a risk of time mismatch between physiological representations and immediate control during rapid task switching or state fluctuations. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide an intelligent dimming method and system based on multimodal physiological feedback. By introducing multimodal physiological feedback and a closed-loop dimming strategy, the lighting system achieves real-time perception and adaptive adjustment of individual physiological states, significantly improving the accuracy and human-factor matching of intelligent lighting.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A smart dimming method based on multimodal physiological feedback includes:

[0008] In the task scenario, eye-tracking data and near-infrared functional imaging brain oxygen signals are collected simultaneously. The physiological event sequence detected by eye-tracking data is used as the alignment anchor point. The individualized brain oxygen latency is estimated by maximizing the event alignment correlation and then compensated to obtain time-aligned multimodal data fragments.

[0009] State indicators representing attention, cognitive load, and fatigue levels are extracted from the multimodal data fragments. An individualized dynamic baseline is established upon first use or task switching and updated online during operation to generate a normalized state feature vector.

[0010] The state feature vector is input into the illumination adjustment decision model to obtain the illumination target parameters;

[0011] Based on the lighting target parameters, a joint control command for light intensity, correlated color temperature, and spatial distribution is generated;

[0012] After the joint control command is executed, a short-term physiological response is collected, and the process returns to the step "synchronously collect eye movement data and near-infrared functional imaging brain oxygen signal in the task scenario" to run in a loop, so as to keep the lighting environment and the user's physiological state in real time in a closed loop.

[0013] Preferably, eye-tracking data and near-infrared functional imaging brain oxygenation signals are simultaneously acquired in the task scenario. Physiological event sequences detected by eye-tracking data are used as alignment anchors. Individualized brain oxygenation latency is estimated and compensated by maximizing event alignment correlation, resulting in time-aligned multimodal data segments, including:

[0014] The eye movement data is smoothed and thresholded to detect fixation initiation, saccade initiation, and blinking events, generating a physiological event sequence arranged in chronological order.

[0015] Detrending and bandpass filtering were performed on near-infrared functional imaging brain oxygen signals to obtain brain oxygen signal sequences for alignment.

[0016] The physiological event sequence is represented as an indicator function within a preset time window. A normalized cross-correlation coefficient is calculated between the physiological event sequence and the brain oxygen signal sequence. Within a physiologically reasonable delay search interval, the delay corresponding to the maximum normalized cross-correlation coefficient is taken as the individualized brain oxygen delay. Based on the individualized brain oxygen delay, non-integer sampling shift compensation is performed on the brain oxygen signal sequence. Equal-length segments are extracted centered on the physiological event sequence to generate time-aligned multimodal data segments. The delay search interval is in the range of seconds to match hemodynamic response characteristics.

[0017] Preferably, the physiological event sequence is represented as an indicator function within a preset time window, and a normalized cross-correlation coefficient is calculated between it and the brain oxygen signal sequence. Within a physiologically reasonable delay search interval, the delay corresponding to the maximum normalized cross-correlation coefficient is taken as the individualized brain oxygen delay. Based on the individualized brain oxygen delay, non-integer sampling shift compensation is performed on the brain oxygen signal sequence, and equal-length segments are extracted centered on the physiological event sequence to generate time-aligned multimodal data segments. The delay search interval is in the range of seconds to match hemodynamic response characteristics, including:

[0018] The set of physiological event moments obtained based on the eye-tracking data detection A discrete event indicator function is constructed within a sampling domain consistent with the brain oxygen signal sequence; the formula for constructing the discrete event indicator function is: ;in, For discrete event indicator functions; For discrete sampling index; For the first Discrete sampling index corresponding to each eye movement physiological event; To indicate the width of the pulse in the sample domain; For a rectangular window, satisfying ,when Otherwise, it is 0;

[0019] The discrete event indicator function Brain oxygen signal sequence compared with near-infrared functional imaging Within a preset time window, mean removal and normalization are performed, the normalized cross-correlation between the two is calculated, and the correlation peak is searched within a physiologically reasonable set of second-level delays to obtain integer sample-level latency; the mean removal calculation formula is: The normalization formula is as follows: The normalized cross-correlation formula is as follows: , The formula for the second-level delay search interval is: ;in, This is a discrete-time sequence of brain oxygenation signals; and The mean within the window; and This is the sequence after removing the mean; This is the normalized cross-correlation coefficient; To prevent small positive numbers with a denominator of zero; For sample-level delay; For in set Internal envoy Maximum integer sample level latency; For a search set with a delay of seconds; The sampling frequency; and These correspond to the boundaries of negative two seconds and eight seconds in the sample domain, respectively; The sample length for the time window; Starting index of the window

[0020] right The cross-correlation peaks at the specified points are refined using parabolic interpolation to obtain individualized brain oxygen delays. Time compensation is then applied to the brain oxygen signal sequence based on these individualized delays to achieve fine alignment, generating time-aligned multimodal data segments. The delay refinement formula is as follows: ;in, The individualized brain oxygen delay obtained by parabolic interpolation refinement is used to analyze brain oxygen signal sequences. Perform fractional sample shift compensation to obtain the discrete event indicator function. Alignment results on the same time base.

[0021] Preferably, state indicators representing attention, cognitive load, and fatigue levels are extracted from the multimodal data fragments. An individualized dynamic baseline is established upon first use or task switching, and updated online during operation to generate a normalized state feature vector, including:

[0022] Within the sliding time window, eye-tracking state indicators and brain oxygenation state indicators are extracted from the multimodal data segments respectively, and all the state indicators extracted in each time window are used to construct the multimodal state data at the current moment.

[0023] Upon first use or task switching, the multimodal state data of the user in a resting or light-task state are selected, and the average value and fluctuation range of each state index are calculated to form the individualized dynamic baseline; the individualized dynamic baseline is used to reflect the user's physiological balance range in a natural state.

[0024] Based on the current task stage and the user's latest physiological performance, the individualized dynamic baseline is updated recursively. The update process adopts a gradual adjustment strategy so that the parameters of the individualized dynamic baseline can gradually adapt to the individual's state drift without being affected by transient anomalies, thereby maintaining the stability and continuity of the physiological reference range.

[0025] The multimodal state data of each time window is compared with the corresponding individualized dynamic baseline parameters to obtain the standardized results of each state index. The standardized results are then combined in a fixed order to form the state feature vector. The state feature vector is used to comprehensively represent the user's current attention level, cognitive load intensity, and fatigue level.

[0026] Preferably, the eye-tracking state indicators include fixation stability, saccade density, pupil diameter change rate, and blink frequency, which are used to reflect the user's visual focus and instantaneous fatigue level; the brain oxygenation state indicators include changes in oxyhemoglobin concentration, changes in deoxyhemoglobin concentration, and the left-right brain oxygenation asymmetry index, which are used to characterize cognitive load and neural activation level.

[0027] Preferably, the state feature vector is input into the illumination adjustment decision model to obtain the illumination target parameters, including:

[0028] A multi-branch regression model based on a neural network is established. The input of the multi-branch regression model is the state feature vector, and the output is the illumination target parameters. The illumination target parameters include at least the target light intensity, the target correlated color temperature, and the target spatial distribution ratio. The multi-branch regression model includes an input layer, several hidden layers, and three output branches corresponding to the target light intensity, the target correlated color temperature, and the target spatial distribution ratio. Each output branch regresses its own target after sharing features.

[0029] Historical data containing task scenarios, physiological signals, and known preferred lighting parameters are collected, and a state feature vector for training is generated based on the historical data. The corresponding preferred light intensity, preferred correlated color temperature, and preferred spatial distribution ratio are used as supervision labels to form a training set and a validation set. The preferred parameters are determined by a combination of expert annotation, subject subjective scores, and objective performance indicators.

[0030] The multi-branch regression model is trained using regression loss and the training set, and early stopping and hyperparameter fixing are performed using the validation set.

[0031] After training is completed, the state feature vector is input into the trained multi-branch regression model to obtain the lighting target parameters.

[0032] Preferably, the joint control command includes:

[0033] Light intensity control commands are used to set the luminous intensity of the lighting unit in order to adjust the lighting brightness;

[0034] Related color temperature control commands are used to set the color temperature output of the lighting unit, so as to adjust the color temperature between cool light and warm light;

[0035] Distributed control commands are used to set the spatial output ratio of lighting units to distribute luminous flux among multiple lighting areas or luminaires.

[0036] Preferably, after the joint control command is executed, a short-term physiological response is collected, and the process returns to the step "synchronously collecting eye-tracking data and near-infrared functional imaging brain oxygenation signals in the task scenario" for cyclical operation, so as to maintain real-time closed-loop matching between the lighting environment and the user's physiological state, including:

[0037] After the lighting system completes the joint adjustment of light intensity, correlated color temperature and spatial distribution, a physiological monitoring trigger signal is activated to indicate the start of a new acquisition cycle.

[0038] Within the adjusted preset time window, the user's eye movement data and near-infrared brain oxygenation signal are collected simultaneously to reflect the immediate response of lighting changes to visual and neural states.

[0039] The collected short-term physiological data are subjected to time truncation and artifact removal processing to ensure signal continuity and temporal consistency with the data from the previous collection cycle, thereby obtaining short-term physiological response data fragments.

[0040] The short-term physiological response data fragments are used as new eye-tracking data and near-infrared functional imaging brain oxygen signals, and the system returns "Synchronous acquisition of eye-tracking data and near-infrared functional imaging brain oxygen signals in the task scenario" for multimodal data fusion and state assessment in the next cycle, so as to achieve real-time closed-loop matching between illumination regulation and physiological state.

[0041] Preferably, the preset time window is 10 seconds.

[0042] A smart dimming system based on multimodal physiological feedback includes:

[0043] The multimodal data synchronous acquisition and time base alignment module is used to synchronously acquire eye-tracking data and near-infrared functional imaging brain oxygen signals in the task scenario. The physiological event sequence detected by eye-tracking data is used as the alignment anchor point. The individualized brain oxygen delay is estimated and compensated by maximizing the event alignment correlation to obtain time-aligned multimodal data segments.

[0044] The state representation and individualized baseline update module is used to extract state indicators representing attention, cognitive load and fatigue from the multimodal data fragments, establish an individualized dynamic baseline when first used or when switching tasks, and update it online during operation to generate a normalized state feature vector.

[0045] The illumination adjustment decision model module is used to input the state feature vector into the illumination adjustment decision model to obtain the illumination target parameters;

[0046] The joint control execution module is used to generate joint control instructions for light intensity, correlated color temperature and spatial distribution based on the lighting target parameters;

[0047] The closed-loop acquisition and feedback module is used to acquire short-term physiological responses after the joint control command is executed, and return to the step "synchronously acquire eye movement data and near-infrared functional imaging brain oxygen signal in the task scenario" to run in a loop, so as to keep the lighting environment and the user's physiological state in real time in a closed loop.

[0048] The present invention discloses the following technical effects:

[0049] This invention achieves precise matching between behavioral and neural physiological signals by simultaneously acquiring eye-tracking data and near-infrared functional imaging brain oxygenation signals, using eye-tracking events as time-base alignment anchors to compensate for individualized time lags in brain oxygenation signals. Compared to traditional dimming methods that rely solely on a single sensor or environmental sensor, this approach more accurately reflects the user's real-time attention and cognitive state, thus ensuring consistency between illumination adjustment results and human physiological responses, significantly improving the physiological adaptability of lighting.

[0050] By establishing and continuously updating a personalized dynamic baseline, this invention can adapt to physiological differences among different users and at different task stages, enabling the system to learn and adjust itself based on individual characteristics. This dynamic baseline mechanism allows the dimming process to no longer rely on a fixed threshold, but rather to make flexible adjustments based on changes in individual states, achieving truly personalized lighting control.

[0051] This invention inputs state feature vectors into a lighting regulation decision model, automatically outputs target lighting parameters, and combines this with joint control of light intensity, correlated color temperature, and spatial distribution to form a closed-loop control chain centered on "physiological state perception—model decision-making—lighting execution—physiological feedback." This closed-loop mechanism enables the lighting system to perceive and respond to minute changes in physiological state in real time, maintaining stable cognitive alertness and visual comfort.

[0052] The method of this invention can achieve dynamic lighting adjustment in various task scenarios such as learning and training, surgical procedures, driving monitoring, and office environments. By balancing visual comfort, cognitive load, and energy efficiency, it reduces fatigue and extends optimal working hours. Compared with traditional dimming methods that rely on environmental parameters, this method significantly improves intelligence, human-centered design, and energy efficiency, providing a new path for the human-centered optimization of intelligent lighting systems. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart of the method provided in an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the system structure provided in an embodiment of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] The purpose of this invention is to provide an intelligent dimming method and system based on multimodal physiological feedback. By integrating two types of physiological signals, eye movement and brain oxygenation, an individualized dynamic baseline is established and combined with a neural network decision model to achieve real-time closed-loop adjustment of the lighting system and the user's physiological state, thereby significantly improving the adaptability, individualization and physiological matching of lighting.

[0058] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Figure 1 The method flowchart provided in the embodiments of the present invention is as follows: Figure 1 As shown, this invention provides an intelligent dimming method based on multimodal physiological feedback, comprising:

[0060] Step 100: Simultaneously collect eye-tracking data and near-infrared functional imaging brain oxygen signals in the task scenario. Use the physiological event sequence detected by eye-tracking data as the alignment anchor point. Maximize the individualized brain oxygen latency through event alignment correlation and compensate for it to obtain time-aligned multimodal data segments.

[0061] Step 200: Extract state indicators representing attention, cognitive load and fatigue from multimodal data fragments, establish individualized dynamic baselines upon first use or task switching, and update them online during operation to generate normalized state feature vectors;

[0062] Step 300: Input the state feature vector into the illumination adjustment decision model to obtain the illumination target parameters;

[0063] Step 400: Generate joint control instructions for light intensity, correlated color temperature and spatial distribution based on the lighting target parameters;

[0064] Step 500: After executing the joint control command, collect short-term physiological responses and return to the step "Synchronously collect eye movement data and near-infrared functional imaging brain oxygen signal in the task scenario" to run in a loop, so as to keep the lighting environment and the user's physiological state in real time in a closed loop.

[0065] Specifically, step 100 in this embodiment includes:

[0066] This embodiment simultaneously acquires eye-tracking data and near-infrared functional imaging (NII) brain oxygenation signals in a task scenario. Eye-tracking data is acquired using desktop or head-mounted acquisition devices with a sampling frequency of 60Hz to 120Hz; the NII brain oxygenation signal sampling frequency is set to 10Hz to 12Hz, covering the task-related frontal or parietal lobe regions. To improve robustness, this embodiment smooths the eye-tracking data to remove transient noise, and then detects three types of events according to fixed threshold rules: ① fixation initiation event, defined as the fixation point maintaining a small range of fluctuation for more than 120ms; ② saccade initiation event, defined as a significant displacement of the fixation point within a time of less than 40ms; ③ blink event, defined as a rapid recovery of the pupil signal after a brief loss. These three types of events are arranged chronologically to form a "physiological event sequence." The "physiological event sequence" refers to the temporally ordered set of fixation initiation, saccade initiation, and blink events, used as a time anchor point for cross-modal data alignment.

[0067] In this embodiment, near-infrared brain oxygenation signals are sequentially detrended and bandpass filtered. The detrending process uses a sliding window baseline of approximately 20 seconds to eliminate slow drift; the bandpass filtering range is 0.01Hz to 0.2Hz, used to preserve the hemodynamic response band while suppressing high-frequency noise and extremely low-frequency drift. To reduce motion artifacts, this embodiment performs amplitude anomaly removal before filtering; when a channel experiences abrupt changes within 0.5 seconds, it is interpolated using the average value of neighboring signals. After processing, the signals from task-related regions (such as the left and right frontal lobe channels) are averaged to obtain an "aligned brain oxygenation signal sequence." This sequence is used to represent stable task-related blood oxygenation changes.

[0068] To estimate the individualized time delay of brain oxygenation signals relative to eye-tracking signals, this embodiment converts the "physiological event sequence" into discrete pulse time stamps within a preset time window. Specifically, a 0.1s wide pulse is inserted at the moment of each event to eliminate slight time jitter. This pulse sequence, called the "indicator function," is used to mark the event location in time. Then, the normalized cross-correlation coefficient between this indicator function and the "brain oxygenation signal sequence for alignment" is calculated. The maximum value of the correlation coefficient is found within a delay search interval, and the corresponding time offset is defined as the individualized brain oxygenation delay. The delay search interval is set to −2s to +8s to match the physiological characteristic that human cerebral hemodynamic responses typically peak between 3s and 5s. For example, if the calculated correlation peak for a subject is at 3.4s, then the individualized brain oxygenation delay is 3.4s.

[0069] After obtaining the individualized brain oxygenation time delay, this embodiment performs non-integer sampling shift compensation on the "brain oxygen signal sequence for alignment," that is, shifts the signal sequence along the time axis at a subsampling level according to the aforementioned time delay. Spline interpolation is preferably used to ensure that the shift accuracy is less than half of the sampling period. After compensation, equal-length segments are extracted centered on each physiological event, for example, each segment contains data ranging from 2 seconds before the event to 8 seconds after the event, resulting in a "time-aligned multimodal data segment." This data segment simultaneously contains eye-tracking signals and aligned brain oxygen signals, and can be directly input into subsequent state index extraction and individualized baseline update modules to achieve precise synchronization and alignment of multimodal data in the time domain.

[0070] Furthermore, in this embodiment, three types of events are first extracted based on eye-tracking data: fixation initiation, saccade initiation, and blink. Event detection can employ common thresholds and minimum duration rules, such as: fixation initiation is defined as the fixation point remaining stable for more than 120ms within a visual field of no more than 0.5 degrees; saccade initiation is defined as a significant displacement of the eye position within less than 40ms; and blink is defined as the pupil signal disappearing briefly and recovering within 200ms. The occurrence times of each event are arranged chronologically to form a "physiological event time set." Under the same sampling rhythm as the brain oxygen signal (e.g., brain oxygen sampling frequency of 10Hz to 12Hz), this embodiment places a narrow pulse at each event time, with a pulse width of 0.1s (corresponding to 1 to 2 brain oxygen sampling points). Its function is to convert discrete event times into a time-stamped sequence with the same frequency as the brain oxygen signal. This time-stamped sequence is referred to as the "discrete event indicator function." The function of the "discrete event indicator function" is to provide a time axis representation corresponding one-to-one with the brain oxygen sequence for subsequent correlation calculations and to allow for tolerance absorption of minor jitter present at the event times.

[0071] To measure the consistency between the event indicator function and the brain oxygenation signal at different time offsets, this embodiment calculates the "normalized cross-correlation coefficient". The calculation process uses a fixed-length time window, ranging from 20 to 60 seconds, with the window starting point sliding in steps of 5 to 10 seconds. Within each window, the two sequences are first normalized by removing the mean and amplitude to eliminate static bias and dimensional differences. Then, the correlation is calculated one by one at a given delay value. The correlation value ranges from -1 to 1, with the absolute value closer to 1 indicating stronger consistency. To avoid instability caused by zero or extremely small denominators, this embodiment introduces a minimal stability constant, such as 0.000001, during the normalization process; if there are insufficient valid data points within a window, that window is skipped. The "normalized cross-correlation coefficient" referred to in this paper is the correlation measure obtained based on the above normalization process, reflecting the strength of linear consistency between the two sequences at a given delay.

[0072] Considering that hemodynamic responses typically have a second-level delay relative to behavioral events, this embodiment limits the delay search interval to -2s to +8s, and converts this time range into discrete values ​​according to the brain oxygen sampling frequency for evaluation (e.g., at a sampling frequency of 10Hz, this corresponds to -20 to +80 sampling points). Within this search interval, the position of the maximum value of the normalized cross-correlation coefficient is selected as the initial estimate of the "integer sample-level delay". To improve alignment accuracy, this embodiment performs quadratic curve fitting on the three discrete values ​​adjacent to this maximum value to obtain a refined result of the "non-integer sample-level delay". This paper defines the refined delay value as "individualized brain oxygen delay", which is used to describe the average time lag of brain oxygen changes relative to eye movement events for the same user in a specific task phase. Taking one subject as an example, if the maximum correlation occurs at approximately 3.4s, the individualized brain oxygen delay is taken as 3.4s; if another subject's delay is 2.8s in the same task, subsequent compensation is based on 2.8s to reflect individual differences.

[0073] After obtaining the individualized brain oxygen delay, this embodiment performs "non-integer sampling shift compensation" on the brain oxygen signal. Specifically, the delay value obtained by shifting the brain oxygen sequence along the time axis is shifted with a shift accuracy preferably better than half of the original sampling period; for example, under 10Hz sampling conditions, the shift error is controlled to be less than 50ms. To avoid jagged edges and distortion, spline interpolation or an equivalent high-precision resampling method is used to complete the sub-sampling level shift. After compensation, an equal-length segment is extracted centered on each event, for example, from 2 seconds before the event to 8 seconds after the event. The eye-tracking and already compensated brain oxygen signals are simultaneously extracted to obtain a "time-aligned multimodal data segment." This data segment will be used in subsequent state indicator extraction and individualized baseline update processes and can be directly input into the next processing step. Thus far, terms such as "discrete event indicator function," "normalized cross-correlation coefficient," "second-level delay search interval," "personalized brain oxygenation delay," and "non-integer sampling shift compensation" have all been given their functions, values, and application locations in this embodiment. Related parameter examples (such as 0.1s pulse width, −2s to +8s search interval, 10Hz to 12Hz sampling frequency, 20s to 60s window length, and stability constant 0.000001) are all reproducible and commonly used engineering value ranges, which can be implemented by those skilled in the art without creative effort.

[0074] Specifically, step 200 in this embodiment includes:

[0075] This embodiment uses a sliding time window to process time-aligned multimodal data. Preferably, the window length is 5 to 10 seconds, and the step size is 1 to 2 seconds. For eye-tracking data, four state indicators are extracted: fixation stability, saccade density, pupil diameter change rate, and blink frequency. Fixation stability measures the stability of the fixation point within the window, which can be achieved by statistically analyzing the percentage of the fixation point falling into a small area, preferably with an angular radius of 0.5 degrees. Saccades density reflects the intensity of rapid eye movements and can be calculated based on the number of events that meet a velocity threshold per unit time, preferably on the order of 100 units per second. The pupil diameter change rate reflects the instantaneous changes in sympathetic arousal, and the unit can be millimeters per second. Blink frequency indicates the instantaneous fatigue level, and the unit can be times per minute. For near-infrared brain oxygenation signals, three state indicators are extracted: changes in oxyhemoglobin concentration, changes in deoxyhemoglobin concentration, and the left-right brain oxygenation asymmetry index. Brain oxygenation indicators are obtained by comparing the average or robust median value within the window with a reference level in the same channel, where the reference level comes from the individualized dynamic baseline described later. The above seven indicators constitute the basic measures of attention, cognitive load, and fatigue.

[0076] This embodiment combines all eye-tracking and brain oxygenation status indicators extracted within a time window into "multimodal state data" in a fixed order. "Multimodal state data" refers to an ordered record of multiple numerical state quantities originating from different physiological channels within the same time window. The preferred order is: fixation stability, saccade density, pupil diameter change rate, blink frequency, oxyhemoglobin concentration change, deoxyhemoglobin concentration change, and left-right brain oxygenation asymmetry index. To reduce noise, this embodiment performs outlier removal and mild smoothing on each indicator before construction: when a single point deviates from the median within the window by more than a reasonable range, it is marked and replaced with a neighboring value; the smoothing length is preferably no more than 20% of the window length. The multimodal state data obtained after this processing serves as the direct input for subsequent baseline establishment and standardization.

[0077] Upon first use or during task switching, this embodiment collects at least 60 seconds of multimodal data in a resting or light-task state, and obtains a continuous multimodal state data sequence according to the aforementioned windowing strategy. For each state indicator, its average level and fluctuation range are calculated to form an "individualized dynamic baseline." The "individualized dynamic baseline" refers to a reference range for each state indicator for a single user in a specific scenario, including both the central level and a reasonable fluctuation width, used to subsequently determine the degree of deviation of the current state. The average level can be the arithmetic mean or robust average of the entire window, and the fluctuation range can be the range from the 5th percentile to the 95th percentile or an equivalent robust range. For example, a user's fixation stability at baseline can be 0.75 (dimensionless, expressed as a percentage), with a fluctuation range of 0.60 to 0.85; the pupil diameter rate of change can be 0.10 mm / s, with a fluctuation range of 0.05 to 0.20 mm / s; and the oxyhemoglobin concentration rate of change can be approximately 0, with the corresponding fluctuation range determined by the device range and resting fluctuation.

[0078] Once in operation, to accommodate the slow drift of individual states, this embodiment performs online updates to the individualized dynamic baseline. The "gradual adjustment strategy" refers to using the current window's state indicators to make small-step corrections to the baseline center and fluctuation range at the end of each window, ensuring the reference interval moves smoothly over time without being dragged by instantaneous anomalies. The coefficient for the small-step correction is preferably between 0.01 and 0.05; when the current indicator exceeds three times the width of the baseline fluctuation range, it is considered abnormal and not included in this update; when multiple consecutive windows show slow shifts in the same direction, the baseline is continuously updated in small steps until a new stable level is reached. This strategy ensures that the individualized dynamic baseline can reflect changes in diurnal rhythms or task phases in a timely manner, while avoiding misjudgments introduced by short-term glare, micro-motions, or sensor jitter.

[0079] In each window, this embodiment compares the multimodal state data with the current individualized dynamic baseline to obtain "standardized results," which are then combined in a fixed order to form a "state feature vector." The "standardized results" refer to converting each state indicator into a comparable dimensionless quantity using the baseline center as a reference and the baseline fluctuation width as a scale, thus eliminating differences in dimensions and ranges. In implementation, the deviation from the baseline center is first calculated, then scaled according to the baseline fluctuation width, and extreme values ​​are clipped to a reasonable range, preferably between -3 and +3. The seven standardized results are arranged sequentially in the fixed order of the second paragraph to form the "state feature vector," which comprehensively represents the current attention level, cognitive load intensity, and fatigue level. This state feature vector directly serves as the input to the next step of the illumination adjustment decision model and is continuously updated in subsequent closed loops, thereby achieving dynamic lighting control that matches the individual's physiological state.

[0080] Specifically, steps 300 and 400 in this embodiment include:

[0081] This embodiment establishes a multi-branch regression model based on a neural network to map the state feature vector generated in step 200 to lighting target parameters. The multi-branch regression model refers to setting independent branches for different output targets after the shared feature extraction layer. The input is a state feature vector containing standardized eye-tracking and brain oxygenation indicators. The shared feature extraction layer uses a fully connected structure and a non-linear activation function; for example, it contains three hidden layers with 128, 64, and 32 neurons respectively. The activation function is a rectified linear unit, and dropout ratios of 0.1 and 0.2 are set between layers to prevent overfitting. A normalization layer is added after each layer to stabilize the data distribution. After shared feature extraction, three output branches are generated, corresponding to target light intensity, target correlated color temperature, and target spatial distribution ratio, respectively. Each branch contains one to two fully connected layers; for example, 16 and 8 nodes, ultimately outputting a single value or vector. The target spatial distribution ratio is used to allocate luminous flux among multiple lighting areas or luminaires; its dimension is equal to the number of controllable areas, and the output, after normalization and boundary clipping, satisfies the constraint that the sum of the ratios is 1.

[0082] This embodiment collects historical data from various task scenarios for model training. The data includes synchronized eye-tracking and brain oxygenation signals, environmental records, and lighting parameters. Following steps 100 and 200, the raw data undergoes temporal alignment, feature extraction, and normalization to obtain the training input. Supervision labels include preferred light intensity, preferred correlated color temperature, and preferred spatial distribution ratio, derived from a combination of expert annotations, subject subjective ratings, and objective performance indicators. Expert annotations are based on glare risk, visual comfort, and cognitive alertness assessments; subjective ratings use a 10-point scale for immediate feedback; objective performance is calculated using indicators such as accuracy and reaction time. These three factors are fused into preferred labels according to preset weights. The label range corresponds to the lighting equipment's range: light intensity range is 0 to 100 (percentage), correlated color temperature is 2700 K to 6500 K, and spatial distribution ratio is between 0 and 1, with the sum of the ratios of each channel being 1. The data is divided into training and validation sets in an 8:2 ratio. Data augmentation can be used on the training set, such as introducing amplitude perturbations of ±10% and fine-tuning the time shift within 0.5 s.

[0083] This embodiment employs a regression loss function for end-to-end training, aiming to minimize the error between the three output classes and their corresponding preferred labels, while adding boundary constraints on the spatial distribution ratio. The optimizer uses an adaptive moment estimation method, with an initial learning rate of 0.001, a batch size of 32, and a maximum training epoch of 200. An early stopping strategy is implemented, automatically stopping and retaining the optimal weights when the validation error fails to improve for 10 consecutive epochs. To determine the optimal configuration, a hyperparameter search is performed: the learning rate is between 0.005, 0.001, and 0.0005; the dropout ratio is between 0.1 and 0.2; and the hidden layer width is between 64 and 128. The final parameters are selected by comparing the mean absolute error and output stability using a validation set. After training, independent test data is used for evaluation, ensuring that the mean absolute errors of output light intensity, color temperature, and distribution ratio remain within the target range (light intensity error not exceeding 5%, color temperature error not exceeding 200 K, and spatial distribution ratio error not exceeding 0.05). All model parameters, training configurations, and data preprocessing rules are recorded for reproducibility.

[0084] In this embodiment, during the runtime phase, the multi-branch regression model trained by receiving state feature vector input outputs the target light intensity, target correlated color temperature, and target spatial distribution ratio. To avoid abrupt changes and flickering, the output results undergo two types of constraint processing: rate of change limitation and hysteresis control. Rate of change limitation means that the single-cycle change of adjacent outputs does not exceed a set amplitude; light intensity is limited to no more than 10% per second, color temperature is limited to no more than 200 K per second, and the spatial distribution ratio per channel does not exceed 0.1. Hysteresis control means that when the change amplitude is below a threshold, the current state is maintained; typical thresholds are 10% light intensity, 100 K color temperature, and 0.05 spatial distribution ratio. The constrained results generate joint control instructions, including light intensity control instructions, correlated color temperature control instructions, and distribution control instructions. The light intensity control instruction is used to set the brightness of the illumination unit, the correlated color temperature control instruction is used to adjust the ratio of warm and cool light, and the distribution control instruction is used to control the distribution of luminous flux among multiple regions. After execution, a short-term physiological response is collected, and the process returns to step 100 for the next round of adjustment.

[0085] Further, step 500 of this embodiment includes:

[0086] In this embodiment, after the light intensity, correlated color temperature, and spatial distribution are jointly adjusted, a "physiological monitoring trigger signal" is immediately issued to mark the starting point of a new round of data acquisition. The "physiological monitoring trigger signal" is a control marker used to unify the start time. It can be generated by a dimming command confirmation receipt or a local timing marker. Its function is to start timing after the dimming action is completed, avoiding the mixing of transient fluctuations during the transition period into the response evaluation. To ensure stability, this embodiment sets a buffer time of 0.5–1.0 seconds after triggering before entering a preset time window for synchronous acquisition; the preset time window is preferably 10 seconds, used to cover the rapid changes in eye movement and the early response period of near-infrared brain oxygenation.

[0087] Within a 10-second time window, this embodiment simultaneously acquires eye-tracking data and near-infrared brain oxygenation signals. Eye-tracking data is sampled at a frequency of 60–120 Hz, recording fixation points, saccades, and pupil diameter, etc.; near-infrared brain oxygenation signals are sampled at a frequency of 10–12 Hz, monitoring brain regions relevant to the task. To avoid bias caused by channel inconsistencies, the acquisition target and channel selection are reused from the previous cycle, maintaining the sensing pose and region of interest unchanged. In this embodiment, the physiological changes acquired within 10 seconds that are directly affected by the current dimming are termed "short-term physiological response," which characterizes the immediate impact of illumination changes on vision and neural states, serving as a rapid feedback quantity in the closed loop.

[0088] This embodiment first truncates the original recording, retaining only the complete 10-second segment after the trigger buffer ends, to avoid including equipment transition periods or human micro-movements in the analysis. Subsequently, artifact removal is performed: blink segment interpolation and short-term noise smoothing are applied to eye-tracking data; body motion artifact identification and amplitude anomaly removal are performed on near-infrared brain oxygenation signals, maintaining the same bandpass range and reference channel as the previous cycle; windows with missing rates exceeding the threshold are discarded and re-acquired. To ensure "time sequence consistency with the data from the previous acquisition cycle," this embodiment uses the "physiological monitoring trigger signal" as a homogeneous reference point, aligning the current 10-second segment with the corresponding segment from the previous cycle, maintaining the same time scale and window length. The above steps yield "short-term physiological response data segments," which are usable data segments that, under a unified starting point and fixed length, have undergone quality control and artifact removal and can be used for closed-loop evaluation and decision updates.

[0089] Corresponding to the above methods, such as Figure 2 As shown, this embodiment also provides an intelligent dimming system based on multimodal physiological feedback, including:

[0090] The multimodal data synchronous acquisition and time base alignment module is used to synchronously acquire eye-tracking data and near-infrared functional imaging brain oxygen signals in the task scenario. The physiological event sequence detected by eye-tracking data is used as the alignment anchor point. The individualized brain oxygen delay is estimated and compensated by maximizing the event alignment correlation to obtain time-aligned multimodal data segments.

[0091] The state representation and individualized baseline update module is used to extract state indicators representing attention, cognitive load and fatigue from the multimodal data fragments, establish an individualized dynamic baseline when first used or when switching tasks, and update it online during operation to generate a normalized state feature vector.

[0092] The illumination adjustment decision model module is used to input the state feature vector into the illumination adjustment decision model to obtain the illumination target parameters;

[0093] The joint control execution module is used to generate joint control instructions for light intensity, correlated color temperature and spatial distribution based on the lighting target parameters;

[0094] The closed-loop acquisition and feedback module is used to acquire short-term physiological responses after the joint control command is executed, and return to the step "synchronously acquire eye movement data and near-infrared functional imaging brain oxygen signal in the task scenario" to run in a loop, so as to keep the lighting environment and the user's physiological state in real time in a closed loop.

[0095] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0096] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for intelligent dimming based on multi-modal physiological feedback, characterized in that, include: In the task scenario, eye-tracking data and near-infrared functional imaging brain oxygen signals are collected simultaneously. The physiological event sequence detected by eye-tracking data is used as the alignment anchor point. The individualized brain oxygen latency is estimated by maximizing the event alignment correlation and then compensated to obtain time-aligned multimodal data fragments. State indicators representing attention, cognitive load, and fatigue levels are extracted from the multimodal data fragments. An individualized dynamic baseline is established upon first use or task switching and updated online during operation to generate a normalized state feature vector. The state feature vector is input into the illumination adjustment decision model to obtain the illumination target parameters; Based on the lighting target parameters, a joint control command for light intensity, correlated color temperature, and spatial distribution is generated; After the joint control command is executed, a short-term physiological response is collected, and the process returns to the step of synchronously collecting eye-tracking data and near-infrared functional imaging brain oxygen signal in the task scenario to keep the lighting environment and the user's physiological state in real time in a closed loop. In a task scenario, eye-tracking data and near-infrared functional imaging brain oxygenation signals are simultaneously acquired. Physiological event sequences detected from eye-tracking data are used as alignment anchors. Individualized brain oxygenation latency is estimated and compensated for by maximizing event alignment correlation, resulting in time-aligned multimodal data fragments, including: The eye movement data is smoothed and thresholded to detect fixation initiation, saccade initiation, and blinking events, generating a physiological event sequence arranged in chronological order. Detrending and bandpass filtering were performed on near-infrared functional imaging brain oxygen signals to obtain brain oxygen signal sequences for alignment. The physiological event sequence is represented as an indicator function within a preset time window. A normalized cross-correlation coefficient is calculated between the physiological event sequence and the brain oxygen signal sequence. Within a physiologically reasonable delay search interval, the delay corresponding to the maximum normalized cross-correlation coefficient is taken as the individualized brain oxygen delay. Based on the individualized brain oxygen delay, non-integer sampling shift compensation is performed on the brain oxygen signal sequence. Equal-length segments are extracted centered on the physiological event sequence to generate time-aligned multimodal data segments. The delay search interval is in the range of seconds to match hemodynamic response characteristics. State indicators representing attention, cognitive load, and fatigue levels are extracted from the multimodal data fragments. Individualized dynamic baselines are established upon first use or task switching and updated online during operation to generate normalized state feature vectors, including: Within the sliding time window, eye-tracking state indicators and brain oxygenation state indicators are extracted from the multimodal data segments respectively, and all the state indicators extracted in each time window are used to construct the multimodal state data at the current moment. Upon first use or task switching, the multimodal state data of the user in a resting or light-task state are selected, and the average value and fluctuation range of each state index are calculated to form the individualized dynamic baseline; the individualized dynamic baseline is used to reflect the user's physiological balance range in a natural state. Based on the current task stage and the user's latest physiological performance, the individualized dynamic baseline is updated recursively. The update process adopts a gradual adjustment strategy so that the parameters of the individualized dynamic baseline can gradually adapt to the individual's state drift without being affected by transient anomalies, thereby maintaining the stability and continuity of the physiological reference range. The multimodal state data of each time window is compared with the corresponding individualized dynamic baseline parameters to obtain the standardized results of each state index. The standardized results are then combined in a fixed order to form the state feature vector. The state feature vector is used to comprehensively represent the user's current attention level, cognitive load intensity, and fatigue level.

2. The intelligent dimming method based on multi-modal physiological feedback according to claim 1, characterized in that, The physiological event sequence is represented as an indicator function within a preset time window. A normalized cross-correlation coefficient is calculated with the brain oxygen signal sequence. Within a physiologically reasonable delay search interval, the delay corresponding to the maximum normalized cross-correlation coefficient is taken as the individualized brain oxygen delay. Non-integer sampling shift compensation is performed on the brain oxygen signal sequence based on the individualized brain oxygen delay. Equal-length segments are extracted with the physiological event sequence as the center to generate time-aligned multimodal data segments. The delay search interval is in the range of seconds to match hemodynamic response characteristics, including: a set of physiological event time instants detected based on the eye movement data constructing a discrete event indicator function in a sampling domain consistent with the cerebral oxygen signal sequence; the discrete event indicator function construction formula is: ; wherein, is a discrete event indicator function; is a discrete sampling index; is a discrete sampling index corresponding to the m-th eye movement physiological event; is a discrete sampling index corresponding to the m-th eye movement physiological event; is the width of the indication pulse in the sample domain; is a rectangular window, satisfying is 1 when is 0 otherwise; The discrete event indicator function Brain oxygen signal sequence compared with near-infrared functional imaging Within a preset time window, mean removal and normalization are performed, the normalized cross-correlation between the two is calculated, and the correlation peak is searched within a physiologically reasonable set of second-level delays to obtain integer sample-level latency; the mean removal calculation formula is: The normalization formula is as follows: The normalized cross-correlation formula is as follows: , The formula for searching a set with a second-level delay is: ;in, This is a discrete-time sequence of brain oxygenation signals; and The mean within the window; and This is the sequence after removing the mean; This is the normalized cross-correlation coefficient; To prevent small positive numbers with a denominator of zero; For sample-level delay; For in set Internal envoy Maximum integer sample level latency; For a search set with a delay of seconds; The sampling frequency; and These correspond to the boundaries of negative two seconds and eight seconds in the sample domain, respectively; The sample length for the time window; Starting index of the window right The cross-correlation peaks at the specified points are refined using parabolic interpolation to obtain individualized brain oxygen delays. Time compensation is then applied to the brain oxygen signal sequence based on these individualized delays to achieve fine alignment, generating time-aligned multimodal data segments. The delay refinement formula is as follows: ;in, The individualized brain oxygen delay obtained by parabolic interpolation refinement is used to analyze brain oxygen signal sequences. Perform fractional sample shift compensation to obtain the discrete event indicator function. Alignment results on the same time base.

3. The intelligent dimming method based on multimodal physiological feedback according to claim 1, characterized in that, The eye-tracking indicators include fixation stability, saccade density, pupil diameter change rate, and blink frequency, which reflect the user's visual focus and transient fatigue level. The brain oxygenation indicators include changes in oxyhemoglobin concentration, changes in deoxyhemoglobin concentration, and the left-right brain oxygenation asymmetry index, which characterize cognitive load and neural activation level.

4. The intelligent dimming method based on multimodal physiological feedback according to claim 1, characterized in that, The state feature vector is input into the illumination adjustment decision model to obtain the illumination target parameters, including: A multi-branch regression model based on a neural network is established. The input of the multi-branch regression model is the state feature vector, and the output is the illumination target parameters. The illumination target parameters include at least the target light intensity, the target correlated color temperature, and the target spatial distribution ratio. The multi-branch regression model includes an input layer, several hidden layers, and three output branches corresponding to the target light intensity, the target correlated color temperature, and the target spatial distribution ratio. Each output branch regresses its own target after sharing features. Historical data containing task scenarios, physiological signals, and known preferred lighting parameters are collected, and a state feature vector for training is generated based on the historical data. The corresponding preferred light intensity, preferred correlated color temperature, and preferred spatial distribution ratio are used as supervision labels to form a training set and a validation set. The preferred parameters are determined by a combination of expert annotation, subject subjective scores, and objective performance indicators. The multi-branch regression model is trained using regression loss and the training set, and early stopping and hyperparameter fixing are performed using the validation set. After training is completed, the state feature vector is input into the trained multi-branch regression model to obtain the lighting target parameters.

5. The intelligent dimming method based on multimodal physiological feedback according to claim 1, characterized in that, The joint control instructions include: Light intensity control commands are used to set the luminous intensity of the lighting unit in order to adjust the lighting brightness; Related color temperature control commands are used to set the color temperature output of the lighting unit, so as to adjust the color temperature between cool light and warm light; Distributed control commands are used to set the spatial output ratio of lighting units to distribute luminous flux among multiple lighting areas or luminaires.

6. The intelligent dimming method based on multimodal physiological feedback according to claim 1, characterized in that, After the joint control command is executed, a short-term physiological response is collected, and the process returns to the step of simultaneously collecting eye-tracking data and near-infrared functional imaging brain oxygenation signals in the task scenario, running in a loop to ensure that the lighting environment and the user's physiological state maintain a real-time closed-loop match, including: After the lighting system completes the joint adjustment of light intensity, correlated color temperature and spatial distribution, a physiological monitoring trigger signal is activated to indicate the start of a new acquisition cycle. Within the adjusted preset time window, the user's eye movement data and near-infrared brain oxygenation signal are collected simultaneously to reflect the immediate response of lighting changes to visual and neural states. The collected short-term physiological data are subjected to time truncation and artifact removal processing to ensure signal continuity and temporal consistency with the data from the previous collection cycle, thereby obtaining short-term physiological response data fragments. The short-term physiological response data fragments are used as new eye-tracking data and near-infrared functional imaging brain oxygen signals. The eye-tracking data and near-infrared functional imaging brain oxygen signals are then collected synchronously in the task scenario and used for multimodal data fusion and state assessment in the next cycle, so as to achieve real-time closed-loop matching between illumination regulation and physiological state.

7. The intelligent dimming method based on multimodal physiological feedback according to claim 6, characterized in that, The preset time window is 10 seconds.

8. A smart dimming system based on multimodal physiological feedback, characterized in that, For implementing the intelligent dimming method as described in any one of claims 1 to 7, the intelligent dimming system comprises: The multimodal data synchronous acquisition and time base alignment module is used to synchronously acquire eye-tracking data and near-infrared functional imaging brain oxygen signals in the task scenario. The physiological event sequence detected by eye-tracking data is used as the alignment anchor point. The individualized brain oxygen delay is estimated and compensated by maximizing the event alignment correlation to obtain time-aligned multimodal data segments. The state representation and individualized baseline update module is used to extract state indicators representing attention, cognitive load and fatigue from the multimodal data fragments, establish an individualized dynamic baseline when first used or when switching tasks, and update it online during operation to generate a normalized state feature vector. The illumination adjustment decision model module is used to input the state feature vector into the illumination adjustment decision model to obtain the illumination target parameters; The joint control execution module is used to generate joint control instructions for light intensity, correlated color temperature and spatial distribution based on the lighting target parameters; The closed-loop acquisition and feedback module is used to acquire short-term physiological responses after the joint control command is executed, and return to the step to synchronously acquire eye-tracking data and near-infrared functional imaging brain oxygenation signals in the task scenario, so as to keep the lighting environment and the user's physiological state in real-time closed-loop matching.

Images