Multi-sensor fusion visual fatigue assessment and light parameter dynamic adjustment method

Abstract

The application discloses a kind of multi-sensor fusion's visual fatigue assessment and light parameter dynamic adjustment method, by obtaining the eye image frame in target user face video stream and the ambient light intensity data of the environment where current display device is located, pixel-level edge detection is carried out to eye image frame, obtain eye contour feature, while ambient light intensity data is carried out multidimensional feature extraction, obtain ambient light distribution feature, and by the dimension alignment and correlation mapping of both, form multi-source fusion feature vector;Again carry out multi-layer convolution and pool processing, output initial fatigue state value, combine current eye time length data and physiological state sequence data collected by wearable device, carry out decision tree integration processing, determine comprehensive fatigue state assessment value, and calculate target backlight brightness value, generate backlight control instruction drive backlight module dynamic adjustment, significantly improve the accuracy of backlight regulation and eye comfort.

Description

Technical Field

[0001] This invention relates to the field of intelligent protection technology for visual health, and in particular discloses a method for visual fatigue assessment and dynamic adjustment of optical parameters through multi-sensor fusion. Background Technology

[0002] The widespread adoption of smart devices has exacerbated visual fatigue caused by prolonged screen time. Accurately assessing human fatigue levels and dynamically adapting to the screen's lighting environment is a core technological direction for preventing eye damage and protecting visual health. Research indicates that existing visual fatigue monitoring and screen dimming solutions mostly rely on independent analysis of single images or single lighting data. They lack in-depth analysis of eye image lighting inversion compensation, weighted fusion of eye diagrams and lighting features, and a multimodal fatigue assessment process integrating eye features, ambient lighting, screen time, and wearable physiological data. Furthermore, they cannot dynamically calculate and adapt backlight brightness based on real-time visual sensitivity to achieve adaptive dimming.

[0003] The shortcomings and corresponding consequences of existing technologies are as follows: First, they only analyze the eye image or ambient illuminance separately, without performing illumination compensation for the edge pixels of the eye in strong light and weak light scenes. Illumination interference will cause distortion in the extraction of the eye contour, resulting in large fluctuations and low reliability of fatigue detection results. Second, they do not quantify the correlation between eye features and ambient light, and use a simple splicing and fusion method, which cannot distinguish the contribution weight of the two types of features to fatigue judgment, resulting in poor multi-source information fusion effect. Third, the fatigue assessment dimension is single, relying only on the output of rough fatigue values ​​from eye images, without combining and classifying physiological time-series data such as eye use time and heart rate, making it difficult to objectively reflect the user's true comprehensive fatigue level. Fourth, the dimming logic uses a fixed illuminance threshold for one-size-fits-all control, without quantifying individual visual sensitivity according to the user's real-time fatigue state, and cannot match the personalized light adaptation needs of different groups and different time periods. Inappropriate brightness adjustment may aggravate eye fatigue, resulting in limited visual protection effect.

[0004] Therefore, how to integrate multi-dimensional data on eye vision, ambient light, and physiology to achieve high-precision comprehensive fatigue assessment, and dynamically adjust screen backlight parameters based on individual visual sensitivity, is a key technical challenge that urgently needs to be solved in the field of intelligent visual health protection. Summary of the Invention

[0005] This invention provides a method for visual fatigue assessment and dynamic adjustment of light parameters through multi-sensor fusion, aiming to solve industry pain points such as fixed backlighting easily aggravating visual fatigue, single image recognition being easily affected by ambient light interference, and insufficient accuracy of fatigue assessment relying solely on facial features.

[0006] This invention relates to a method for visual fatigue assessment and dynamic adjustment of optical parameters using multi-sensor fusion, comprising the following steps:

[0007] S100: Collect the facial video stream of the target user and the ambient light intensity data of the current display device environment. Perform pixel-level edge detection processing on the eye image frames in the facial video stream through the image edge feature extraction algorithm. Perform back illumination compensation operation on the detected edge pixels based on the ambient light intensity data to obtain the eye contour feature vector of the eye image frame. At the same time, use the illumination feature extraction model to extract multi-dimensional features from the ambient light intensity data to obtain the ambient light distribution features.

[0008] S200. The eye contour feature vector and the ambient light distribution feature are dimensionally aligned and stitched together using a multimodal feature stitching method. The Pearson correlation coefficient of the two types of features is quantified through statistical correlation operation. Based on the Pearson correlation coefficient, adaptive weighted stitching fusion is performed to establish the correlation mapping relationship between the eye contour feature vector and the ambient light distribution feature, and obtain the multi-source fusion feature vector.

[0009] S300: The multi-source fusion feature vector is processed by multi-layer convolution and pooling through a pre-trained fatigue state evaluation convolutional neural network model to extract deep fatigue representation information in the multi-source fusion feature vector and output the initial fatigue state value for the multi-source fusion feature vector.

[0010] S400: Obtain the target user's current eye usage time data and the physiological state sequence data collected by the wearable device. Use a comprehensive fatigue state classification model to perform decision tree integration processing on the initial fatigue state value, the current eye usage time data, and the physiological state sequence data to determine the target user's comprehensive fatigue state assessment value.

[0011] S500. Determine whether the comprehensive fatigue state assessment value is greater than the preset fatigue warning threshold. If the comprehensive fatigue state assessment value is greater than the preset fatigue warning threshold, then quantify the visual sensitivity based on the fluctuation characteristics of the ambient light intensity time series data and the comprehensive fatigue state assessment value, and retrieve the preset sensitivity-brightness weight mapping relationship based on the visual sensitivity to calculate the target backlight brightness value of the display screen of the current display device.

[0012] S600: Based on the target backlight brightness value, generate a backlight control command for the current display device, and perform illumination regulation operation on the backlight driver module of the current display device according to the backlight control command to adjust the backlight intensity of the display screen of the current display device.

[0013] Furthermore, step S100 specifically includes:

[0014] S110. Collect the target user's facial video stream and the ambient light intensity data of the environment where the display device is located. Perform equal-interval frame separation and sampling processing on the facial video stream to obtain eye image frames. Simultaneously collect the time-series illuminance data output by the ambient light sensor and simultaneously collect and cache the time-series illuminance data.

[0015] S120. Perform pixel-level grayscale gradient edge detection on the eye image frame, filter valid edge pixels by preset edge gradient threshold, remove invalid background pixels, and construct the initial edge pixel set corresponding to the eye image frame.

[0016] S130. Calculate statistics on the cached temporal illumination data to obtain the average ambient light intensity, illumination temporal standard deviation, and illumination peak deviation, forming an ambient light distribution feature vector; perform illumination inverse compensation operation on the initial edge pixel set based on the average ambient light intensity, dividing the eye image into... For each local region, the compensated mean value is calculated to form the eye contour feature vector.

[0017] Further, step S200 includes:

[0018] S210. Extract the original feature dimensions of the eye contour feature vector and the ambient light distribution feature respectively, and determine the dimensional parameters of the two types of heteromodal features.

[0019] S220. For eye contour features and ambient lighting distribution features with inconsistent dimensions, a linear interpolation method is used to perform a unified dimension alignment transformation to generate a fixed-dimensional eye feature matrix and lighting feature matrix, and the two are normalized to make them dimensionless.

[0020] S230. Quantify the coupling relationship between the eye feature matrix and the illumination feature matrix through statistical correlation operations, calculate the Pearson correlation coefficient between the two types of features, and construct a stable correlation mapping relationship between eye contour features and ambient illumination distribution features.

[0021] S240. Based on the Pearson correlation coefficient obtained by quantization, perform adaptive weighted splicing and fusion, dynamically allocate the fusion ratio of the two types of features according to the correlation weight, and generate a multi-source fusion feature vector with unified dimensions and correlation matching.

[0022] Furthermore, step S300 specifically includes:

[0023] S310. Call the pre-trained fatigue state assessment convolutional neural network model, use a fixed-size fatigue feature convolution kernel to perform point-by-point sliding convolution operation on the multi-source fusion feature vector, extract local fatigue detail features from the fusion features, and generate a local fatigue feature matrix.

[0024] S320. Perform multi-layer alternating convolution and max pooling downsampling on the local fatigue feature matrix to filter out redundant noise features, enhance deep fatigue semantic information, and extract high-dimensional deep fatigue characterization features.

[0025] S330. The deep fatigue characterization features are weighted and normalized through the fully connected weight layer of the convolutional neural network model, and the initial fatigue state value in the range of 0 to 1 is quantized and output.

[0026] Furthermore, step S400 specifically includes:

[0027] S410: Simultaneously collect real-time eye usage time data of the target user, as well as the continuous physiological state time series output by the wearable physiological monitoring device;

[0028] S420. Perform equal-interval uniform segmentation on the continuous physiological state time sequence to obtain multiple short-term physiological state sub-sequences, thereby achieving refined decomposition of physiological fluctuation characteristics.

[0029] S430. Perform mean quantization on each physiological state subsequence, collect standardized physiological feature parameters, and construct a physiological state feature vector with regular dimensions.

[0030] S440. Perform Min-Max normalization on the real-time eye usage duration data and physiological state feature vector so that each component is in the [0, 1] interval and is dimensionless; perform feature fusion on the initial fatigue state value, the normalized eye usage duration data, and the normalized physiological state feature vector to construct a 7-dimensional fatigue feature vector containing multi-dimensional fatigue parameters.

[0031] S450. Preset safe eye use time threshold. When the eye use time parameter in the 7-dimensional fatigue feature vector exceeds the preset safe eye use time threshold, the comprehensive fatigue state classification model is activated. The 7-dimensional fatigue feature vector is analyzed through decision tree integration operation to quantify and determine the comprehensive fatigue state assessment value of the target user.

[0032] Further, step S500 includes:

[0033] S510, Preset fatigue warning threshold, compare the comprehensive fatigue state assessment value with the preset fatigue warning threshold, and output a binary fatigue warning indicator.

[0034] S520. When it is determined that a fatigue warning has been triggered, retrieve the ambient light intensity time series data within a preset time period and construct an ambient light intensity time series sequence.

[0035] S530. Extract the fluctuation characteristic parameters of the ambient light intensity time series, combine them with the comprehensive fatigue state assessment value to perform coupling operation, and quantify the visual sensitivity metric of the target user.

[0036] S540. Normalize the visual sensitivity quantification value to [0, 1], and then retrieve the preset sensitivity-brightness weight mapping relationship based on the normalized visual sensitivity to obtain the corresponding initial backlight brightness adjustment weight.

[0037] S550 combines the current backlight brightness value of the current display device with the initial backlight brightness adjustment weight and the comprehensive fatigue state assessment value, performs adaptive brightness calculation, and solves for the target backlight brightness value of the display screen that meets the needs of human eye protection.

[0038] Further, step S600 includes:

[0039] S610. Based on the current backlight brightness value and the target backlight brightness value, a linear interpolation algorithm is used to generate a multi-frame continuous brightness smooth transition sequence to avoid the visual impact caused by instantaneous brightness changes.

[0040] S620: Based on the preset linear mapping relationship between screen brightness and PWM duty cycle, the brightness smooth transition sequence is converted frame by frame to generate the corresponding PWM duty cycle sequence;

[0041] S630: Based on the hardware response delay parameters of the current display device, the basic control cycle is compensated and corrected to generate a backlight instruction execution cycle that adapts to the hardware response characteristics. Based on the backlight instruction execution cycle and the PWM duty cycle sequence, a backlight control instruction is generated.

[0042] S640: Extract the driving current amplitude and instruction execution cycle according to the backlight control instruction, and perform illumination control operation on the backlight driving module of the current display device according to the driving current amplitude and instruction execution cycle to adjust the backlight intensity of the display screen of the current display device.

[0043] The beneficial effects achieved by this invention are as follows:

[0044] This invention provides a multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of light parameters. Addressing the problem that existing display device backlight adjustments are difficult to accurately adapt to the user's actual fatigue state, this invention acquires eye image frames from the target user's facial video stream and ambient light intensity data of the current display device's environment. It uses an image edge feature extraction algorithm to perform pixel-level edge detection on the eye image frames to obtain eye contour features. Simultaneously, it employs an illumination feature extraction model to extract multi-dimensional features from the ambient light intensity data to obtain ambient light distribution features. Finally, it uses a multi-modal feature stitching method to align and correlate the two data, forming a multi-source fused feature vector. The system first measures the amount of fatigue; then, using a pre-trained fatigue state assessment convolutional neural network, it performs multi-layer convolution and pooling processing on the multi-source fusion feature vector to output an initial fatigue state value. Combining the current eye usage time data and the physiological state sequence data collected by the wearable device, it performs decision tree ensemble processing through a comprehensive fatigue state classification model to determine the comprehensive fatigue state assessment value. Based on the pre-established fatigue degree and brightness adjustment mapping relationship, it calculates the target backlight brightness value and generates backlight control commands to drive the backlight module to dynamically adjust. This achieves deep coupling analysis of eye features, ambient light, and physiological state, significantly improving the accuracy of backlight control and eye comfort. The specific beneficial effects achieved are as follows:

[0045] 1. This invention simultaneously collects user facial video data and ambient lighting data. By combining pixel-level eye edge detection with backlight compensation calculation, it eliminates the interference of ambient lighting intensity changes on eye feature extraction, ensuring the stability and accuracy of eye contour feature extraction under different lighting conditions. At the same time, it independently mines the multi-dimensional distribution features of ambient lighting, realizing refined perception of dual-dimensional information of human eye state and external light environment, providing complete basic feature support for subsequent fatigue assessment.

[0046] 2. This invention establishes a precise correlation mapping relationship between eye fatigue features and ambient light features by aligning multiple modal dimensions and splicing features, and by relying on the Pearson correlation coefficient to achieve adaptive weighted fusion. This solves the problem that traditional visual fatigue detection only extracts eye features and ignores the coupling interference of light environment. It deeply explores the implicit impact of light changes on visual fatigue state and improves the rationality and pertinence of multi-source feature fusion.

[0047] 3. This invention utilizes a pre-trained convolutional neural network model to perform deep convolutional pooling operations on fused features, fully mining implicit fatigue details that cannot be represented by shallow image features. It overcomes the limitations of traditional threshold judgment methods, which have shallow feature mining and single fatigue representation, and realizes refined initial fatigue state recognition based on multi-source fusion features, thereby improving the accuracy of initial fatigue judgment.

[0048] 4. This invention integrates eye usage time, wearable physiological sequence data, and initial fatigue value, and completes multi-dimensional information joint decision-making through a decision tree ensemble model. It breaks through the one-sidedness of traditional methods that rely solely on visual images to determine fatigue, and takes into account eye appearance characteristics, the cumulative effect of eye usage time, and the actual physiological state of the human body, so as to achieve a comprehensive and objective assessment of the user's overall fatigue state.

[0049] 5. This invention accurately distinguishes between normal eye use and fatigued state by using fatigue threshold. Based on the identification of visual fatigue conditions, it combines the characteristics of light time-series fluctuations and fatigue level to quantify the user's visual sensitivity. It also adaptively matches the screen target backlight brightness based on a preset weight mapping relationship, realizing dynamic adaptation of brightness adjustment to the user's real-time visual state and changes in ambient light. This avoids the shortcomings of fixed backlight parameters being unable to adapt to complex eye use scenarios.

[0050] 6. This invention generates control commands based on precisely calculated target backlight brightness, dynamically adjusting the backlight drive module of the display device in a closed loop. This allows for real-time correction of screen luminous intensity to adapt to human eye fatigue and changes in ambient light, effectively alleviating visual fatigue from prolonged screen use and reducing eye damage caused by strong light stimulation and uneven lighting. This invention achieves intelligent control throughout the entire process, including environmental perception, multimodal feature fusion, intelligent fatigue assessment, and dynamic adjustment of light parameters. It is adaptable to various complex indoor and outdoor lighting scenarios and prolonged screen use conditions. The fatigue assessment is comprehensive, and the adjustment strategy aligns with the human visual mechanism, demonstrating a high degree of intelligence and strong practical applicability. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating an embodiment of the multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters according to the present invention.

[0052] Figure 2 This is a detailed flowchart of an embodiment of step S100 in the multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters of the present invention. Detailed Implementation

[0053] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0054] like Figure 1As shown, the first embodiment of this invention proposes a multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of light parameters. This method is applicable to various display terminals such as computer monitors, mobile phones, tablets, and industrial monitoring screens. It relies on a front-facing camera, ambient light sensor, and wearable physiological sensor to collaboratively collect data, enabling intelligent assessment of human eye fatigue and automatic adaptation and adjustment of screen backlight brightness. Through features extraction for illumination compensation in eye images, adaptive fusion of multi-modal feature correlations, deep fatigue feature mining using CNNs, comprehensive fatigue assessment integrating eye usage time and physiological time series, and quantitative adaptive backlight dimming based on human visual sensitivity, the method achieves intelligent dynamic adjustment of light parameters for the display terminal based on the user's real-time fatigue state and ambient light, reducing visual fatigue damage. The method includes the following steps:

[0055] Step S100: Collect the facial video stream of the target user and the ambient light intensity data of the current display device. Perform pixel-level edge detection processing on the eye image frames in the facial video stream using an image edge feature extraction algorithm. Perform backlight compensation operation on the detected edge pixels based on the ambient light intensity data to obtain the eye contour feature vector of the eye image frame. At the same time, use the illumination feature extraction model to extract multi-dimensional features from the ambient light intensity data to obtain the ambient light distribution features.

[0056] The system simultaneously activates the front-facing image sensor and onboard ambient light sensor of the display device to collect data in parallel: the camera continuously outputs a real-time facial video stream of the target user, while the ambient light sensor collects high-frequency temporal data of ambient light intensity around the device; frame-by-frame, image frames of the region of interest around the eyes are cropped from the facial video stream, and pixel-level eye contour edge detection is completed using an image edge feature extraction algorithm; an illumination compensation coefficient is constructed using the synchronously collected ambient light intensity values, and an inverse illumination compensation operation is performed on the grayscale values ​​of all edge pixels to eliminate edge distortion caused by overexposure in strong light and underexposure in dark light, and a fixed-dimensional eye contour feature vector is generated after normalization; the continuous temporal ambient light intensity data is input into a pre-trained illumination feature extraction model, and multi-dimensional indicators such as illumination mean, illumination variance, illumination flicker amplitude, and brightness gradient are extracted and integrated to obtain standardized ambient light distribution features.

[0057] The facial video stream is a continuous temporal data stream of images containing the human eyes and facial regions, output in real time by the device's front-facing camera; the ambient light intensity data is a temporal value of the real-time illuminance of the environment around the screen collected by the ambient light sensor; pixel-level edge detection is a fine edge extraction operation that traverses the eye image pixel by pixel to identify the contours of the eye fissure, eyelids, and eyeballs; the illumination inverse compensation operation is a pixel correction algorithm that corrects the grayscale of the edge pixels of the eye according to the ambient illuminance to offset the image feature distortion caused by strong / weak ambient light; the eye contour feature vector is a one-dimensional standardized feature vector that represents eye fatigue-related information such as the width of the eye fissure, the blink amplitude, and the degree of eyelid drooping; the ambient light distribution feature is a multi-dimensional feature composed of the light mean, fluctuation variance, flicker, and brightness gradient, representing the comprehensive stimulation degree of ambient light on the human eye.

[0058] Step S200: The eye contour feature vector and the ambient light distribution feature are dimensionally aligned and stitched together using a multimodal feature stitching method. The Pearson correlation coefficient of the two types of features is quantified by statistical correlation operation. Based on the Pearson correlation coefficient, adaptive weighted stitching fusion is performed to establish the correlation mapping relationship between the eye contour feature vector and the ambient light distribution feature, and obtain the multi-source fusion feature vector.

[0059] Dimension alignment is achieved by performing extreme value normalization scaling on eye contour feature vectors and ambient light distribution features with different dimensions. The two types of features are then concatenated into a basic concatenated feature set. Statistical correlation operations are performed on the eye feature sequence and the illumination feature sequence under synchronous time sequence to solve for the Pearson correlation coefficient, which characterizes the coupling strength of the two types of features. Using the Pearson correlation coefficient as a dynamic weight, fusion weights are assigned to the eye contour features and ambient light distribution features respectively to complete adaptive weighted concatenation fusion, establishing a one-to-one correlation mapping relationship between eye fatigue features and the degree of ambient light stimulation. Finally, a multi-source fusion feature vector that integrates both human eye state and ambient light interference information is generated.

[0060] Multimodal dimension alignment is a standardized preprocessing method that scales image-related eye features and sensor-related illumination features to a unified value range and matches feature dimensions. The Pearson correlation coefficient, with a value of [-1, 1], is a statistical indicator that quantifies the strength of the linear coupling between fluctuations in eye fatigue features and fluctuations in ambient illumination. Adaptive weighted stitching fusion is a feature fusion strategy that dynamically allocates fusion weights based on the real-time correlation between the two types of features, increasing the weight of illumination features when illumination interference is strong and increasing the weight of eye features when the image is distorted in low light. The multi-source fusion feature vector is an integrated standardized feature that integrates eye fatigue information and ambient illumination interference information and is adapted to the input of the fatigue recognition model.

[0061] Step S300: Perform multi-layer convolution and pooling processing on the multi-source fusion feature vector using a pre-trained fatigue state evaluation convolutional neural network model, extract deep fatigue representation information from the multi-source fusion feature vector, and output the initial fatigue state value for the multi-source fusion feature vector.

[0062] The multi-source fusion feature vector generated in step S200 is input into a fatigue state assessment convolutional neural network that has been trained and converged based on a large number of human eye-lighting samples. The network sequentially performs multi-layer convolutional kernel feature extraction and pooling dimensionality reduction operations to mine deep fatigue latent features that cannot be represented by shallow contours, including continuous squinting features, progressive eyelid drooping features, high-frequency blinking time sequence features, and eye fatigue superposition features under strong light stimulation. The fully connected layer integrates all deep fatigue representation information and maps and outputs an initial fatigue state value that is continuously quantized in the range of 0 to 1, which only represents the basic fatigue degree based on eye and ambient light image sensing.

[0063] The fatigue state assessment convolutional neural network model is a CNN network trained offline based on multi-source fusion feature samples to mine deep fatigue features of the eye. The deep fatigue representation information is the subtle, long-term latent fatigue features extracted by multi-layer convolution, which cannot be directly observed through the edge contour of a single frame. The initial fatigue state value is a basic fatigue quantification value obtained only by image and ambient light sensing data, with a value range of [0, 1]. The larger the value, the more severe the visual fatigue of the eye.

[0064] Step S400: Obtain the target user's current eye usage time data and the physiological state sequence data collected by the wearable device. Use a comprehensive fatigue state classification model to perform decision tree integration processing on the initial fatigue state value, the current eye usage time data, and the physiological state sequence data to determine the target user's comprehensive fatigue state assessment value.

[0065] The system reads the cumulative runtime of the display device's system timing module to obtain the user's continuous current eye usage time data; it synchronously collects the physiological state time series output by the wristband and glasses-type wearable sensors via Bluetooth / wireless communication, including heart rate, blood oxygen, and blink frequency; it standardizes and inputs the three multi-dimensional indicators—initial fatigue state value, eye usage time, and physiological state sequence—into a comprehensive fatigue state classification model; the model employs a parallel ensemble learning mechanism using multiple decision trees to separately determine three types of fatigue causes: visual fatigue, cumulative fatigue from prolonged eye use, and physiological drowsiness; it then weights and fuses the output branches of each decision tree to eliminate the bias of single-image evaluation and output a comprehensive fatigue state assessment value that integrates multi-dimensional information.

[0066] The current eye usage time data is the continuous cumulative time that users spend watching the display screen; the physiological state sequence data is the continuous time-series physiological parameters of heart rate, blood oxygen, and eye micro-movements collected by wearable sensors; the decision tree ensemble processing is an ensemble learning algorithm that evaluates a single fatigue dimension separately by multiple independent decision trees and merges the branch results to improve the robustness of the evaluation; the comprehensive fatigue state assessment value is the final standardized fatigue quantification index that integrates eye images, ambient light, eye usage time, and human physiological multi-source data.

[0067] Step S500: Determine whether the comprehensive fatigue state assessment value is greater than the preset fatigue warning threshold. If the comprehensive fatigue state assessment value is greater than the preset fatigue warning threshold, then visual sensitivity is quantified based on the fluctuation characteristics of the ambient light intensity time series data and the comprehensive fatigue state assessment value. Based on the visual sensitivity, the preset sensitivity-brightness weight mapping relationship is retrieved, and the target backlight brightness value of the display screen of the current display device is calculated.

[0068] The system retrieves a preset fatigue warning threshold and compares the real-time comprehensive fatigue assessment value with the preset threshold. When the comprehensive fatigue assessment value exceeds the preset threshold, it determines that the user has developed significant visual fatigue and initiates the backlight adaptive dimming logic. The mean and variance of ambient light intensity over time are extracted as light fluctuation features and coupled with the comprehensive fatigue assessment value for weighted calculation to quantify and output the current visual sensitivity of the human eye (the higher the fatigue and the greater the strong light fluctuation, the higher the visual sensitivity value and the more sensitive the human eye is to screen brightness stimulation). The system retrieves a locally pre-stored sensitivity-brightness weight mapping table calibrated based on human visual physiological experiments, matches the dimming weight coefficient corresponding to the current visual sensitivity, and calculates the target backlight brightness value of the screen that is suitable for the human eye's tolerance level by weighted calculation combined with the ambient baseline illuminance.

[0069] The fatigue warning threshold is a critical assessment value that distinguishes between no obvious fatigue and moderate visual fatigue, serving as the benchmark for dimming activation. Visual sensitivity is a dimensionless index that quantifies the human eye's tolerance to screen brightness stimuli under fatigue conditions; the higher the sensitivity, the lower the required screen backlight brightness. The sensitivity-brightness weight mapping relationship is a lookup table mapping rule that corresponds one-to-one between offline calibrated visual sensitivity and backlight dimming correction coefficients. The target backlight brightness value is the optimal backlight output illuminance value calculated by taking into account ambient light and the user's real-time fatigue level.

[0070] Step S600: Generate a backlight control command for the current display device based on the target backlight brightness value, and perform illumination adjustment operation on the backlight driver module of the current display device according to the backlight control command to adjust the backlight intensity of the display screen of the current display device.

[0071] Based on the screen backlight driver chip communication protocol, the calculated target backlight brightness value is converted into a standardized PWM duty cycle format backlight control instruction. The instruction includes the target brightness value, smooth dimming step size, and dimming duration constraint parameters. The encrypted and verified backlight control instruction is sent to the display device's backlight driver module. The driver module adjusts the backlight PWM output duty cycle piecewise linearly according to the instruction, smoothly changing the LED backlight luminous power, and completing the adaptive downward / upward adjustment of the screen backlight luminous intensity to match the user's real-time visual fatigue and ambient lighting conditions, thereby alleviating visual stimulation and reducing the rate of fatigue aggravation.

[0072] The backlight control command is a standardized control message that adapts to the communication protocol of the backlight driver module and includes target brightness and smooth dimming parameters; the backlight driver module is the hardware driver unit for powering the screen LED backlight and PWM dimming; the backlight luminous intensity is the overall illuminance output of the screen backlight LED, which is directly controlled by the duty cycle of the driving PWM.

[0073] Further, see Figure 2 The multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment includes step S100, which specifically includes:

[0074] Step S110: Collect the target user's facial video stream and the ambient light intensity data of the environment where the display device is located. Perform equal-interval frame separation and sampling processing on the facial video stream to obtain eye image frames. Simultaneously collect the time-series illuminance data output by the ambient light sensor and simultaneously collect and cache the time-series illuminance data.

[0075] The system synchronously acquires user facial video streams and ambient light sensor temporal illuminance data; performs equally spaced frame separation sampling on the facial video stream, outputting a standardized eye image frame sequence; and defines a temporally synchronized and aligned original illuminance quantization benchmark to constrain the range of original illuminance acquisition values. This enables the matching of facial visual features with lighting time-series data timestamps, providing standardized synchronous input data for multi-sensor fusion fatigue assessment.

[0076] The facial video stream is subjected to equal-interval frame separation to obtain facial image frames. The facial region is located by face detection algorithm (such as MTCNN, RetinaFace), and the left and right eye key points are located based on facial key point detection (such as Dlib 68-point model, MediaPipeFaceMesh 468-point model). The eye image frames are then cropped according to the preset bounding box size.

[0077] The quantization values ​​for time-series illuminance data are defined by the following formula:

[0078] (1)

[0079] In formula (1), for The raw ambient light intensity collected by the time sensor; the unit of measurement is lux (lux). Illuminance is a standard unit of measurement; the range of values ​​is a closed interval. Covering all lighting conditions in all scenarios: This indicates a completely dark, enclosed environment; Indicates general lighting for offices and indoor homes; This indicates a bright, sunny outdoor environment with direct sunlight near the window. The sampling time is synchronized with the sampling timestamp of the facial video frame to achieve timeline matching of facial fatigue features and ambient light data. Formula (1) defines the original ambient light intensity collected by the sensor at any given time. The value ranges from 0 to 10000 lux and is used as the raw temporal illumination data acquired synchronously with facial video for subsequent multi-sensor fusion visual fatigue assessment.

[0080] Step S120: Perform pixel-level grayscale gradient edge detection on the eye image frame, filter valid edge pixels by preset edge gradient threshold, remove invalid background pixels, and construct the initial edge pixel set corresponding to the eye image frame.

[0081] The horizontal and vertical gray-level partial derivatives of the sampled eye image frames are calculated pixel by pixel to synthesize a two-dimensional gray-level gradient magnitude to characterize the intensity of pixel brightness abrupt changes. A preset edge gradient threshold is used for binary discrimination to mark the effective edge pixels of the eye contour with a pixel gray-level gradient magnitude higher than the preset edge gradient threshold. Invalid pixels with low gradient smoothing background are removed to generate an initial set of eye edge pixels, providing an accurate contour data source for subsequent extraction of eye geometric fatigue features.

[0082] The calculation for pixel edge validity is shown in the following formula:

[0083] (2)

[0084] In formula (2), coordinates Grayscale values ​​of the eye image at pixel locations, 8-bit image value range Dimensionless; The horizontal gray-level partial derivative of a pixel represents the rate of change of gray-level in the image. The vertical gray-level partial derivative of a pixel represents the rate of change of gray-level in the vertical direction of the image. This represents the pixel grayscale gradient magnitude, measured in grayscale per pixel; it characterizes the intensity of the change in brightness at that location, with a larger value indicating a clearer outline edge. The preset edge gradient threshold is set to 20 in this embodiment, with the unit being grayscale / pixel; it is used to distinguish the eye contour from the smooth background. Valid identifier for pixel edges, binary and dimensionless: This indicates that a pixel with a grayscale gradient magnitude greater than or equal to a preset edge gradient threshold is determined to be a valid eye contour edge pixel. If the pixel grayscale gradient magnitude is less than the preset edge gradient threshold, it is determined to be an invalid pixel in the smooth background and is directly removed. Formula (2) first calculates the two-dimensional grayscale gradient magnitude of each pixel in the image to represent the degree of brightness change, and then completes the binary judgment through the preset edge gradient threshold, marking the effective edge pixels of the eye and the invalid background pixels, and quickly generating a set of edge pixels of the eye contour for subsequent extraction of geometric features of visual fatigue.

[0085] Step S130: Calculate statistics on the cached temporal illuminance data to obtain the average ambient light intensity, the temporal standard deviation of illumination, and the peak illumination deviation, thus constructing an ambient light distribution feature vector; perform illumination inverse compensation operation on the initial edge pixel set based on the average ambient light intensity to divide the eye image into... For each local region, the compensated mean value is calculated to form the eye contour feature vector.

[0086] Retrieve cached temporal illumination window data, and sequentially calculate the window's average illumination intensity, illumination temporal standard deviation, and illumination peak deviation. These three are then concatenated to generate a three-dimensional ambient illumination distribution feature vector. Using the window's average illumination as the compensation benchmark, the eye image is equally divided into four local regions. Illumination inverse grayscale compensation is performed only on the effective eye edge pixels selected in step S120 within each region. After non-negative truncation to eliminate invalid negative grayscale values, the mean value of the region pixels is calculated. The four quadrant compensation mean values ​​are combined to output a four-dimensional illumination robust eye contour feature vector. Simultaneously, illumination features and anti-illumination distortion eye contour features are output as dual-path inputs to the multi-sensor fusion fatigue assessment model.

[0087] The statistic is calculated using the following formula:

[0088] (3)

[0089] In formula (3), The average ambient light intensity is expressed in units of... , as the baseline quantity for illumination compensation; The standard deviation of the illumination time series characterizes the degree of drastic fluctuation in ambient illumination. The peak light deviation represents the instantaneous impact amplitude of strong light / dark light. The characteristic vector of ambient light distribution is represented by the average ambient light intensity. Standard deviation of illumination time series Peak illumination deviation It consists of three components, used to characterize the steady-state level, the degree of fluctuation, and the instantaneous impact amplitude of ambient light; This represents the total number of sampling points in the illumination sampling time window.

[0090] The eye contour feature vector is constructed using the following formula:

[0091] (4)

[0092] In formula (4), For image region subscripts, This means that the eye image is divided equally into four local regions: upper left, upper right, lower left, and lower right. For the first Satisfying within quadrant The total number of effective eye edge pixels; To only traverse the first Within the region, the effective eye contour pixels selected in step S120 are excluded from the calculation; background pixels are not included in the calculation. The original grayscale value of the pixel, with a value range of ; The standard reference illuminance is based on the standard value of illuminance for general office work surfaces as specified in GB 50034-2013 "Standard for Lighting Design of Buildings"; This is the ratio of the current average illumination to the reference illumination. The light compensation attenuation coefficient is dimensionless and controls the intensity of grayscale suppression under strong light. This is a strong light grayscale compensation suppression term; the stronger the ambient light, the greater the suppression value. It is a non-negative truncation operator; if the grayscale value after compensation is less than 0, it is directly set to 0 to avoid negative grayscale values ​​that have no physical meaning; the whole fraction is used. The arithmetic mean of gray levels after compensation of effective edge pixels in the region; For the first Quadrant illumination-compensated eye contour feature values; the four-dimensional eye contour feature vector is obtained by concatenating the numerical values ​​of the four regions. Formulas (3)-(4) first calculate the mean, standard deviation, and peak deviation of the time-series illumination data to obtain the ambient illumination features; then divide the eye image into four equal parts, introduce average illumination only on the effective edge pixels for grayscale inverse compensation, take the mean of the region after non-negative truncation, and splice them to obtain a four-dimensional illumination-independent eye contour feature vector, which is used as input for the visual fatigue fusion evaluation model.

[0093] The multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of light parameters provided in this embodiment, in step S100, firstly, a facial video stream of the target user is acquired through a camera, and simultaneously, ambient light intensity data of the environment where the display device is located is acquired through an ambient light sensor (such as BH1750, TSL2591, VEML7700, APDS-9960, OPT3001, etc., I²C interface digital light sensors). The facial video stream is sampled by equal-interval frame separation to obtain a standardized eye image frame sequence. The time-series illuminance data output by the ambient light sensor is recorded, and its quantization value is shown in formula (1), with a value range of 0~10000. .

[0094] Next, pixel-level grayscale gradient edge detection is performed on the eye image frame. The gradient magnitude of each pixel is calculated using formula (2) and compared with a preset edge gradient threshold. By comparison, valid edge pixels are selected and background pixels are removed.

[0095] Then, a statistic was calculated from the time-series data of ambient light: average ambient light intensity. Standard deviation of illumination time series Peak illumination deviation This constitutes the characteristic vector of ambient light distribution. Simultaneously, the eye image is divided into four quadrants, and for the effective edge pixels in each quadrant, illumination inversion compensation is performed according to formula (4): subtracting the strong light grayscale compensation suppression term that is proportional to the average illumination ( The non-negative mean is taken to obtain the four-dimensional eye contour feature vector. .in This is a standard value for illuminance in a typical indoor office environment, used to normalize the average ambient light to a range comparable to the image grayscale value. The light compensation attenuation coefficient was determined experimentally using 50 subjects under three lighting conditions (dark light 50...). Normal 500 3000 strong light Under the condition that the variance of the compensated eye contour features is minimized, the result is obtained.

[0096] Preferably, the multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment includes step S200 as follows:

[0097] Step S210: Extract the original feature dimensions of the eye contour feature vector and the ambient light distribution feature respectively, and determine the dimensional parameters of the two types of heteromodal features.

[0098] The four-dimensional eye contour feature vector and the three-dimensional ambient light distribution feature vector output in step S130 are analyzed respectively. The number of original feature components of each of the two types of heteromodal features is read, the dimension parameters of the eye contour feature and the dimension parameters of the ambient light feature are locked, and the dimension quantization and calibration of the dual-path heterogeneous feature is completed, providing fixed dimension benchmark parameters for subsequent multimodal feature dimension alignment and fusion mapping.

[0099] Step S220: For eye contour features and ambient lighting distribution features with inconsistent dimensions, perform a uniform dimension alignment transformation using linear interpolation to generate a fixed-dimensional eye feature matrix and lighting feature matrix, and perform normalization processing on both to make them dimensionless.

[0100] To address the dimensionality mismatch between 4D eye contour features and 3D ambient lighting features, a linear interpolation resize operator is used to upsample and align the two types of heteromodal features, uniformly mapping them to a preset fixed dimension. We obtain eye-aligned feature vectors and illumination-aligned feature vectors of equal length; then we perform independent normalization scaling based on their respective physical value ranges, converting them all into dimensionless standardized features of [0, 1], eliminating modal dimension differences and dimensional differences, and providing standardized input data for subsequent multi-feature coupling and fusion.

[0101] The dimension alignment transformation operation is shown in the following formula:

[0102] (5)

[0103] In formula (5), The original eye contour feature vector is 4-dimensional and is derived from the output of equation (3); The original ambient light distribution feature vector is 3-dimensional and comes from the light statistical calculation in step S130. To ensure a globally consistent fixed feature dimension, the value is fixed at 32; This is a linear interpolation dimension alignment operator that performs linear interpolation upsampling on the input low-dimensional vector, mapping it to a fixed 32-dimensional length. The standardized eye feature matrix (32-dimensional vector) after dimension alignment. The standardized illumination feature matrix (32-dimensional vector) is obtained after dimension alignment. Formula (5) stretches the 4-dimensional eye features and 3-dimensional illumination features to a fixed 32 dimensions by using the linear interpolation Resize operator; then, it is normalized and scaled to the dimensionless interval [0, 1] according to the upper limit of their respective physical ranges to generate standardized feature vectors of the same dimension, which are used as fusion inputs for the fatigue assessment model.

[0104] Step S230: Quantify the coupling relationship between the eye feature matrix and the illumination feature matrix through statistical correlation operations, calculate the Pearson correlation coefficient between the two types of features, and construct a stable correlation mapping relationship between eye contour features and ambient illumination distribution features.

[0105] Using the 32-dimensional eye feature matrix and illumination feature matrix after dimension alignment and normalization as paired samples, the Pearson correlation formula is used to calculate the linear correlation coefficient between the two types of heteromodal features, quantifying the coupling strength between ambient illumination and eye contour grayscale. The absolute value of the negative correlation coefficient is taken to generate unsigned correlation weights, and the coupling strength is used as the dynamic weighting coefficient to establish a quantitative correlation mapping between eye visual features and illumination sensing features, providing an adaptive weight benchmark for multi-feature weighted fusion.

[0106] The Pearson correlation coefficient is calculated using the following formula:

[0107] (6)

[0108] In formula (6), The 32-dimensional standardized eye contour feature vector output by formula (5); The 32-dimensional normalized ambient lighting feature vector output by formula (5); As a covariance operator, using 32 dimensions as paired samples, the overall linear and coordinated change trend of eye features and illumination features is calculated; , These are the variances of eye features and illumination features, respectively, representing the degree of discrete fluctuation of a single feature. The standard deviation is the square root of the variance to eliminate the dimension of square. The Pearson linear correlation coefficient has a range of values. Dimensionless: This indicates a positive correlation; the stronger the light, the higher the grayscale of the eye contour. This indicates a negative correlation; the stronger the illumination, the lower the gray level of the eye contour. The closer the absolute value is to 1, the stronger the linear coupling between the two types of features; the closer it is to 0, the weaker the coupling. The association weight rule is to take the absolute value if the coefficient is negative. As a fusion weight, only the coupling strength is retained and the positive and negative correlation directions are discarded. Formula (6) uses a unified 32-dimensional standardized feature vector as paired samples to calculate the Pearson correlation coefficient between eye contour features and ambient lighting features to characterize the linear coupling strength between the two; when the coefficient is negative, the absolute value is taken as the feature fusion weight, which is used to dynamically balance the contribution ratio of the two types of features in the fatigue assessment model.

[0109] Step S240: Based on the Pearson correlation coefficient obtained by quantization, perform adaptive weighted splicing and fusion, dynamically allocate the fusion ratio of the two types of features according to the correlation weight, and generate a multi-source fusion feature vector with unified dimensions and correlation matching.

[0110] The absolute value of the Pearson correlation coefficient is used as an adaptive weighting coefficient to perform complementary weighted scaling on the 32-dimensional standardized eye features and the 32-dimensional standardized illumination features. The two weighted features of equal length are horizontally concatenated and transposed to output a 64-dimensional unified dimensionless multi-source fusion feature vector. The contribution ratio of the two types of features is dynamically allocated according to the coupling strength between illumination and eye contour, weakening the interference of illumination distortion and generating standardized fusion input features adapted to the fatigue assessment network.

[0111] The weighted fusion operation of multi-source fusion feature vectors is shown in the following formula:

[0112] (7)

[0113] In formula (7), The absolute value of the Pearson correlation coefficient output by formula (6), the dynamic weight of feature fusion, and the range are given. The larger the value, the stronger the coupling interference between lighting and eye contour, thus increasing the weight of eye physiological features; the smaller the value, the weaker the lighting interference, thus increasing the weight of ambient lighting features for auxiliary correction. Output a 32-dimensional standardized eye contour feature vector for step S220; Output a 32-dimensional normalized ambient lighting feature vector for step S220; For the weighted eye feature branch, each dimension element is simultaneously multiplied by the associated weight; The weighted illumination feature branch is summed with the complementary weights of the eye weights, and the sum is always equal to 1. To transpose the horizontally concatenated data into a column vector, the weighted 32-dimensional eye and 32-dimensional illumination features are concatenated. The final 64-dimensional multi-source fusion feature vector remains dimensionless and retains its range. As the sole input to the fatigue assessment network, Formula (7) uses the absolute value of the Pearson correlation coefficient as the adaptive weight to scale the eye and illumination-standardized features respectively, and then concatenates the two 32-dimensional features into a 64-dimensional single-column fusion feature vector to achieve multi-source feature fusion with adaptive weighting of illumination interference, which is used as the input to the visual fatigue assessment model.

[0114] The multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of light parameters provided in this embodiment, in step S200, firstly, extracts the original feature dimensions of the eye contour feature vector and the ambient light distribution feature, respectively, and determines the dimensional parameters of the two heteromodal features, the eye contour feature vector and the ambient light distribution feature. The eye contour feature vector is 4-dimensional, and the ambient light distribution feature vector is 3-dimensional, so their dimensions are inconsistent. Both are stretched to a fixed 32 dimensions through linear interpolation and normalized to the [0, 1] interval to obtain the dimension-aligned standardized eye feature matrix. Normalized illumination feature matrix after dimensional alignment .

[0115] Then, calculate the Pearson correlation coefficient according to formula (6). The absolute value is taken as the fusion weight.

[0116] Finally, adaptive weighted stitching is performed according to formula (7): the eye features are multiplied by Illumination characteristics multiplied by The 64-dimensional multi-source fusion feature vector is obtained by concatenation. .

[0117] Furthermore, the multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment specifically includes step S300 as follows:

[0118] Step S310: Call the pre-trained fatigue state assessment convolutional neural network model, use a fixed-size fatigue feature convolution kernel to perform point-by-point sliding convolution operation on the multi-source fusion feature vector, extract local fatigue detail features from the fusion features, and generate a local fatigue feature matrix.

[0119] A 64-dimensional multi-source fusion feature vector is input into a pre-trained one-dimensional convolutional network for fatigue assessment. A dedicated one-dimensional fatigue convolutional kernel of length 3 is used to slide the convolution point by point along the feature sequence without filling, to mine the local coupling fatigue details between adjacent dimensions of eye contour and ambient lighting features, and output a local fatigue feature matrix, providing fine local feature input for subsequent high-order fatigue feature extraction and fatigue level quantification.

[0120] The local feature matrix extraction operation of convolution is shown in the following formula:

[0121] (8)

[0122] In formula (8), The 64-dimensional multi-source fusion feature vector output by formula (7) is a one-dimensional linear sequence that fuses two types of standardized features: illumination and eye contour. It is a one-dimensional point-by-point sliding convolution operator that slides without padding along the length of the feature vector, and is used to uncover details of local fatigue correlations between adjacent dimensions. This is a one-dimensional fatigue-specific convolutional kernel with a fixed length of 3, belonging to the network weights that are fixed after offline pre-training, in real space. To specifically capture the local coupling fatigue patterns between the four quadrant features of the eye and the temporal features of ambient light; The local fatigue feature matrix is ​​output for the convolution operation. It is dimensionless and carries the subtle fatigue change information of the continuous dimensions within the fused features, and is used as the input of the next layer of the CNN. Formula (8) uses a pre-trained one-dimensional fatigue convolution kernel of size 3 to perform one-dimensional sliding convolution on the 64-dimensional multi-source fusion feature vector to extract local fatigue details and obtain the convolution feature matrix; then it is fed into a fixed-level one-dimensional CNN network to extract high-order fatigue features layer by layer, and finally outputs a one-dimensional quantified visual fatigue level.

[0123] Step S320: Perform multi-layer alternating convolution and max pooling downsampling on the local fatigue feature matrix to filter out redundant noise features, enhance deep fatigue semantic information, and extract high-dimensional deep fatigue characterization features.

[0124] The local fatigue feature matrix output by convolution is subjected to three layers of alternating one-dimensional convolution and 2×1 window max pooling downsampling iteration. Max pooling preserves effective features of strong fatigue and filters out illumination fluctuations and redundant noise of image pixels, and mines the deep fatigue semantics of eye contour coupled with ambient lighting layer by layer. After all iterations are completed, global average pooling compression is applied to output a fixed 64-dimensional balanced deep fatigue representation feature vector for fatigue level classification output.

[0125] The feature extraction calculation for deep fatigue characterization is shown in the following formula:

[0126] (9)

[0127] In formula (9), The local fatigue feature matrix output by the one-dimensional sliding convolution of formula (8) carries the local fatigue details of shallow adjacent dimensions, including redundant interference information such as small fluctuations of the light sensor and image pixel noise. It is a one-dimensional max pooling downsampling operator. The pooling window is fixed at 2×1 and slides along the length of the feature sequence. The maximum value in the window is taken as the output, and low-value noise and redundant information in the window are discarded. The deep fatigue characterization features after downsampling are dimensionless; after multiple alternating convolution-max pooling iterations and finally global average pooling compression, they are uniformly regularized into a 64-dimensional balanced feature vector; the multi-layer alternating logic is as follows: After input, the three-layer alternating iteration of "convolution operation → formula (9) max pooling downsampling" is repeated to gradually filter out shallow redundant noise and strengthen the deep semantic association features of eye fatigue coupled with ambient light. Formula (9) uses a 2×1 window one-dimensional max pooling operator to downsample the local fatigue features of convolution, and combines multi-layer alternating convolution-pooling iteration to filter out redundant noise and extract deep fatigue features. Finally, a fixed 64-dimensional deep fatigue characterization feature is obtained by global average pooling.

[0128] Step S330: Perform weight allocation and normalization processing on the deep fatigue characterization features through the fully connected weight layer of the convolutional neural network model, and quantize the output of the initial fatigue state value in the range of 0 to 1.

[0129] The 64-dimensional deep fatigue characterization features are input into a fully connected weight layer. Offline pre-trained, solidified weight vectors are used to perform a linear weighted summation of the fatigue features across all dimensions. The unbounded raw fatigue scores are compressed and mapped using a Sigmoid activation function. The interval outputs continuous dimensionless initial fatigue quantization values, providing a standardized fatigue quantization basis for dynamic graded control of backlight brightness in display devices.

[0130] The initial fatigue state value is quantified using the following formula:

[0131] (10)

[0132] In formula (10), Step S320 outputs a 64-dimensional deep fatigue characterization feature vector, which is dimensionless and carries the high-order eye-light coupled fatigue semantics after noise filtering. The pre-trained weight vectors for the fully connected layer have a length that matches the dimension of the deep features and are fixed offline. They are used to allocate the contribution ratio of each dimension of deep fatigue features to the fatigue result. The vector inner product operation is used to perform a linear weighted summation of all deep fatigue features, resulting in the unbounded original fatigue score. The standard expression for the activation normalization function for the Sigmoid function is: The unbounded linear fraction is compressed and mapped to a continuous closed interval. ; The initial fatigue state quantization value, value range Dimensionless: This indicates that the user has no visual fatigue; This indicates that the user is in a state of severe visual fatigue. Formula (10) uses a fully connected pre-trained weight vector to linearly weight the 64-dimensional deep fatigue features, and then normalizes and compresses them through the Sigmoid activation function to output a continuous initial visual fatigue quantization value in the range of 0 to 1, which is used for graded smooth control of screen light parameters.

[0133] The multi-sensor fusion visual fatigue assessment and dynamic adjustment of optical parameters method provided in this embodiment includes step S300, in which a 64-dimensional fused feature vector is input into a pre-trained convolutional neural network. The convolutional neural network structure is as follows: Input layer (64-dimensional) → Convolutional layer 1 (kernel size 3, channels 16, activation function ReLU) → Max pooling (window 2, stride 2) → Convolutional layer 2 (kernel size 3, channels 32, activation function ReLU) → Max pooling (window 2, stride 2) → Convolutional layer 3 (kernel size 3, channels 64, activation function ReLU) → Global average pooling (output 64-dimensional) → Fully connected layer (output 1-dimensional, activation function Sigmoid). The initial fatigue state value is then output. .

[0134] The training data for the convolutional neural network model came from eye images and synchronous physiological data of 50 subjects under different lighting conditions (dim light, normal light, and strong light). Fatigue levels were labeled by ophthalmologists (0 = no fatigue, 1 = fatigue), totaling 5000 samples. The loss function was binary cross-entropy, the optimizer was Adam, the learning rate was 0.001, and the number of training epochs was 100.

[0135] Furthermore, the multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment specifically includes step S400 as follows:

[0136] Step S410: Synchronously collect real-time eye usage time data of the target user, as well as the continuous physiological state time sequence output by the wearable physiological monitoring device.

[0137] Two types of time-series auxiliary monitoring data are collected simultaneously: one is the time-series data of the target user's cumulative real-time eye use duration; the other is the time-series sequence of continuous physiological states such as heart rate, blood oxygen, and blink frequency output by the wearable physiological monitoring device. The two types of data are aligned with the timestamp of the aforementioned visual fatigue assessment data stream and used as auxiliary feature inputs for fatigue correction, which are used to compensate and correct the initial fatigue quantification value in terms of physiological and duration dimensions.

[0138] Step S420: Perform equal-interval uniform segmentation on the continuous physiological state time sequence to obtain multiple short-term physiological state sub-sequences, thereby achieving refined decomposition of physiological fluctuation characteristics.

[0139] The continuous physiological state time sequence output by the wearable device is evenly divided into 5 short-term physiological subsequences with a fixed number of segments. All subsequences are combined to form a physiological state set. The details of heart rate and blood oxygen fluctuations at different time periods are refined, and physiological fatigue auxiliary features are extracted in segments and time periods to correct the initial fatigue value of the visual modality output.

[0140] The physiological state subsequence segmentation rule is shown in the following formula:

[0141] (11)

[0142] In formula (11), It is a collection of segmented physiological states, consisting of all short-term physiological subsequences; For the first The short-term physiological subsequences after uniform segmentation carry local short-term physiological fluctuation characteristics such as heart rate and blood oxygenation. The total number of time-series segments is fixed at 5, and the complete physiological time sequence is divided into 5 segments at equal intervals for fine-grained segmentation. Formula (11) divides the complete physiological time sequence output by the wearable device into 5 fixed short-term physiological subsequences at equal intervals, forming a set of physiological state features, which is used to finely extract the physiological fluctuation fatigue features of different time periods.

[0143] Step S430: Perform mean quantization on each physiological state subsequence, collect standardized physiological feature parameters, and construct a physiological state feature vector with regular dimensions.

[0144] The segmented arithmetic mean of the five short-term physiological subsequences obtained in step S420 is calculated. The five means are then horizontally concatenated and transposed to generate a five-dimensional original physiological state column vector. The components are then uniformly mapped to the dimensionless interval [0, 1] by Min-Max scaling, and a standardized physiological feature vector is output as a physiological modality auxiliary correction feature to participate in the comprehensive fatigue value correction.

[0145] The formula for constructing physiological state feature vectors is shown below:

[0146] (12)

[0147] In formula (12), For the first Segment physiological subsequence The arithmetic mean; representing the average physiological level of segments, such as the average heart rate and the average blood oxygen saturation of segments; To ensure a fixed total number of segments, therefore It is a 5-dimensional column vector; The original physiological state feature vector is represented by each component, which retains the original physiological physical units (heart rate bpm, blood oxygen %) and has not eliminated dimensions. To ensure a unified feature input format, the horizontally concatenated and transposed column vector is used. Formula (12) calculates the mean of the five equally divided physiological subsequences and concatenates them to generate a 5-dimensional original physiological feature vector. Then, it is normalized to the dimensionless interval [0, 1] by window Min-Max scaling to obtain a standardized physiological feature vector, which is used for multimodal comprehensive fatigue fusion assessment.

[0148] Step S440: Perform Min-Max normalization on the real-time eye usage duration data and physiological state feature vector so that each component is in the [0, 1] interval and is dimensionless; perform feature fusion on the initial fatigue state value, the normalized eye usage duration data, and the normalized physiological state feature vector to construct a 7-dimensional fatigue feature vector containing multi-dimensional fatigue parameters.

[0149] The 5-dimensional original physiological features were independently normalized component by component using Min-Max extreme value scaling to eliminate the differences in physical dimensions of heart rate and blood oxygen, and the output value range [0, 1] was standardized physiological features. The initial visual fatigue value, normalized eye use duration, and 5-dimensional normalized physiological features were horizontally spliced ​​and transposed to construct a 7-dimensional unified dimensionless multidimensional fatigue feature column vector. The information of four dimensions, namely visual imaging, ambient light, eye use load and human physiology, was integrated as the input of the comprehensive fatigue assessment integrated model.

[0150] (13)

[0151] In formula (13), For step S430, output the original 5-dimensional physiological feature vector. One portion, It carries original physical dimensions such as heart rate (bpm) and blood oxygen (%); The first in the current window The minimum value of all sampling points in the segmented physiological subsequence; The first in the current window The maximum value of all sampling points in the segmented physiological subsequence; For the first Normalized physiological feature components; independent extreme value scaling is performed on each physiological indicator, and all components are uniformly mapped to... The interval eliminates the differences in dimensions and numerical magnitudes between different physiological indicators; the minimum and maximum values ​​of each component are determined by the reference range or historical extreme values ​​provided by the equipment manufacturer. Output the initial fatigue value of visual single modality in the range of 0~1 for formula (10); To accumulate screen time in real time, in minutes; min is the preset safe upper limit of screen time, used as the normalized denominator; To normalize the eye use duration feature, the value range is... A threshold of 0.5 corresponds to a 30-minute early warning node. A 5-dimensional dimensionless physiological characteristic row vector; This is the concatenated and transposed one-dimensional fatigue feature column vector, with a total dimension of [missing information]. Dimension, Unified Dimensionless As input to the integrated model. Formula (13) splices visual fatigue, normalized eye use time, and normalized physiological characteristics to generate a 7-dimensional standardized multidimensional fatigue feature vector.

[0152] Step S450: Preset safe eye use time threshold. When the eye use time parameter in the 7-dimensional fatigue feature vector exceeds the preset safe eye use time threshold, start the comprehensive fatigue state classification model, analyze the 7-dimensional fatigue feature vector through decision tree integration operation, and quantify and determine the comprehensive fatigue state assessment value of the target user.

[0153] A safe eye-use duration threshold is set; a comprehensive fatigue assessment is triggered when the normalized eye-use duration feature exceeds the threshold. A 7-dimensional standardized fatigue feature vector is input into an integrated decision tree model, employing... Parallel prediction using single decision trees with a maximum depth of 5 and Gini index splitting. Each tree outputs a fatigue probability score of 0 to 1, which is then weighted and summed according to preset weights. The final output is a comprehensive fatigue assessment value in the range of [0, 1], which fully integrates fatigue information from multiple dimensions, including visual, lighting, physiological, and eye usage duration.

[0154] The comprehensive fatigue assessment value is calculated using the following formula:

[0155] (14)

[0156] In formula (14), The total number of decision trees is fixed for the ensemble model; For the first Single decision tree prediction function: 1. Maximum tree depth is limited to 5 to prevent overfitting; 2. Gini index is used as the node splitting criterion; 3. Single tree outputs fatigue probability score in the range of 0 to 1; For the first The ensemble weights of the trees satisfy the following conditions: Cross-validation can be used to optimize assignment or to distribute equally. The value range is for multimodal fusion comprehensive fatigue assessment. 0 represents no fatigue and 1 represents severe comprehensive fatigue. Formula (14) uses a weighted ensemble of 10 Gini decision trees with a maximum depth of 5 when the duration of eye use exceeds the threshold, and outputs a comprehensive fatigue assessment value in the range of 0 to 1.

[0157] The multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment includes step S400, which synchronously collects the user's cumulative eye usage time. (Unit: min) and physiological time-series data output from wearable devices (smart bracelets, smartwatches, biopatch monitors, chest strap heart rate belts, finger clip SpO2 monitors, ear-worn PPG sensors, etc.). Physiological time-series data includes, but is not limited to, heart rate, heart rate variability (HRV), oxygen saturation (SpO2), skin conductance (GSR), skin temperature, respiratory rate, and acceleration of movement. Wearable devices can acquire this data through PPG photoplethysmography, ECG, EDA impedance analysis, etc. The physiological time series is divided into 5 equally spaced segments, and the average value of each segment is taken to obtain a 5-dimensional original physiological feature vector. Since physiological quantities such as heart rate and blood oxygen have different physical units and ranges, each component is normalized using Min-Max (using the reference range or historical extreme values ​​provided by the equipment manufacturer) to obtain the... Normalized physiological characteristic components .

[0158] Will Normalized screen time ( min) Concatenate into a 7-dimensional fatigue feature vector .

[0159] When the normalized screen time exceeds 0.5 (corresponding to 30 minutes), the random forest ensemble model (10 decision trees, maximum depth 5, Gini index splitting) is activated, and a comprehensive fatigue assessment value is output. The weights of each decision tree are determined through out-of-bag error estimation, with equal-weight ensemble used by default. The training data consists of 7-dimensional fatigue feature vectors and corresponding subjective fatigue scores, totaling 3000 samples.

[0160] Furthermore, in the multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment, step S500 includes:

[0161] Step S510: Preset fatigue warning threshold, compare the comprehensive fatigue state assessment value with the preset fatigue warning threshold, and output a binary fatigue warning indicator.

[0162] Preset fatigue warning threshold calibrated through human subjects The integrated fatigue assessment value output by the model is compared with the fatigue warning threshold, and a binary fatigue warning label is output. When the label is 1, the user is judged to be severely fatigued, and the screen backlight, color temperature, and blue light ratio eye protection adjustment process is automatically started. When the label is 0, the existing light parameters of the screen remain unchanged.

[0163] The threshold comparison judgment rule is shown in the following formula:

[0164] (15)

[0165] In formula (15), The multimodal comprehensive fatigue assessment value in the 0~1 interval output by formula (14) is a fusion of four-dimensional information including eye vision, ambient light, physiological signs, and cumulative eye use time. It is a globally fixed fatigue early warning threshold, dimensionless; and serves as the criterion for determining the boundary between moderate and severe fatigue. As an empirical threshold, it was calibrated through subject experiments: In a fatigue-inducing experiment with 50 subjects, when the comprehensive fatigue value exceeded 0.6, 92% of the subjects reported significant visual discomfort (symptoms such as dry eyes and soreness). Therefore, 0.6 was set as the critical value for triggering eye protection dimming. For binary fatigue warning indicators, only 0 or 1 is used, directly serving as the screen light control switch signal: If the overall fatigue value exceeds 0.6, the user is determined to be in the severe fatigue range, and the screen brightness, color temperature, and blue light ratio will be automatically adjusted. This indicates that the overall fatigue value has not reached the preset fatigue warning threshold, and the fatigue level is judged to be relatively mild. The current light output parameters of the screen remain unchanged. Formula (15) compares the multimodal overall fatigue value with the preset fatigue warning threshold of 0.6 and outputs a 0 / 1 binary warning label; when the label is 1, the screen eye protection dimming is activated, and when the label is 0, the original light output parameters are maintained.

[0166] Step S520: When it is determined that a fatigue warning has been triggered, retrieve the ambient light intensity time series data within a preset time period and construct an ambient light intensity time series sequence.

[0167] If the binary fatigue warning sign The eye-protection dimming process is triggered, and the original ambient light sampling values ​​for a preset duration in the cache window are read and aggregated to form a continuous ambient light intensity time sequence, providing real-time ambient light reference data for subsequent adaptive compensation and adjustment of backlight brightness.

[0168] Step S530: Extract the fluctuation characteristic parameters of the ambient light intensity time series, combine them with the comprehensive fatigue state assessment value to perform coupling operation, and quantify the visual sensitivity metric of the target user.

[0169] Two types of fluctuation features, namely the temporal standard deviation and the window-average illumination intensity, are extracted from the ambient illumination time series. The basic visual sensitivity is obtained by coupling the comprehensive fatigue assessment value with the relative illumination fluctuation coefficient. A minimum constant is introduced to avoid overflow when the denominator is zero. The numerical range is then constrained by truncating the maximum value, and the value range is output. The visual sensitivity quantification value indicates that the higher the value, the more sensitive the user is to changes in screen light, and the greater the eye protection dimming compensation.

[0170] The quantification calculation of visual sensitivity is shown in the following formula:

[0171] (16)

[0172] In formula (16), The value is a dimensionless, real-time quantification of the user's visual sensitivity. The larger the value, the more sensitive the human eye is to changes in screen brightness and color temperature, and the greater the dimming compensation needs to be. Output the comprehensive fatigue assessment value for formula (14) This indicates the current overall level of fatigue; the higher the fatigue, the higher the visual sensitivity. The standard deviation of ambient light time series, in units It characterizes the intensity of instantaneous fluctuations in ambient light; the greater the light fluctuation, the stronger the visual stimulation to the human eye, and the higher the sensitivity. The average ambient light intensity within the window, in units of The higher the ambient brightness, the weaker the visual stimulation caused by the same light fluctuation. For the minimum divisibility constant, the unit is... Avoid low-light environments This results in a denominator of 0 and an overflow of the calculated value. The relative fluctuation coefficient of ambient light is used to quantify the interference intensity of light fluctuation relative to the average illuminance. Set a fixed upper limit for visual sensitivity; adopt Truncation operator to prevent infinite amplification of sensitivity in scenarios with strong light fluctuations and severe fatigue, and to constrain the value range. Formula (16) uses the comprehensive fatigue value weighted by the relative fluctuation coefficient of illumination to obtain the basic visual sensitivity. A minimum constant is introduced to prevent the denominator from being zero. The value is truncated by a fixed upper limit of 10, and the real-time visual sensitivity in the range of 0~10 is output for adaptive adjustment of the screen's eye protection dimming amplitude.

[0173] Step S540: Normalize the visual sensitivity quantification value to [0, 1], and then retrieve the preset sensitivity-brightness weight mapping relationship based on the normalized visual sensitivity to obtain the corresponding initial backlight brightness adjustment weight.

[0174] The original visual sensitivity in the range [0,10] is linearly scaled and truncated at the upper limit to convert it into a normalized visual sensitivity in the range [0,1]. Based on the preset linear mapping rule, the normalized sensitivity is substituted to calculate the backlight brightness adjustment weight. The weight value is [0.2,0.8]. The larger the weight value, the greater the reduction in screen backlight.

[0175] The normalized value of the visual sensitivity metric is shown in the following formula:

[0176] (17)

[0177] In formula (17), Output the original visual sensitivity for formula (16), with a value range of ; To normalize visual sensitivity, the value range is strictly defined. Divide by 10 to achieve range scaling. Upper limit truncation is used to prevent extreme calculation errors from causing the normalized value to exceed the range. Formula (17) divides the original visual sensitivity by a fixed upper limit of 10 for linear scaling, and then truncates the maximum value of 1 using the minimum value operator to output the value range. The normalized visual sensitivity is used to linearly solve for the backlight brightness adjustment weights.

[0178] The sensitivity-brightness weight mapping relationship is a linear mapping, as shown in the following formula:

[0179] (18)

[0180] In formula (18), The weight for backlight brightness adjustment is dimensionless and has a range of values. The higher the value, the greater the reduction in screen backlight. To normalize visual sensitivity, the value range is strictly defined. Mapping coefficient explanation: 1. Base offset of 0.2 indicates the minimum eye protection adjustment weight, ensuring that there is still a basic eye protection brightness reduction in mildly sensitive scenarios; 2. Slope of 0.6 indicates that for every unit increase in sensitivity, the weight increases by 0.6, fully covering the weight range of 0.2 to 0.8. Formula (18) first scales and truncates the original visual sensitivity in the range of 0 to 10 to a normalized sensitivity of 0 to 1, and then obtains the backlight brightness adjustment weight in the range of 0.2 to 0.8 through linear mapping with fixed coefficients, which is used to adaptively reduce the screen backlight to achieve eye protection dimming.

[0181] Step S550: Combining the current backlight brightness value of the current display device with the initial backlight brightness adjustment weight and the comprehensive fatigue state evaluation value, perform adaptive brightness calculation to obtain the target backlight brightness value of the display screen that meets the needs of human eye protection.

[0182] Using the current real-time backlight brightness of the display device as the calculation benchmark, the brightness attenuation ratio is solved by coupling the backlight adjustment weight and the comprehensive fatigue assessment value, and the current brightness is proportionally attenuated; then, the minimum safe brightness is constrained by the lower limit truncation operator, and the target backlight brightness in the value range [50, 1000] is output and sent to the backlight driver chip to complete the adaptive eye protection dimming.

[0183] The target backlight brightness value is calculated using the following formula:

[0184] (19)

[0185] In formula (19), The target backlight brightness of the screen, in units The range is constrained to This refers to the final brightness value sent to the backlight driver chip. The current real-time backlight brightness of the device, in units Dimming calculation benchmark. The backlight adjustment weight is output for step S540, in the range of This represents the baseline downward adjustment magnitude corresponding to visual sensitivity. This represents the multimodal comprehensive fatigue assessment value, within the range. This indicates the overall level of user fatigue and is used to amplify and adjust the amplitude of the adjustment. This is the composite attenuation coefficient; the larger the value, the higher the brightness reduction ratio. This represents the brightness attenuation factor, with a value range of... The brightness is reduced by multiplying the current brightness. Minimum safe backlight brightness allowed by the hardware; The lower limit truncation operator is used to prevent the brightness from decaying to 50. The following results in an image that is too dark to be discernible. Formula (19) uses the current screen brightness as a benchmark, and obtains the brightness attenuation ratio by coupling the backlight adjustment weight with the overall fatigue value. After proportionally reducing the brightness, it is truncated to 50. The lower limit is to output compliant eye-protection target backlight brightness.

[0186] The multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment, in step S500, if (This fatigue warning threshold was calibrated through subject experiments: when the overall fatigue value exceeds 0.6, more than 90% of subjects report significant visual discomfort), thus triggering dimming. First, ambient light time-series data from the most recent period is retrieved, and the standard deviation of the ambient light time-series is calculated. and average ambient light intensity inside the window Then calculate the visual sensitivity according to formula (16). Then normalize to [0,1] according to formula (17), and obtain the brightness adjustment weight by linear mapping according to formula (18). Finally, the target backlight brightness is calculated according to formula (19). The lower limit protection is 50 cd / m².

[0187] The sensitivity-brightness mapping relationship was calibrated through a subjective experiment: Twenty healthy subjects aged 20-40 years were recruited. Under controlled fatigue levels (achieved through an induced task) and ambient lighting conditions (dark room, normal, bright light), subjects were asked to adjust the screen brightness to a comfortable level, and the corresponding values ​​were recorded. and The mapping coefficients were obtained by linear regression fitting, with 0.2 (intercept) and 0.6 (slope).

[0188] Preferably, in the multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment, step S600 includes:

[0189] Step S610: Based on the current backlight brightness value and the target backlight brightness value, a multi-frame continuous brightness smooth transition sequence is generated using a linear interpolation algorithm to avoid the visual impact caused by instantaneous brightness changes.

[0190] Using the current backlight brightness as the interpolation start point and the target eye-protection backlight brightness as the interpolation end point, a fixed total number of interpolation frames is set. The system generates a 20-frame continuous and smooth brightness transition sequence by evenly distributing the brightness difference through linear interpolation; brightness commands are issued frame by frame to avoid screen flicker caused by single jumps, thus achieving soft and shock-free backlight adjustment.

[0191] The calculation for generating a smooth brightness transition sequence is shown in the following formula:

[0192] (20)

[0193] In formula (20), For the first Frame interpolation transition backlight brightness, unit ; To adjust the current backlight brightness of the front screen, the starting point of the interpolation sequence must satisfy... ; The target eye-protecting backlight brightness obtained by solving formula (19), the endpoint of the interpolation sequence, satisfies ; To maintain a fixed total number of frames for smooth interpolation, the complete transition process is completed in 20 steps. The frame index has an integer value. ; is the brightness step difference of a single frame, and is the brightness change between two adjacent frames, to achieve uniform linear gradient. Formula (20) takes the current brightness as the starting point and the target brightness as the ending point, and generates a frame-by-frame smooth brightness transition sequence by linear interpolation in 20 equal frames, and outputs it frame by frame to achieve flicker-free soft backlight adjustment.

[0194] Step S620: Based on the preset linear mapping relationship between screen brightness and PWM duty cycle, the brightness smooth transition sequence is converted frame by frame to generate the corresponding PWM duty cycle sequence.

[0195] For each frame of smooth transition backlight brightness generated in step S610, a linear normalization conversion is performed based on the hardware rated maximum backlight brightness, and the brightness value is converted into a percentage format PWM duty cycle sequence frame by frame; the duty cycle timing signal is sent to the backlight driver chip to realize smooth graded dimming control at the hardware level.

[0196] The duty cycle conversion calculation is shown in the following formula:

[0197] (twenty one)

[0198] In formula (21), For the first The backlight drive PWM duty cycle corresponding to frame smooth transition, in %, with a value range of %. It directly outputs to the backlight driver chip to control the output power. Output the first value for formula (20) Frame interpolation backlight brightness, unit This serves as the input reference for converting brightness to duty cycle. The maximum rated backlight brightness of the screen hardware, in units This is the upper limit of the brightness measurement range. The value range is 300-1000 In this embodiment, 500 is used. This corresponds to the maximum rated brightness of a typical laptop / tablet monitor; The normalized ratio of the current frame's brightness to the maximum brightness, with a value range of... Multiply by 100 to convert to a percentage duty cycle. Formula (21) scales the ratio of the smooth transition backlight brightness of each frame to the maximum brightness of the screen, converts it into a percentage PWM drive duty cycle, and generates a synchronous timing drive signal to control the screen backlight hardware.

[0199] Step S630: Combine the hardware response delay parameters of the current display device to compensate and correct the basic control cycle, generate a backlight instruction execution cycle that adapts to the hardware response characteristics, and generate a backlight control instruction based on the backlight instruction execution cycle and the PWM duty cycle sequence.

[0200] The inherent response delay parameters of the backlight driver hardware are read. Based on the 20ms basic control cycle aligned with the screen refresh rate, hardware delay completion cycle compensation correction is added to generate a backlight instruction execution cycle that adapts to the hardware lag characteristics. Using this backlight instruction execution cycle as the time interval, the PWM duty cycle sequence is encapsulated frame by frame to generate backlight control instructions, eliminating dimming timing misalignment caused by hardware transmission lag, and ensuring uniform and accurate output of the 20-frame smooth transition sequence.

[0201] The backlight instruction execution cycle correction calculation is shown in the following formula:

[0202] (twenty two)

[0203] In formula (22), The backlight instruction execution cycle is measured in milliseconds (ms); it serves as the standard time interval for issuing PWM duty cycle instructions, adapting to the inherent hysteresis characteristics of the hardware. The base control cycle is measured in milliseconds (ms). The screen refresh cycle is taken as an integer multiple of the screen refresh cycle. For example, the refresh cycle of a 60Hz screen is 16.67ms. A fixed 20ms cycle is selected as the base cycle to ensure alignment with the screen frame timing. The backlight driver hardware response delay, measured in milliseconds (ms), is caused by the inherent transmission lag of the driver chip and backlight circuit. It is typically measured in factory calibration or the equipment specification sheet, and ranges from 5 to 20 ms. In this embodiment, the response time of the PWM signal in the backlight driver circuit is measured using an oscilloscope. =10ms, or directly read from the Turn-on Time parameter of the driver chip datasheet (such as TPS61165, PT4103). The correction logic is to add the inherent hardware delay to the basic cycle to offset the dimming timing misalignment caused by hardware lag, and ensure that the 20-frame smooth PWM sequence is output uniformly according to the design timing. Formula (22) adds the backlight hardware response delay to the basic control cycle synchronized with the screen refresh to obtain the corrected instruction issuance cycle, which is used to encapsulate the PWM duty cycle timing instruction to compensate for hardware lag and ensure the accurate execution of the smooth dimming timing.

[0204] Step S640: Extract the driving current amplitude and instruction execution cycle according to the backlight control instruction, and perform illumination control operation on the backlight driving module of the current display device according to the driving current amplitude and instruction execution cycle to adjust the backlight luminous intensity of the display screen of the current display device.

[0205] Based on the rated maximum drive current of the backlight hardware, the real-time drive current amplitude of the LED in the corresponding frame is obtained by linearly scaling the PWM duty cycle of each frame. Combined with the backlight instruction execution cycle corrected in step S630, the frame-by-frame drive current and timing cycle are encapsulated into a backlight control instruction and sent to the backlight drive module. The LED luminous intensity is adjusted according to the smooth timing gradient to complete the final eye-protection dimming hardware execution.

[0206] The drive current scaling calculation is shown in the following formula:

[0207] (twenty three)

[0208] In formula (23), For the first The frame backlight real-time drive current, in mA, is directly output to the LED backlight drive circuit, which determines the backlight luminous intensity. The rated maximum drive current of the backlight hardware, in mA, is fixed at the factory by the specifications of the screen backlight LED beads and the hardware parameters of the driver chip. The output of formula (21) is the first The frame PWM duty cycle; the operation logic is based on the maximum drive current as the upper limit, and scaled according to the PWM duty cycle ratio corresponding to the smooth timing to obtain the LED drive current amplitude matched for each frame. Formula (23) takes the maximum backlight drive current of the hardware as the benchmark, scales according to the single frame PWM duty cycle ratio to obtain the timing synchronized LED backlight drive current amplitude, and with the corrected instruction cycle output, realizes gradient smooth eye protection backlight control.

[0209] The multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment includes step S600, in which a 20-frame linear interpolated brightness sequence is generated according to formula (20). ,in , The brightness of each frame is converted into PWM duty cycle according to formula (21). (Percentage form). Correct the instruction execution cycle according to formula (22): base cycle Take the screen refresh rate cycle rounded up (60Hz corresponds to 16.67ms, rounded up to 20ms), and add the hardware response latency. (Obtained from the equipment specifications; typical value 5~20ms). Convert the percentage to a decimal (divide by 100), and map it to the drive current according to formula (23). The image is output frame by frame to the backlight driver module to achieve flicker-free gradual dimming.

[0210] The multi-sensor fusion method for visual fatigue assessment and dynamic adjustment of optical parameters provided in this embodiment achieves the following beneficial effects compared with the prior art:

[0211] 1. This embodiment integrates four types of data sources: facial images from a camera, ambient light sensors, wearable physiological sensors, and device timing modules. It designs a pixel-level reverse compensation algorithm for eye edges to address ambient light interference, which significantly improves the accuracy of eye contour feature extraction in low-light and high-light scenes, and solves the pain point of distortion caused by ambient light interference in single image recognition.

[0212] 2. By introducing the Pearson correlation coefficient, we can achieve adaptive weighted fusion of eye and illumination dual-modal features, fully explore the coupling relationship between ambient light and eye fatigue. Compared with fixed weight concatenation, the fused feature discrimination is significantly improved, providing high-quality input features for fatigue neural networks.

[0213] 3. By using convolutional neural networks to mine deep-seated latent fatigue characteristics of the eyes, subtle and progressive fatigue features can be identified and an accurate initial fatigue value can be output. Then, by combining the duration of eye use and physiological time series data, a comprehensive fatigue assessment value is obtained through a decision tree ensemble model, which takes into account both visual fatigue and human physiological drowsiness. The assessment results are comprehensive and the misjudgment rate is greatly reduced.

[0214] 4. After fatigue warning, the real-time visual sensitivity of the human eye is quantified by coupling the light timing fluctuation and the comprehensive fatigue value. The target backlight brightness is dynamically calculated based on the pre-calibrated sensitivity-brightness mapping relationship. Fixed dimming level is abandoned. The brightness adjustment conforms to the real-time tolerance characteristics of the human eye and improves viewing comfort.

[0215] 5. Standardized generation of backlight drive control commands with smooth constraints, segmented linear adjustment of backlight intensity, avoiding sudden brightness changes and flickering that could cause secondary damage to the human eye, and realizing fully automatic closed-loop dynamic control of light parameters based on the user's real-time visual fatigue state and ambient light, which can effectively delay the deepening of visual fatigue with long-term use.

[0216] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for visual fatigue assessment and dynamic adjustment of optical parameters using multi-sensor fusion, characterized in that, Includes the following steps: S100: Collect the facial video stream of the target user and the ambient light intensity data of the current display device environment; perform pixel-level edge detection processing on the eye image frames in the facial video stream using an image edge feature extraction algorithm; and perform illumination inverse compensation operation on the detected edge pixels based on the ambient light intensity data to obtain the eye contour feature vector of the eye image frame. At the same time, use an illumination feature extraction model to extract multi-dimensional features from the ambient light intensity data to obtain the ambient light distribution features. S200. The eye contour feature vector and the ambient light distribution feature are dimensionally aligned and stitched together using a multimodal feature stitching method. The Pearson correlation coefficient of the two types of features is quantified by statistical correlation operation. Based on the Pearson correlation coefficient, adaptive weighted stitching fusion is performed to establish the correlation mapping relationship between the eye contour feature vector and the ambient light distribution feature, and a multi-source fusion feature vector is obtained. S300. The multi-source fusion feature vector is subjected to multi-layer convolution and pooling processing through a pre-trained fatigue state evaluation convolutional neural network model to extract deep fatigue representation information from the multi-source fusion feature vector and output the initial fatigue state value for the multi-source fusion feature vector. S400: Obtain the current eye usage duration data of the target user and the physiological state sequence data collected by the wearable device; use a comprehensive fatigue state classification model to perform decision tree integration processing on the initial fatigue state value, the current eye usage duration data, and the physiological state sequence data to determine the comprehensive fatigue state assessment value of the target user. S500. Determine whether the comprehensive fatigue state assessment value is greater than a preset fatigue warning threshold. If the comprehensive fatigue state assessment value is greater than the preset fatigue warning threshold, then quantify the visual sensitivity based on the fluctuation characteristics of the ambient light intensity time series data and the comprehensive fatigue state assessment value, and retrieve the preset sensitivity-brightness weight mapping relationship based on the visual sensitivity to calculate the target backlight brightness value of the display screen of the current display device. S600. Based on the target backlight brightness value, generate a backlight control command for the current display device, and perform illumination regulation operation on the backlight driving module of the current display device according to the backlight control command to adjust the backlight luminous intensity of the display screen of the current display device.

2. The method for visual fatigue assessment and dynamic adjustment of optical parameters based on multi-sensor fusion according to claim 1, characterized in that, Step S100 specifically includes: S110. Collect the target user's facial video stream and the ambient light intensity data of the environment where the display device is located, perform equal-interval frame separation sampling processing on the facial video stream to obtain eye image frames, and simultaneously collect the time-series illuminance data output by the ambient light sensor, and simultaneously collect and cache the time-series illuminance data. S120. Perform pixel-level grayscale gradient edge detection operation on the eye image frame, filter valid edge pixels by preset edge gradient threshold, remove invalid background pixels, and construct an initial set of edge pixels corresponding to the eye image frame. S130. Calculate statistics on the cached temporal illuminance data to obtain the average ambient light intensity, the temporal standard deviation of illumination, and the peak illumination deviation, thus constructing an ambient light distribution feature vector; perform illumination inverse compensation operation on the initial edge pixel set based on the average ambient light intensity, dividing the eye image into... For each local region, the compensated mean value is calculated to form the eye contour feature vector.

3. The method for visual fatigue assessment and dynamic adjustment of optical parameters based on multi-sensor fusion according to claim 2, characterized in that, Step S200 specifically includes: S210. Extract the original feature dimensions of the eye contour feature vector and the ambient light distribution feature respectively, and determine the dimensional parameters of the two types of heteromodal features. S220. For eye contour features and ambient lighting distribution features with inconsistent dimensions, a linear interpolation method is used to perform a unified dimension alignment transformation to generate a fixed-dimensional eye feature matrix and lighting feature matrix, and the two are normalized to make them dimensionless. S230. Quantify the coupling relationship between the eye feature matrix and the illumination feature matrix through statistical correlation operations, calculate the Pearson correlation coefficient between the two types of features, and construct a stable correlation mapping relationship between eye contour features and ambient illumination distribution features. S240. Based on the Pearson correlation coefficient obtained by quantization, perform adaptive weighted splicing and fusion, dynamically allocate the fusion ratio of the two types of features according to the correlation weight, and generate a multi-source fusion feature vector with unified dimensions and correlation matching.

4. The method for visual fatigue assessment and dynamic adjustment of optical parameters based on multi-sensor fusion according to claim 1, characterized in that, Step S300 specifically includes: S310. Call the pre-trained fatigue state assessment convolutional neural network model, use a fixed-size fatigue feature convolution kernel to perform point-by-point sliding convolution operation on the multi-source fusion feature vector, extract local fatigue detail features from the fusion features, and generate a local fatigue feature matrix. S320. Perform multi-layer alternating convolution and max pooling downsampling on the local fatigue feature matrix to filter out redundant noise features, enhance deep fatigue semantic information, and extract high-dimensional deep fatigue characterization features. S330. The deep fatigue characterization features are weighted and normalized through the fully connected weight layer of the convolutional neural network model, and the initial fatigue state value in the range of 0 to 1 is quantized and output.

5. The method for visual fatigue assessment and dynamic adjustment of optical parameters based on multi-sensor fusion according to claim 4, characterized in that, Step S400 specifically includes: S410: Simultaneously collect real-time eye usage time data of the target user, as well as the continuous physiological state time series output by the wearable physiological monitoring device; S420. Perform equal-interval uniform segmentation on the continuous physiological state time sequence to obtain multiple short-term physiological state sub-sequences, thereby achieving refined decomposition of physiological fluctuation characteristics. S430. Perform mean quantization on each physiological state subsequence, collect standardized physiological feature parameters, and construct a physiological state feature vector with regular dimensions. S440. Perform Min-Max normalization on the real-time eye use duration data and the physiological state feature vector so that each component is in the [0, 1] interval and is dimensionless; perform feature fusion on the initial fatigue state value, the normalized eye use duration data, and the normalized physiological state feature vector to construct a 7-dimensional fatigue feature vector containing multi-dimensional fatigue parameters. S450. A preset safe eye use time threshold is established. When the eye use time parameter in the 7-dimensional fatigue feature vector exceeds the preset safe eye use time threshold, the comprehensive fatigue state classification model is activated. The 7-dimensional fatigue feature vector is analyzed through decision tree integration operation to quantify and determine the comprehensive fatigue state assessment value of the target user.

6. The method for visual fatigue assessment and dynamic adjustment of optical parameters based on multi-sensor fusion according to claim 5, characterized in that, Step S500 specifically includes: S510, Preset fatigue warning threshold, compare the comprehensive fatigue state assessment value with the preset fatigue warning threshold, and output a binary fatigue warning indicator. S520. When it is determined that a fatigue warning has been triggered, retrieve the ambient light intensity time series data within a preset time period and construct an ambient light intensity time series sequence. S530. Extract the fluctuation characteristic parameters of the ambient light intensity time series, and perform coupling operation in combination with the comprehensive fatigue state evaluation value to quantify the visual sensitivity metric of the target user. S540. Normalize the visual sensitivity quantification value to [0, 1], and then retrieve the preset sensitivity-brightness weight mapping relationship based on the normalized visual sensitivity to obtain the corresponding initial backlight brightness adjustment weight. S550. Combining the current backlight brightness value of the current display device with the initial backlight brightness adjustment weight and the comprehensive fatigue state evaluation value, perform adaptive brightness calculation to obtain the target backlight brightness value of the display screen that meets the needs of human eye protection.

7. The method for visual fatigue assessment and dynamic adjustment of optical parameters based on multi-sensor fusion according to claim 1, characterized in that, Step S600 specifically includes: S610. Based on the current backlight brightness value and the target backlight brightness value, a linear interpolation algorithm is used to generate a multi-frame continuous brightness smooth transition sequence to avoid the visual impact caused by instantaneous brightness changes. The calculation for generating a smooth brightness transition sequence is shown in the following formula: ； in, For the first Frame interpolation transition backlight brightness, To adjust the current backlight brightness of the front screen, To protect the eyes, the backlight brightness is optimized. To fix the total number of frames for smooth interpolation, For frame index, This represents the brightness step difference for a single frame, and the brightness change between two adjacent frames. S620. Based on the preset linear mapping relationship between screen brightness and PWM duty cycle, the brightness smooth transition sequence is converted frame by frame to generate the corresponding PWM duty cycle sequence. S630. Based on the hardware response delay parameters of the current display device, the basic control cycle is compensated and corrected to generate a backlight instruction execution cycle that adapts to the hardware response characteristics. Based on the backlight instruction execution cycle and the PWM duty cycle sequence, a backlight control instruction is generated. S640. Extract the driving current amplitude and instruction execution cycle according to the backlight control instruction, and perform illumination control operation on the backlight driving module of the current display device according to the driving current amplitude and instruction execution cycle to adjust the backlight luminous intensity of the display screen of the current display device.

8. The method for visual fatigue assessment and dynamic adjustment of optical parameters based on multi-sensor fusion according to claim 7, characterized in that, In step S620, the duty cycle conversion calculation is shown in the following formula: ； in, For the first The duty cycle of the backlight drive PWM corresponding to smooth frame transition. The maximum backlight brightness is rated for the screen hardware. It is the normalized ratio of the current frame's brightness to the maximum brightness.

9. The method for visual fatigue assessment and dynamic adjustment of optical parameters based on multi-sensor fusion according to claim 8, characterized in that, In step S630, the backlight instruction execution cycle correction calculation is shown in the following formula: ； in, For the backlight instruction execution cycle, Based on the basic control cycle, This is due to the hardware response delay of the backlight driver.

10. The method for visual fatigue assessment and dynamic adjustment of optical parameters by multi-sensor fusion according to claim 9, characterized in that, In step S640, the drive current scaling calculation is shown in the following formula: ； in, For the first Real-time drive current for frame backlight The maximum rated drive current for the backlight hardware.

Images