A screen refresh rate intelligent adjustment system and method based on content analysis

Abstract

The present invention provides a system and method for intelligently adjusting screen refresh rate based on content analysis. The system obtains the user's physiological data in real time through a multimodal data acquisition module, including eye movement signals, heart rate variability, galvanic skin response and epidermal temperature, and generates the user's fatigue prediction results and attention concentration through a data processing and fusion module. The system further analyzes the type of content displayed on the screen through a content perception module, generates refresh rate adjustment parameters for each area in combination with the user's real-time status, and dynamically adjusts the refresh rate of each area of ​​the screen through a regional adjustment module. In addition, the control module adjusts the overall refresh rate and brightness of the screen in real time to optimize the user's visual experience, reduce visual fatigue, and reduce power consumption. The invention can significantly improve the intelligence level of screen display effects, and is particularly suitable for users who use electronic devices for a long time.

Description

Technical Field

[0001] The present invention relates to the field of electronic equipment, and in particular to a system and method for intelligently adjusting screen refresh rate based on content analysis. Background Art

[0002] With the development of display technology, high refresh rate screens have gradually become popular. However, continuous high refresh rates may lead to excessive power consumption and easily cause visual fatigue when users watch the screen for a long time. Traditional screen refresh rate adjustment methods usually rely on fixed refresh rate settings and cannot be dynamically adjusted according to the actual needs and usage scenarios of users. In addition, the refresh rate adjustment for different types of screen content in the prior art is relatively rough, and the relationship between the user's physiological state and visual needs is not fully considered. Therefore, the prior art has the defect that the refresh rate adjustment is not intelligent enough and cannot effectively reduce visual fatigue and save energy consumption. There is an urgent need for a system that can intelligently adjust the screen refresh rate according to the content and user status. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a screen refresh rate intelligent adjustment system based on content analysis, the system comprising:

[0004] A multimodal data acquisition module, the multimodal data acquisition module comprising an eye movement sensor for collecting user eye movement signals, a sensor for collecting heart rate variability, a sensor for collecting skin electrical response, and a temperature sensor for collecting skin temperature;

[0005] Specifically, the system of the present invention can be applied not only to fixed display devices, but also to portable devices such as smart phones or tablets. Through built-in or external sensors, the device can support real-time detection of eye movement signals, heart rate variability, galvanic skin response and skin temperature. Since GSR (Galvanic Skin Response) requires direct contact with the skin, if the user does not hold the device, the system can continue to complete the detection through an external wearable device (such as a smart bracelet or finger ring) and transmit the data to the display device through a wireless connection, ensuring that the method of the present invention can be successfully executed under any circumstances.

[0006] A data processing and fusion module, connected to the multimodal data acquisition module, for processing and fusing the physiological data and generating fatigue prediction results and attention concentration of the user;

[0007] The content perception module is connected to the data processing and fusion module and is used to analyze the display content type and generate refresh rate adjustment parameters for each area according to the user's fatigue status and real-time screen content;

[0008] A regional adjustment module, connected to the content perception module and the data processing and fusion module, is used to adjust the refresh rate of each area of ​​the screen in real time based on a dynamic weight algorithm;

[0009] The control module is used to receive the adaptive response curve generated by the data processing and fusion module, and adjust the overall refresh rate and brightness of the screen in real time.

[0010] Furthermore, the eye movement sensor is a high frame rate camera;

[0011] The heart rate variability sensor uses photoplethysmography (PPG) and electrode patches;

[0012] The galvanic skin response sensor is a highly sensitive electrode;

[0013] The temperature sensor is a high-precision thermistor.

[0014] Furthermore, the data processing and fusion module includes:

[0015] A multimodal signal preprocessing unit that processes physiological signals using an adaptive filtering algorithm;

[0016] A feature extraction and analysis unit, used to extract feature data related to the user's fatigue state from the processed physiological signals;

[0017] A real-time state assessment and weight allocation unit for dynamically adjusting the weights of physiological signals;

[0018] The fusion and decision-making unit is used to generate the user's fatigue prediction results and attention concentration. The fatigue prediction results are achieved by fusing eye movement data, heart rate variability data and skin electrical response data, combined with a time series analysis model.

[0019] Furthermore, the feature extraction and analysis unit extracts pupil dilation velocity, HRV (Heart Rate Variability) low-frequency / high-frequency ratio, skin conductivity variation amplitude and epidermal temperature fluctuation respectively through principal component analysis (PCA) and independent component analysis (ICA) algorithms.

[0020] Furthermore, the content perception module analyzes the content type displayed on the screen, including text, video and game content, based on a support vector machine (SVM) algorithm, and generates corresponding refresh rate adjustment parameters.

[0021] Furthermore, the regional adjustment module adjusts the refresh rate of each area of ​​the screen in real time according to the user's line of sight focus time and physiological data through a predetermined dynamic weight algorithm.

[0022] Furthermore, the regional adjustment module includes a multi-level regional division structure, which divides the screen into a core area, a sub-core area and a peripheral area, and sets a refresh rate weight for each area.

[0023] Furthermore, the control module also includes a task management unit, which is used to dynamically adjust the refresh rate of different areas of the screen according to the user's task type and priority.

[0024] Furthermore, the task management unit monitors and analyzes the user's operation behavior and application status in real time through a support vector machine (SVM) algorithm to identify the type of task currently being executed and allocate a corresponding refresh rate according to priority.

[0025] The present invention also provides a method for intelligently adjusting screen refresh rate based on content analysis, the method comprising:

[0026] Step 1: Obtain the user's physiological data and generate fatigue prediction results through the data processing and fusion module;

[0027] Step 2: Analyze the screen display content type and generate refresh rate adjustment parameters for each area;

[0028] Step 3: According to the user's sight focus time and fatigue state, the refresh rate of each area is adjusted in real time through the regional adjustment module;

[0029] Step 4: Control the overall refresh rate and brightness of the screen.

[0030] The present invention provides a screen refresh rate intelligent adjustment system based on content analysis. First, through the multimodal data acquisition module, the system collects the user's physiological data in real time, such as heart rate variability (HRV), galvanic skin response (GSR), skin temperature and eye movement signals. After preprocessing, these data are input into the data processing and fusion module. The data processing and fusion module uses a high-precision fusion algorithm to process these physiological data to generate the user's fatigue prediction results and attention concentration. These processing results provide accurate basic data for the subsequent screen refresh rate adjustment.

[0031] The user status data generated by the data processing and fusion module is passed to the content perception module. The content perception module analyzes the displayed content type (such as text, video, game) according to the user's fatigue status and real-time screen content, and preliminarily sets the refresh rate adjustment parameters for each area. Then, the content perception module passes these parameters together with the user's real-time physiological state to the regional adjustment module. The regional adjustment module adjusts the refresh rate of each screen area in real time based on a predetermined dynamic weight algorithm and combined with the user's line of sight focus data, thereby ensuring that the display effects of different areas match the user's needs.

[0032] As the core coordination unit of the system, the control module receives and analyzes the adaptive response curve (ARC) generated by the data processing and fusion module. By analyzing the ARC, the control module adjusts the overall refresh rate and brightness of the screen in real time to adapt to the user's current state and reduce visual fatigue. During the adjustment process, the control module also introduces double buffering technology and DMA (direct memory access) technology to improve screen refresh efficiency and reduce the burden on the main control processor. Through these optimization technologies, the system can improve the smoothness of screen refresh without increasing the processor load.

[0033] The task management unit monitors the user's operation behavior (such as keyboard input, mouse clicks, window switching), uses the support vector machine (SVM) algorithm to identify the type of task currently running, and dynamically adjusts the refresh rate of each area according to the priority. For example, in a multi-tasking environment, the task management unit prioritizes the refresh rate of video conferencing and important document editing areas, and reduces the refresh rate of background tasks to save system resources. The task management unit works closely with the control module to intelligently predict the user's operation trend and adjust the refresh rate of related areas in advance to ensure a seamless transition of the display effect when switching tasks.

[0034] The present invention provides a screen refresh rate intelligent adjustment system and method based on content analysis. By adjusting the screen refresh rate and brightness in real time, the system can dynamically optimize the display effect of the screen according to the user's fatigue state and the current usage scenario, thereby significantly reducing visual fatigue, and is particularly suitable for users who use electronic devices for a long time. The system adjusts the refresh rate of each area according to the screen content type and the user's attention distribution, reduces the refresh rate of non-core areas, reduces unnecessary power consumption, and achieves energy-saving effects.

[0035] The multimodal data acquisition module collects the user's physiological data (such as eye movement signals, heart rate variability, skin electrical response, etc.) in real time, and generates the user's fatigue prediction results and attention concentration through data fusion, making the screen refresh rate adjustment more intelligent and accurate.

[0036] The present invention also dynamically adjusts the refresh rates of different areas according to the user's current task type and priority through the task management unit to ensure the display effect of the key task area, while reducing the refresh rate of the background task and optimizing the utilization of system resources. The present invention introduces double buffering technology and DMA technology to reduce the burden on the main control processor, improve the smoothness of screen refresh, and ensure seamless transition of display effects when users switch tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a structural schematic diagram of a screen refresh rate intelligent adjustment system based on content analysis of the present invention;

[0038] Figure 2 It is a structural schematic diagram of a multimodal data acquisition module of the present invention;

[0039] Figure 3 It is a structural schematic diagram of the data processing and fusion module of the present invention;

[0040] Figure 4 A schematic diagram of the structure of a predetermined dynamic weight algorithm of the present invention;

[0041] Figure 5 It is a structural schematic diagram of the content perception module and the regionalization adjustment module of the present invention;

[0042] Figure 6 It is a structural schematic diagram of the control module of the present invention;

[0043] Figure 7 It is a schematic diagram of a method for intelligently adjusting screen refresh rate based on content analysis of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0046] Example 1: Please refer to Figure 1 and Figure 2 In a possible implementation, the screen refresh rate intelligent adjustment system based on content analysis provided in this embodiment realizes intelligent dynamic adjustment of the screen refresh rate through the organic combination of a multimodal data acquisition module, a data processing and fusion module, a content perception module, a regional adjustment module and a control module, so as to optimize the user's visual experience and reduce visual fatigue.

[0047] The multimodal data acquisition module in the embodiment of the present invention collects various physiological data of the user, such as eye movement signals, heart rate variability (HRV), galvanic skin response (GSR), and skin temperature, through the sensing device. The above physiological data not only provide important information related to the user's fatigue state, but also realize the coordinated processing of multi-source data through the combination with the data processing and fusion module.

[0048] Specifically, the system of the embodiment of the present invention can be applied not only to fixed display devices, but also to portable devices such as smart phones or tablets. Through built-in or external sensors, the device can support real-time detection of eye movement signals, heart rate variability, galvanic skin response and skin temperature. Since galvanic skin response (GSR) requires direct contact with the skin, if the user does not hold the device, the system can continue to complete the detection through an external wearable device (such as a smart bracelet, finger ring), and transmit the data to the display device through a wireless connection, ensuring that the method of the present invention can be successfully executed under any circumstances.

[0049] The sensing device comprises:

[0050] (1) Eye tracking sensor combining laser scanning and high-frame rate camera: Through the combination of laser scanner and high-frame rate camera, pupil movement data is collected in real time to obtain the user's eye movement signals, including blinking frequency, pupil dilation and gaze trajectory, so as to monitor the user's visual concentration and attention changes.

[0051] (2) Photoplethysmography (PPG) heart rate variability sensor combined with electrode patch: Through a miniature photoplethysmography sensor and electrode patch, multi-band heart rate signals are collected and heart rate variability (HRV) is analyzed, including low-frequency and high-frequency HRV data, to evaluate the user's autonomic nervous system status.

[0052] (3) Near-infrared spectroscopy (NIRS) brain oxygen saturation monitoring sensor: An integrated miniature near-infrared spectroscopy sensor monitors the blood oxygen saturation of the cerebral cortex in a non-invasive manner, providing local brain hemodynamic information for identifying the user's cognitive load and mental fatigue status.

[0053] (4) Galvanic Skin Response (GSR) Sensor: A highly sensitive sensor is used to detect changes in skin conductance and collect the user's galvanic skin response data in real time to reflect emotional fluctuations and stress levels, which serves as an important parameter for assessing fatigue status.

[0054] (5) High-precision thermistor temperature sensor: An embedded high-precision thermistor is used to collect tiny changes in skin temperature in real time, and combined with other physiological data, it can more accurately assess the user's overall fatigue status.

[0055] The data processing and fusion module in this embodiment performs fusion processing on the physiological data from the multimodal data acquisition module based on a high-precision fusion algorithm. Through fusion processing, the system can dynamically adjust the weights of different physiological signals, generate high-precision user fatigue prediction results, and provide an accurate basis for subsequent screen refresh rate adjustment.

[0056] The content perception module is connected to the data processing and fusion module, and obtains the user's fatigue prediction results from it. At the same time, the content perception module identifies different types of content (such as text, video, games, etc.) through real-time analysis of the screen display content, and generates regional refresh rate adjustment parameters based on the user's interaction frequency. The above parameters not only take into account the user's fatigue state, but also reflect the complexity of the screen content and the user's focus area.

[0057] The regional adjustment module is connected to the content perception module and the data processing and fusion module. This module dynamically adjusts the refresh rate of each area of ​​the screen based on the parameters generated by the content perception module and the user's fatigue prediction results. The regional adjustment module uses regional classification and a predetermined dynamic weight algorithm to provide a higher refresh rate in the core area to ensure the best visual effect, while reducing the refresh rate in the peripheral area to save resources. In this way, the system can fine-tune each area of ​​the screen according to the real-time needs of the user.

[0058] As the core coordination part of the system, the control module is not only responsible for coordinating the data acquisition of the multimodal data acquisition module, the calculation processing of the data processing and fusion module, and the content analysis of the content perception module, but also receives the adaptive response curve generated by the data processing and fusion module, and adjusts the overall refresh rate and brightness of the screen in real time. In this way, the refresh rate and brightness of the screen can be adjusted in real time. Through the coordination of the control module, the functions of each module can be organically combined, thereby achieving the overall technical effect of the system.

[0059] Example 2: Please refer to Figure 3 In a possible implementation, this embodiment provides a data processing and fusion module based on a signal processing and fusion mechanism, which is used to dynamically adjust the weights from different physiological signals according to the real-time state of the user to improve the accuracy and sensitivity of fatigue prediction.

[0060] The data processing and fusion module is connected to the multimodal data acquisition module to obtain various physiological signal data of the user, including but not limited to eye movement signals, heart rate variability (HRV), galvanic skin response (GSR) and skin temperature, etc. The built-in signal processing and fusion mechanism of the module can dynamically adjust the importance of different physiological signals, so that the system can optimize the accuracy of fatigue prediction according to the real-time status of the user.

[0061] The signal processing and fusion mechanism includes:

[0062] (1) Multimodal signal preprocessing unit: Apply adaptive filtering algorithms (such as LMS filters) to process multimodal physiological signals collected in real time, dynamically adjust the filter coefficients to effectively remove environmental noise and equipment interference, thereby improving the signal-to-noise ratio of the signal. Apply the Z-score normalization algorithm (Z-score) to each type of physiological signal to convert the signal data into a zero mean and unit variance form. Specifically, for each data point, use the formula Transformation is performed, where x is the original signal value, μ is the mean value of the signal, and σ is the standard deviation of the signal. This standardization process ensures that different signals have a unified processing scale, which is convenient for subsequent fusion processing.

[0063] (2) Feature extraction and analysis unit: The feature extraction and analysis unit processes the preprocessed physiological signals through the following steps: first, the principal component analysis (PCA) and independent component analysis (ICA) algorithms are applied to extract the blinking frequency and pupil dilation speed from the eye movement signal, the low frequency / high frequency ratio from the heart rate variability (HRV), the conductivity change amplitude from the galvanic skin response (GSR), and the small temperature fluctuations from the epidermal temperature data. Through these feature extraction algorithms, the data dimension is reduced and the most representative signal features are highlighted. Subsequently, the extracted signal features are analyzed in time series by combining real-time and historical data, and their changing trends are evaluated and the future signal trends are predicted to understand the user's fatigue changes in different time periods, providing a basis for subsequent dynamic weight adjustment.

[0064] (3) Real-time state evaluation and weight allocation unit: This unit combines the results of feature extraction and time series analysis to calculate the change rate and fluctuation range of each physiological signal feature in real time, and combines the user's historical data to evaluate the user's current physiological state. Based on the evaluation results, the Bayesian network or fuzzy logic control algorithm is used to dynamically adjust the weight allocation of each physiological signal. For example, when it is detected that the user stares at the screen for a long time, the system will increase the weight of eye movement signals and heart rate variability; when the user is in a relaxed state, the weight of skin electrical response and skin temperature will increase. This dynamic weight allocation mechanism enables the system to flexibly adapt to the user's real-time state and optimize the accuracy and sensitivity of fatigue prediction.

[0065] (4) Fusion and decision unit: This unit performs weighted fusion of the characteristic values ​​of each physiological signal according to the dynamically assigned weights to generate a comprehensive fatigue index. The fusion process can achieve complex nonlinear fusion through weighted averaging, weighted accumulation or neural networks to ensure that the comprehensive index accurately reflects the user's overall fatigue state. Subsequently, the comprehensive fatigue index undergoes multi-level decision analysis, such as using algorithms such as decision trees or support vector machines (SVMs), to generate the final fatigue prediction result. The prediction result can be directly used in the system's adaptive adjustment function, such as dynamically adjusting the screen refresh rate and brightness to optimize the user experience and reduce the accumulation of visual fatigue.

[0066] In summary, this embodiment can effectively improve the accuracy and sensitivity of fatigue prediction, ensure that the system can accurately respond to the user's real-time status, and dynamically adjust display parameters to optimize the user's visual experience and reduce fatigue accumulation.

[0067] Example 3: Please refer to Figure 4 and Figure 5 In one possible implementation, the content perception module in this embodiment is configured to analyze different types of content displayed on the screen, including text, video, and games, and determine the refresh rate adjustment parameters for each area. The content perception module cooperates with the regional adjustment module to adjust the refresh rate of each area of ​​the screen based on a predetermined dynamic weight algorithm, combined with the user's sight focus time and physiological data.

[0068] Specifically, the content perception module uses classification algorithms to classify screen content in real time, such as using the support vector machine (SVM) algorithm to distinguish text, video, and game content. For text content, the content perception module analyzes its static nature and uses a low-pass filter to determine a lower refresh rate parameter to reduce resource consumption; for video content, a frame rate detection algorithm is used to calculate the dynamic nature of the video and generate a higher refresh rate parameter to ensure smooth playback; for game content, especially high-speed action games, the content perception module uses an analysis method based on frame rate and user operation response to determine the highest level of refresh rate parameters to ensure timely response to user operations and smooth visual experience.

[0069] The content perception module not only generates refresh rate parameters according to the content type, but also adjusts the refresh rate of each area using a predetermined dynamic weight algorithm in combination with the user's sight focus time and physiological data. Specifically, the regional adjustment module allocates the refresh rate weight of each area in real time through a predetermined dynamic weight algorithm (such as a Bayesian network or a fuzzy logic control algorithm). The algorithm obtains the user's sight focus time data, identifies the area where the user stays on the screen the longest, and sets the area as a high-weight area. According to the dynamic weight algorithm, combined with the user's sight data and physiological state, the system automatically increases the refresh rate weight of the current area to ensure that the display effect meets the user's needs. At the same time, the module also combines the user's physiological data, such as heart rate variability (HRV) and galvanic skin response (GSR), to judge the user's fatigue state, and adjust the refresh rate of each area accordingly. For example, when the user's sight is focused on a certain video area for a long time, and the physiological data shows that the user is in a low fatigue state, the system will assign a higher refresh rate to enhance the visual experience; conversely, when the user's fatigue level is high, the system can reduce the refresh rate to reduce the visual burden.

[0070] Specifically, the predetermined dynamic weight algorithm refers to dynamically adjusting the refresh rate weight of each area according to the user's real-time status and content display requirements during system operation, which is specifically reflected in the following steps:

[0071] (1) Collection and preprocessing of input data: The user's physiological data, including heart rate variability (HRV), galvanic skin response (GSR), and skin temperature, are acquired through the multimodal data acquisition module. After filtering and standardization, these data are used as input parameters for weight adjustment. The system uses eye tracking technology to acquire the user's gaze data, such as gaze dwell time and gaze trajectory, and identifies the core area, sub-core area, and peripheral area of ​​the user's gaze.

[0072] (2) Analysis of content types and preliminary weight setting: The content perception module analyzes the content types displayed on the screen (such as text, video, and games) and generates preliminary refresh rate adjustment parameters based on the dynamics and complexity of the content. For example, static text content generates a low weight, while dynamic video content generates a high weight. Based on the analysis results of the content perception module, the system preliminarily assigns weight values ​​to each area. For example, the refresh rate weight of the core area is set to 0.7, the secondary core area is set to 0.2, and the peripheral area is set to 0.1.

[0073] (3) Real-time status evaluation and weight adjustment: Based on the user's real-time physiological data and sight data, the system evaluates the user's fatigue status and visual concentration. The fuzzy logic control algorithm is used to process these data and calculate the user's attention distribution in different areas. Based on the real-time evaluation results, the refresh rate weight of each area is dynamically adjusted through the Bayesian network or fuzzy logic control algorithm. For example, when the user stares at a core area for a long time, the system will increase the weight value of the area to 0.8, and the weights of other areas will be reduced accordingly.

[0074] (4) Fusion and application: The dynamically adjusted weights are weighted and fused with the initially generated refresh rate parameters to obtain the final refresh rate adjustment parameters. Based on the weighted fusion results, the system adjusts the refresh rate of each area in real time to ensure that the display effect matches the user's real-time needs.

[0075] Example 4: Please refer to Figure 4 and Figure 5 The regional adjustment module provided in this embodiment includes a configuration structure for dividing the screen into multi-level areas, and the multi-level areas include a core area, a sub-core area, and a peripheral area. The core area refers to the main visual area where the user's sight most often stays, the sub-core area is located around the core area, and the peripheral area is the edge area where the user's sight pays less attention. The regional adjustment module independently adjusts the refresh rate of each area based on the analysis results of the content perception module and the user's fatigue status.

[0076] Specifically, the regional adjustment module first analyzes the user's gaze focus data based on a clustering algorithm (such as the K-means algorithm) to determine the core area, sub-core area, and peripheral area on the screen that are most frequently gazed at. Based on these area divisions, the regional adjustment module receives the refresh rate adjustment parameters generated by the content perception module, and further combines the user's physiological feedback data (such as analyzing the user's fatigue state through the heart rate variability (HRV) monitoring algorithm and the galvanic skin response (GSR)) to dynamically adjust the refresh rate of each area.

[0077] During the adjustment process, the regional adjustment module adopts a multi-level weight allocation algorithm. First, the system assigns a higher refresh rate weight to the core area to ensure the visual experience of the user's main focus area; the secondary core area is assigned a medium refresh rate weight based on the user's line of sight and the dynamic nature of the content; the peripheral area is usually assigned a lower refresh rate weight to save resources due to its lower visual importance. In addition, the module also combines the fuzzy logic control algorithm to dynamically adjust the refresh rate of each area according to the user's fatigue state. For example, when it is detected that the user's fatigue state is aggravated, the system will reduce the refresh rate of the core area to reduce the burden on the eyes; on the contrary, when the user is focused, the system will increase the refresh rate of the core area to ensure the clarity and fluency of the content.

[0078] Example 5: Please refer to Figure 6 In a possible implementation, the control module includes a control unit for receiving an adaptive response curve (ARC) generated by a data processing and fusion module, and the control unit is configured to adjust the refresh rate and brightness of the screen in real time. The control module also includes a task management unit, which is configured to adjust the refresh rate of different areas according to the user's task type and priority, and give priority to ensuring the display effect of key task areas in multi-task scenarios.

[0079] Specifically, the control unit analyzes the user's fatigue status and visual demand information provided by ARC, and uses rule-based algorithms (such as PID control algorithms) to optimize the screen display parameters in real time. The system introduces machine learning algorithms such as support vector machines (SVM) to analyze the user's interactive behavior and physiological state, dynamically adjust the refresh rate, and adjust the screen's refresh rate and brightness in real time to ensure that the GRAM reading speed of the LCD driver is increased without increasing the main control performance and memory consumption, thereby improving the smoothness of the screen refresh.

[0080] Specifically, the control unit separates the GRAM data reading task from the main processor by introducing DMA (direct memory access) technology, and directly transmits data through the DMA channel. This method avoids excessive involvement of the main processor in the data transmission process, reduces the CPU load, and speeds up data reading. In order to further optimize the smoothness of screen refresh, the control unit also integrates double buffering technology and cache optimization strategy, so that data reading and display operations can be performed in parallel. Double buffering technology sets two independent display buffers, so that when one buffer is refreshing the display data, the other buffer can synchronously read the next frame of data from GRAM, thereby avoiding mutual interference between data transmission and display operations, and ensuring that there is no jamming during the screen refresh process.

[0081] In addition, the control unit dynamically adjusts the refresh rate through intelligent algorithms to adapt to different content types and usage scenarios. For example, when processing static content, the system can reduce the refresh rate to reduce resource consumption; while in dynamic content such as video or game scenarios, the system automatically increases the refresh rate to ensure the smoothness of the picture. All these optimization measures are aimed at improving the refresh effect of the screen while ensuring that the performance and memory usage of the main control processor are not over-consumed, thereby achieving efficient system resource management and excellent user visual experience.

[0082] In addition, the control module also includes a task management unit, which is configured to dynamically adjust the refresh rate of different areas of the screen according to the user's task type and priority, giving priority to ensuring the display effect of key task areas in multi-tasking scenarios. In a possible implementation, the specific implementation of the task management unit includes the following parts:

[0083] First, the task management unit monitors and analyzes the user's operation behavior and application status in real time based on the support vector machine (SVM) algorithm to identify the type of task currently being performed, such as office, entertainment, reading, etc. By extracting the user's keyboard input, mouse clicks, window switching frequency and other operation features, the task type is accurately classified to obtain the task identification result.

[0084] Secondly, according to the task identification results, the corresponding priority is assigned to each task. For example, video conferencing or important document editing tasks are set as high priority to ensure that these key task areas can obtain a higher refresh rate and brightness to ensure smooth and clear display effects; for background tasks or non-critical windows, a lower refresh rate is assigned to save system resources.

[0085] In addition, the task management unit will dynamically adjust the refresh rate of each area according to the set priority and user line of sight data. By using the prediction mechanism, by analyzing the user's operation trends (such as mouse hovering, line of sight lingering), the task window that the user may activate is identified in advance, and its refresh rate is adjusted in advance to ensure a seamless transition of the display effect when switching tasks.

[0086] Finally, the task management unit also adjusts the refresh rate allocation strategy in real time according to the system resource usage. For example, when it detects that the system resources are tight, it reduces the refresh rate of low-priority tasks to prioritize the smooth display of high-priority tasks. At the same time, the task management unit continuously optimizes the refresh rate adjustment strategy through user interaction feedback (such as mouse clicks and keyboard input frequency) to ensure that users always get the best visual experience in multi-tasking operations. Through the collaborative work of the above modules, the task management unit achieves efficient resource allocation and intelligent display effect management.

[0087] Example 6: Please refer to Figure 7In a possible implementation, a method for intelligently adjusting screen refresh rate based on content analysis is provided, and the method is applied to the above system. The method comprises the following steps:

[0088] Step 1: Acquire the user's physiological data, and analyze and process the physiological data through a data processing and fusion module to generate the user's fatigue prediction result and sight focus area;

[0089] Step 2: Analyze the content type currently displayed on the screen through the content analysis module, identify the content displayed in each area of ​​the screen, and generate refresh rate adjustment parameters for each area based on the user's usage scenario;

[0090] Step 3: According to the user's sight focus time and fatigue state, the refresh rate of each area is adjusted in real time through the regional adjustment module;

[0091] Step 4: Combine the refresh rate adjustment parameters of each area and the user's overall fatigue state to dynamically control the overall refresh rate of the screen and adjust the screen brightness accordingly.

[0092] The method first evaluates the user's real-time fatigue status by acquiring the user's physiological data, such as heart rate, eye movement trajectory, and brain wave signals. The data is comprehensively analyzed through the data processing and fusion module to generate fatigue prediction results. Specifically, the method will predict the user's fatigue level and identify the key factors that may cause visual fatigue by continuously monitoring physiological data and comparing historical data.

[0093] On this basis, the method also analyzes the type of content displayed on the screen and distinguishes the content characteristics of each display area. Through the content analysis module, the method can identify parameters such as image or video content in different areas, the complexity of text content, the frequency of dynamic changes, and color contrast. Based on these content characteristics, the refresh rate adjustment parameters for each area are generated. For example, for areas with more frequent dynamic changes, the method will recommend a higher refresh rate, while for static content areas, it will recommend lowering the refresh rate to reduce energy consumption.

[0094] In addition, the method uses a regional adjustment module to adjust the refresh rate of each area in real time according to the user's line of sight focus time and fatigue state. When the user's line of sight is focused on a specific area for a long time, the method will prioritize the refresh rate of the area to ensure the clarity and smoothness of the display effect. At the same time, by reducing the refresh rate of other non-focused areas, energy saving and unnecessary visual stimulation can be achieved. In addition, the method will adjust the brightness and refresh rate of the overall screen in a timely manner according to the user's fatigue state. When it is detected that the user's fatigue level is increasing, the method will actively reduce the screen brightness or the overall refresh rate to reduce the visual burden and slow down the speed of fatigue.

[0095] Through the above intelligent refresh rate adjustment method, the method can significantly optimize the user's visual experience and reduce the visual fatigue caused by long-term use of screen devices. The advantage of this method is that it combines the feedback of the user's physiological data and the analysis of the displayed content, thereby realizing a dynamic and accurate screen adjustment strategy, allowing the user to maintain a comfortable state for a longer period of time during use, which is particularly suitable for users who engage in activities such as reading, video watching or games for a long time.

[0096] Embodiment 7: In a specific application scenario, the screen refresh rate intelligent adjustment system and method of the present invention are used to optimize the user's visual experience and system resource management efficiency in a multi-tasking office environment. This scenario involves how the various modules of the system work together to ensure the best display effect and resource utilization when the user is simultaneously processing document editing, participating in video conferences, and browsing web pages.

[0097] 1. The role of the multimodal data acquisition and fusion module: The system obtains a variety of user physiological data in real time through the multimodal data acquisition module, including heart rate variability (HRV), galvanic skin response (GSR) and skin temperature. After filtering and standardization, these data are input into the data processing and fusion module. At the same time, the system captures the user's line of sight data through eye tracking technology to identify the area that the user pays most attention to on the screen. Based on these physiological data and line of sight data, the data processing and fusion module generates the user's fatigue status and attention distribution information, providing an accurate basis for the subsequent refresh rate adjustment of each area.

[0098] 2. Content perception and regional refresh rate adjustment: The content perception module analyzes the content type displayed on the screen in real time and uses the support vector machine (SVM) algorithm to classify the content, such as text, video, and web pages. The system sets different refresh rate adjustment strategies for different content. For example, for the document editing window, the system recognizes it as static content and sets a lower refresh rate for the area through a low-pass filter to reduce resource consumption; for the video conferencing window, the system recognizes it as dynamic content and generates a higher refresh rate parameter to ensure the smoothness of video playback; for the web browsing window, the system appropriately adjusts its refresh rate according to the dynamic nature of the page to ensure the timeliness of information presentation.

[0099] 3. Collaboration between regionalized adjustment and real-time weight allocation: The regionalized adjustment module uses a dynamic weight algorithm to adjust the refresh rate of each area in real time according to the user's line of sight focus time and physiological data. The task management unit identifies the priority of the task, sets the document editing and video conferencing windows as high-priority task areas, and assigns higher refresh rates to these areas. For example, when a user stares at a video conferencing window for a long time, the system automatically increases the refresh rate weight of the area to ensure the clarity and smoothness of the video. At the same time, the web browsing area is identified as a low-priority task, and the system reduces its refresh rate to optimize resource utilization.

[0100] 4. Adaptive adjustment of the control module: The control module dynamically adjusts the refresh rate and brightness of the entire screen based on the adaptive response curve (ARC) generated by the data processing and fusion module. When the user is highly concentrated and less fatigued when editing a document, the system increases the refresh rate and brightness of that area to ensure clear presentation of the text; if the physiological data indicates that the user's fatigue level is increasing, the system lowers the overall brightness and reduces the refresh rate of unnecessary areas to reduce the visual burden. Through this adaptive adjustment mechanism, the system can effectively balance the user's visual needs and resource consumption.

[0101] 5. Intelligent prediction and optimization of task switching: The task management unit further predicts the task window that the user may switch to in advance by analyzing the user's mouse hovering position and eye contact pattern. For example, when it is detected that the user's eyes frequently switch between the document editing and video conferencing windows, the system increases the refresh rate of the video conferencing window in advance to ensure smooth display without lag when switching tasks. At the same time, the refresh rate of the web browsing window is appropriately reduced to achieve reasonable allocation of resources.

[0102] Through the above specific application scenarios, the system provided in this embodiment can not only optimize the display effect of each task in real time to meet the user's visual needs, but also effectively manage system resources to ensure the smooth display of key tasks, thereby significantly improving the user's overall office experience.

[0103] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A screen refresh rate intelligent adjustment system based on content analysis, characterized in that: The system comprises: A multimodal data acquisition module, the multimodal data acquisition module comprising an eye movement sensor for collecting user eye movement signals, a sensor for collecting heart rate variability, a sensor for collecting skin electrical response, and a temperature sensor for collecting skin temperature; A data processing and fusion module, connected to the multimodal data acquisition module, is used to process and fuse physiological data and generate fatigue prediction results and attention concentration of the user; The content perception module is connected to the data processing and fusion module and is used to analyze the display content type and generate refresh rate adjustment parameters for each area according to the user's fatigue status and real-time screen content; A regional adjustment module, connected to the content perception module and the data processing and fusion module, is used to adjust the refresh rate of each area of ​​the screen in real time based on a dynamic weight algorithm; The control module is used to receive the adaptive response curve generated by the data processing and fusion module, and adjust the overall refresh rate and brightness of the screen in real time; The data processing and fusion module includes: A multimodal signal preprocessing unit that processes physiological signals using an adaptive filtering algorithm; A feature extraction and analysis unit, used to extract feature data related to the user's fatigue state from the processed physiological signals; A real-time state assessment and weight allocation unit for dynamically adjusting the weights of physiological signals; The fusion and decision-making unit is used to generate the user's fatigue prediction results and attention concentration. The fatigue prediction results are achieved by fusing eye movement data, heart rate variability data, and skin electrical response data, combined with a time series analysis model; The multimodal signal preprocessing unit applies an adaptive filtering algorithm to process the multimodal physiological signals collected in real time, and dynamically adjusts the filter coefficients; applies a Z-score normalization algorithm (Z-score) to each type of physiological signal to convert the signal data into a form of zero mean and unit variance; for each data point, uses the formula Transform, where x is the original signal value, μ is the mean of the signal, and σ is the standard deviation of the signal; The feature extraction and analysis unit processes the preprocessed physiological signals through the following steps: first, applying principal component analysis (PCA) and independent component analysis (ICA) algorithms to extract blink frequency and pupil dilation speed from eye movement signals, extract low frequency / high frequency ratio from heart rate variability (HRV), extract conductivity variation amplitude from galvanic skin response (GSR), and extract tiny temperature fluctuations from epidermal temperature data; then, combining real-time and historical data, using time series analysis methods to perform time series analysis on the extracted signal features; The real-time state evaluation and weight distribution unit calculates the change rate and fluctuation range of each physiological signal feature in real time by combining the results of feature extraction and time series analysis, and evaluates the user's current physiological state by combining the user's historical data; based on the evaluation results, the Bayesian network or fuzzy logic control algorithm is used to dynamically adjust the weight distribution of each physiological signal; The fusion and decision unit performs weighted fusion on the characteristic values ​​of each physiological signal according to dynamically assigned weights to generate a comprehensive fatigue index; the fusion process achieves complex nonlinear fusion through weighted averaging, weighted accumulation or neural network, and the comprehensive fatigue index generates the final fatigue prediction result through multi-level decision analysis.

2. The screen refresh rate intelligent adjustment system based on content analysis according to claim 1 is characterized in that: The eye movement sensor is a high frame rate camera; The sensor for collecting heart rate variability adopts photoplethysmography and electrode patches; The sensors used to collect galvanic skin responses are highly sensitive electrodes; The temperature sensor is a high-precision thermistor.

3. The screen refresh rate intelligent adjustment system based on content analysis according to claim 1 is characterized in that: The feature extraction and analysis unit extracts pupil dilation velocity, HRV low-frequency / high-frequency ratio, skin conductivity variation and epidermal temperature fluctuation respectively through principal component analysis and independent component analysis algorithms.

4. The screen refresh rate intelligent adjustment system based on content analysis according to claim 1 is characterized in that: The content perception module analyzes the type of content displayed on the screen based on a support vector machine algorithm and generates corresponding refresh rate adjustment parameters.

5. The screen refresh rate intelligent adjustment system based on content analysis according to claim 1 is characterized in that: The regional adjustment module adjusts the refresh rate of each area of ​​the screen in real time according to the user's sight focus time and physiological data through a predetermined dynamic weight algorithm.

6. The screen refresh rate intelligent adjustment system based on content analysis according to claim 5 is characterized in that: The regional adjustment module includes a multi-level regional division structure, which divides the screen into a core area, a sub-core area and a peripheral area, and sets a refresh rate weight for each area.

7. The screen refresh rate intelligent adjustment system based on content analysis according to claim 1 is characterized in that: The control module also includes a task management unit, which is used to dynamically adjust the refresh rates of different areas of the screen according to the user's task type and priority.

8. The screen refresh rate intelligent adjustment system based on content analysis according to claim 7 is characterized in that: The task management unit monitors and analyzes the user's operation behavior and application status in real time through a support vector machine algorithm to identify the type of task currently being executed and allocate a corresponding refresh rate according to priority.

Images