Screen control method and device, equipment and medium

Abstract

The invention discloses a screen control method and device, equipment and a medium, and relates to the technical field of vehicle-mounted screens, and the method comprises the following steps: obtaining biological characteristic data of a target object; determining a current state of the target object based on the biological characteristic data; and adjusting the target screen area based on the current state. According to the method, the user state is comprehensively judged by using the multi-dimensional biological characteristic data, so that the accuracy and robustness of state recognition are improved, and misjudgment and limitation possibly existing in a single data source are overcome. On the basis, by adjusting the screen display attribute, the information transmission efficiency can be effectively improved, and the distribution of screen resources is optimized while the key information is ensured to be preferentially displayed to improve the safety, so that more personalized interaction experience meeting the current demand is provided for the user.

Description

Technical Field

[0001] This application relates to the field of vehicle screen technology, and in particular to a screen control method, device, equipment and medium. Background Technology

[0002] With the rapid development of smart cockpit technology, the function of in-vehicle screens has expanded from simple information display to multi-tasking and personalized interaction. Currently, most in-vehicle systems use fixed split-screen layouts or rely on manual user adjustments, failing to adapt to the needs of passengers. This is especially problematic in complex environments with multiple users or dynamically changing user states, where the screen cannot perceive user intentions and states in real time, resulting in low interaction efficiency. Therefore, a screen control method is urgently needed to address these issues. Summary of the Invention

[0003] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solutions, nor is it intended to determine the scope of protection of the claimed technical solutions.

[0004] In a first aspect, this application provides a screen control method, including: Obtain biometric data of the target object; Based on the biometric data, the current state of the target object is determined; Based on the current state, the target screen area is adjusted.

[0005] In some implementations, the biometric data includes eye-tracking data, heart rate monitoring data, skin conductance data, and voice sampling data.

[0006] In some implementations, determining the current state of the target object based on the biometric data includes: Based on the eye-tracking data, the duration of gaze deviation and blink frequency of the target object are determined; Based on the heart rate monitoring data, the heart rate value and heart rate variability data of the target object are determined; Based on the skin conductance data, the skin conductance waveform characteristics of the target object are determined; Based on the speech sampling data, the fundamental frequency value and speech rate value of the target object are determined; The current state of the target object is determined by analyzing the duration of gaze deviation, blink frequency, heart rate value, heart rate variability data, skin conductance waveform characteristics, speech fundamental frequency value, and speech rate value.

[0007] In some implementations, the target object is a driver, and the current state includes a distracted state, a fatigued state, a tense state, or an excited state. The process of analyzing the duration of gaze deviation, blink frequency, heart rate value, heart rate variability data, skin conductance waveform characteristics, voice fundamental frequency value, and speech rate value to determine the current state of the target object includes: When the duration of the gaze deviation exceeds a first preset duration, the driver is determined to be in a distracted state. When the blinking frequency is less than a preset frequency and the heart rate is less than a preset heart rate for a second preset duration, the driver is determined to be in a state of fatigue. When the heart rate variability data is less than a preset variability threshold, or the peak amplitude of the skin conductance waveform is greater than a preset amplitude threshold, the driver is determined to be in a state of tension. When the fundamental frequency value of the speech is greater than a preset fundamental frequency threshold, or the speech rate value is greater than a preset speech rate threshold, the driver is determined to be in an excited state.

[0008] In some implementations, the target object is a passenger, and the current state includes a fatigued state, a tense state, or an excited state. The current state of the target object is determined by analyzing the blink frequency, the heart rate variability data, the skin conductance waveform characteristics, the fundamental frequency value of the speech, and the speech rate value, including: When the blinking frequency is less than the preset frequency, and the frequency domain energy of the skin conductance waveform characteristics is in the preset fatigue frequency band, the passenger is determined to be in a fatigued state. When the heart rate variability data is less than the preset variability threshold, or the peak amplitude of the skin conductance waveform feature is greater than the preset amplitude threshold, the passenger is determined to be in a state of tension. When the voice base frequency value is greater than the preset base frequency threshold, or the speech rate value is greater than the preset speech rate threshold, the passenger is determined to be in an excited state.

[0009] In some implementations, the adjustment operation of the target screen area based on the current state includes: Based on the target object, determine the target screen area to be adjusted; Based on the current state and the target screen area, determine the corresponding display adjustment strategy; Based on the aforementioned display adjustment strategy, the display attributes of the target screen area are adjusted.

[0010] In some implementations, it also includes: When there are multiple target objects, the adjustment priority of each target object is determined based on its type; The adjustment operation of the target screen area based on the current state includes: Based on the current state of each target object and the adjustment priority, the target screen area is adjusted.

[0011] Secondly, this application proposes a screen control device, comprising: The data acquisition unit is used to acquire the biometric data of the target object; A state recognition unit is used to determine the current state of the target object based on the biometric data and a preset state recognition model. The screen control unit is used to perform adjustment operations on the target screen area based on the current state.

[0012] Thirdly, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program stored in the memory to implement the screen control method of any of the first aspects.

[0013] Fourthly, this application also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the screen control method of any of the first aspects.

[0014] In summary, the screen control method provided in this application determines the current state of the target object by acquiring and analyzing its biometric data, and adaptively adjusts the screen area accordingly, achieving linkage between screen display and user state. This method utilizes multi-dimensional biometric data to comprehensively judge the user's state, improving the accuracy and robustness of state recognition and overcoming the potential misjudgments and limitations of a single data source. Furthermore, by adjusting screen display attributes, the efficiency of information transmission can be effectively improved. While ensuring the priority display of critical information to enhance security, it also optimizes the allocation of screen resources, thereby providing users with a more personalized interactive experience tailored to their current needs, and enhancing the intelligence level of human-computer interaction and user experience. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a schematic flowchart of a screen control method provided in an embodiment of this application; Figure 2This is a schematic diagram of a screen control device provided in an embodiment of this application; Figure 3 This is a schematic diagram of a screen control electronic device provided in an embodiment of this application. Detailed Implementation

[0016] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0017] Please see Figure 1 This is a schematic flowchart of a screen control method provided in an embodiment of this application, which may specifically include: S110. Obtain the biometric data of the target object; For example, in an in-vehicle environment, non-invasive biosensors deployed in different locations within the vehicle continuously collect multi-dimensional biometric data of the target subject, including eye movements, heart rate, skin conductance, and voice. These sensors employ a low-power design and transmit data via encrypted communication.

[0018] S120. Determine the current state of the target object based on biometric data; For example, based on the acquired multidimensional biometric data, the physiological and emotional states of the target object are analyzed through a pre-defined state recognition model. This process first preprocesses various biometric data to eliminate environmental interference, then extracts key features, and utilizes an analysis mechanism that integrates a rule engine and machine learning algorithms to accurately identify various states of the user, such as distraction, fatigue, tension, or excitement.

[0019] S130. Based on the current state, perform adjustment operations on the target screen area.

[0020] For example, based on the currently identified user state, the target screen area that needs to be adjusted is determined, and the display attributes of the area are dynamically modified according to the preset adjustment strategy, thereby achieving adaptive control of the screen display content and layout, so that the screen output can adapt to the real-time changes in the user's state.

[0021] In summary, this application's embodiments acquire multidimensional biometric data of the target object and identify its current state based on this data, thereby adaptively adjusting the screen area to achieve linkage between screen display and user state. This method utilizes multi-source biometric data such as eye movement, heart rate, skin conductance, and voice for fusion analysis, improving the accuracy and robustness of state recognition and effectively overcoming the misjudgments and limitations of single data sources. By dynamically adjusting screen display attributes, it not only optimizes information presentation, ensuring key information is highlighted at appropriate times to improve driving safety, but also adaptively adjusts the interface layout and content according to the user's real-time physiological and emotional state, making screen interaction more aligned with the user's current needs. This enhances the intelligence and personalized experience of human-computer interaction, achieving efficient allocation of screen resources and improved user experience in the complex and ever-changing cockpit environment.

[0022] In some instances, biometric data includes eye-tracking data, heart rate monitoring data, skin conductance data, and voice sampling data.

[0023] For example, in an in-vehicle environment, non-invasive biosensors deployed in locations such as the dashboard area in front of the steering wheel, seat back, steering wheel or seat armrest, and microphone arrays continuously collect biometric data of the target subject. Specifically, infrared imaging technology is used to acquire eye-tracking data, including eye movement trajectory, blink frequency, and pupil diameter changes; contact or non-contact millimeter-wave sensors are used to collect heart rate monitoring data, including heart rate value and heart rate variability; skin conductivity sensors are used to record skin conductivity waveform characteristics; and voice sampling data is acquired through a voice acquisition device, and then the fundamental frequency and speech rate of the voice are analyzed. During the data collection process, the sensors adopt a low-power design and transmit encrypted data through the in-vehicle local area network to ensure user privacy and security. In the data preprocessing stage, the raw biometric data is subjected to noise reduction, filtering, and data cleaning to remove interference introduced by environmental factors such as vehicle bumps and changes in lighting.

[0024] In summary, the embodiments of this application improve the accuracy and robustness of state recognition by utilizing multi-dimensional biometric data such as eye movement, heart rate, skin conductance, and voice. This method overcomes the misjudgment problem that may be caused by individual differences or environmental interference from a single data source. Through cross-validation of multi-source information, it can more comprehensively and accurately capture changes in the user's physiological and emotional state, providing a data foundation for screen adaptive control, thereby enhancing the perception and decision-making capabilities of the above method in complex in-vehicle environments.

[0025] In some instances, the current state of a target object is determined based on biometric data, including: Based on eye-tracking data, determine the duration of gaze deviation and blink frequency of the target object; Based on heart rate monitoring data, determine the heart rate value and heart rate variability data of the target subjects; Based on skin electrical conductivity data, determine the skin electrical conductivity waveform characteristics of the target object; Based on speech sampling data, determine the fundamental frequency and speech rate of the target object; The current state of the target object is determined by analyzing the duration of gaze deviation, blink frequency, heart rate value, heart rate variability data, skin conductance waveform characteristics, speech fundamental frequency value, and speech rate value.

[0026] For example, eye-tracking data is used to determine the duration of gaze deviation and blink frequency of a target object. Infrared eye trackers deployed at specific locations within the vehicle continuously collect the user's eye movement trajectory. By analyzing the deviation of the eye coordinates relative to a preset reference area (such as the road ahead), the duration of gaze deviation from the reference area is calculated, i.e., the gaze deviation duration. Simultaneously, by detecting the cycle of eyelid opening and closing, the number of blinks per unit time is counted to obtain the blink frequency.

[0027] Heart rate monitoring data is used to determine the target subject's heart rate value and heart rate variability data. A millimeter-wave sensor integrated into the seat back allows for non-contact capture of the user's heartbeat signal. Time-domain analysis of this signal calculates the heart rate, the number of heartbeats per minute. Frequency-domain or nonlinear analysis of continuous heartbeat interval sequences extracts heart rate variability data, which effectively reflects the activity balance of the autonomic nervous system and serves as an indicator for assessing the user's psychological stress and emotional load.

[0028] It should be noted that the determination of the target object's heart rate value and heart rate variability data based on heart rate monitoring data is achieved through a non-contact millimeter-wave sensor deployed inside the seat back. This sensor emits millimeter-wave signals of a specific frequency band towards the target object's body surface and receives micro-Doppler phase-modulated echo signals caused by heartbeats and blood flow. First, the raw echo signal undergoes preprocessing, including eliminating respiratory harmonic interference through a bandpass filter and suppressing signal distortion caused by vehicle vibration using an adaptive noise reduction algorithm based on variational mode decomposition, thereby extracting the heartbeat harmonic components. In the time-domain analysis stage, a peak detection algorithm based on continuous wavelet transform is used to identify continuous heartbeat characteristic peaks in the preprocessed signal, and the time interval between adjacent characteristic peaks is calculated to obtain a heartbeat interval sequence. After averaging this sequence, dividing 60 seconds by the average heartbeat interval yields the number of heartbeats per minute for the target object, i.e., the heart rate value.

[0029] Based on the obtained inter-cardiac interval sequences, heart rate variability data were extracted using a combination of frequency domain analysis and nonlinear analysis. In the frequency domain analysis phase, power spectrum estimation based on Lomb-Scargle periodograms was performed on the inter-cardiac interval sequences to calculate their power spectral density in specific frequency bands. Specifically, the low-frequency band (0.04-0.15Hz) power spectral density was extracted as a representation of sympathetic nerve activity intensity, primarily reflecting mechanisms such as vasomotor regulation and thermoregulation; the high-frequency band (0.15-0.4Hz) power spectral density was extracted as a representation of parasympathetic nerve activity intensity, closely related to respiratory sinus arrhythmia. By calculating the ratio of low-frequency power to high-frequency power, the LF / HF ratio, reflecting the balance of the autonomic nervous system, was obtained. The level of this ratio directly reflects the tension balance between the sympathetic and parasympathetic nervous systems. For example, when users are in a state of tension or stress, sympathetic nerve activity increases while parasympathetic nerve activity decreases, and the LF / HF ratio usually increases significantly, possibly rising from 1.5-2.0 in the baseline state to above 3.0; conversely, in a state of deep relaxation or sleep, the ratio will drop significantly to below 1.0.

[0030] In the nonlinear analysis phase, the root mean square (RMSSD) of the difference between adjacent heartbeats is used to assess short-term changes in heart rate variability. This parameter primarily reflects the regulatory activity of the parasympathetic nervous system. Simultaneously, the standard deviations SD1 along the secondary diagonal and SD2 along the main diagonal are calculated based on a Poincaré scatter plot. SD1 represents the instantaneous fluctuations (short-term variability) of heart rate, while SD2 represents the long-term trend of heart rate changes. Finally, the LF / HF ratio, low-frequency power absolute value, high-frequency power absolute value obtained from frequency domain analysis, and the RMSSD, SD1, and SD2 obtained from nonlinear analysis are dimensionality-reduced and fused using principal component analysis. The first principal component score is extracted, or the principal component scores are weighted and synthesized to generate a scalar value. This value serves as the heart rate variability data for subsequent state determination, providing physiological state input for subsequent screen adjustments.

[0031] Determining the skin conductivity waveform characteristics of a target object based on skin conductivity data is achieved through a non-invasive electrode sensor installed on the steering wheel grip area or seat armrest surface of a vehicle. This sensor continuously monitors changes in the conductivity level of the user's skin surface at a preset sampling frequency, acquiring the raw skin conductivity signal. This physiological signal is primarily regulated by the activity of the sympathetic nervous system. When a user's arousal level increases due to emotional stress, cognitive load, or external stimuli, sympathetic nerve excitation triggers increased sweat gland secretion, causing a momentary change in the electrolyte concentration on the skin surface. Microscopically, this manifests as a rapid increase in conductivity value within seconds. After acquiring the raw skin conductivity signal, signal preprocessing is performed, including using a low-pass filter to eliminate high-frequency noise introduced by slight limb movements or vehicle vibrations, and using a baseline drift correction algorithm to eliminate trend interference caused by slow changes in ambient temperature and humidity, thus obtaining the skin conductivity response waveform.

[0032] Multidimensional features in the time and frequency domains are extracted from the preprocessed signal. In the time domain, a peak detection algorithm is used to identify the peak amplitude of the skin conductance response waveform within a specific time window; the peak amplitude characterizes the intensity of the response. In the frequency domain analysis, a Fast Fourier Transform is performed on the skin conductance signal to calculate its energy distribution in the 0.08-0.2 Hz frequency band, which is closely related to the mid-frequency rhythmic activity of the sympathetic nervous system. These quantified waveform features together constitute a multidimensional index characterizing the user's autonomic nervous system activity state, providing a physiological basis for subsequently assessing the user's level of tension, anxiety, or fatigue.

[0033] The fundamental frequency and speech rate of the target subject are determined based on speech sampling data. The user's speech signal is continuously acquired using a microphone array deployed in the headliner or dashboard. This microphone array has beamforming capabilities, effectively focusing on sound sources in specific seating areas and suppressing ambient noise. The acquired raw speech signal is first preprocessed, including using a Wiener filtering-based noise reduction algorithm to suppress steady-state background noise, employing a high-pass filter to eliminate low-frequency interference caused by vehicle vibration, and using pre-emphasis technology to enhance high-frequency components to improve the accuracy of subsequent analysis.

[0034] Based on preprocessing, the fundamental frequency of the speech is calculated using a fundamental frequency extraction algorithm. The fundamental frequency is detected using the autocorrelation function method, which combines the advantages of both the time and frequency domains. First, a short-time autocorrelation function is calculated for the framed speech signal. The pitch period is estimated by finding the peak position of the autocorrelation function, and then the fundamental frequency value is obtained. To ensure the smoothness and reliability of the fundamental frequency trajectory, median filtering and parabolic interpolation are used for post-processing to remove outliers caused by voiced / unvoiced transitions or noise interference. The final speech fundamental frequency value sequence reflects the variation pattern of vocal cord vibration frequency, and its statistical characteristics, such as mean and variance, are significantly correlated with the user's emotional arousal level.

[0035] Speech rate is calculated using speech endpoint detection and syllable segmentation techniques. A dual-threshold endpoint detection algorithm based on short-time energy and zero-crossing rate is employed to accurately identify the start and end positions of speech segments, eliminating the influence of silent and noisy segments. Within the effective speech segment, syllable boundary positions are identified by detecting the amplitude envelope peak of the speech signal and combining it with spectral abrupt change characteristics. The speech rate is obtained by counting the total number of syllables within the effective speech segment and dividing it by the total duration of the speech segment.

[0036] The above technical solution enables the acquisition of the fundamental frequency and speech rate values ​​of the voice, which reflect the user's emotional state. These two parameters, as acoustic features, are combined with biometric data obtained from other modalities such as eye movement, electrocardiogram, and skin conductance to form the data foundation for multi-dimensional state recognition, improving the comprehensiveness of identifying user states such as tension, excitement, or fatigue in complex in-vehicle environments.

[0037] The acquired data on gaze deviation duration, blink frequency, heart rate, heart rate variability, skin conductance waveform features, speech fundamental frequency, and speech rate are analyzed to determine the current state of the target object. This analysis process does not simply compare each parameter to a fixed threshold; instead, it employs a multimodal information fusion strategy combining a rule engine and a machine learning model. A pre-defined rule base for different states (such as distraction, fatigue, tension, and excitement) is established. For example, when the gaze deviation duration exceeds a first preset threshold, a preliminary determination of distraction is triggered. Simultaneously, a machine learning model (such as a support vector machine or deep learning network) trained on a large amount of data receives all these feature parameters as input and outputs a state probability distribution. The output of the rule engine and the inference results of the machine learning model are cross-validated and fused to ultimately generate a reliable judgment of the target object's current state. This fusion mechanism effectively utilizes the complementarity between different biometric signals, improving the accuracy and robustness of state recognition.

[0038] In summary, the embodiments of this application achieve user state perception through the aforementioned comprehensive state recognition method based on multi-source biometric data. This process fully utilizes various physiological and behavioral signals such as eye movement, heart rate, skin conductance, and voice. Through data fusion and intelligent analysis techniques, it overcomes the shortcomings of single signal sources being susceptible to interference and having limited representational information, providing a decision-making basis for subsequent adaptive screen adjustments, thereby laying the technical foundation for the intelligent and personalized screen control scheme.

[0039] In some instances, the target is a driver whose current state includes distraction, fatigue, tension, or excitement. Analysis of gaze deviation duration, blink frequency, heart rate, heart rate variability, skin conductance waveform characteristics, fundamental frequency of speech, and speech rate is used to determine the target's current state, including: When the driver's gaze deviates from the pre-set duration for more than the first preset duration, the driver is identified as being distracted. When the blinking frequency is less than the preset frequency and the heart rate is less than the preset heart rate for a second preset duration, the driver is determined to be in a state of fatigue. When the heart rate variability data is less than the preset variability threshold, or the peak amplitude of the skin conductance waveform is greater than the preset amplitude threshold, the driver is determined to be in a state of tension. When the voice fundamental frequency value is greater than the preset fundamental frequency threshold, or the speech rate value is greater than the preset speech rate threshold, the driver is determined to be in an excited state.

[0040] For example, when the target is a driver, and the current state includes distraction, fatigue, tension, or excitement, the system analyzes the duration of gaze deviation, blink frequency, heart rate, heart rate variability data, skin conductance waveform characteristics, fundamental frequency of speech, and speech rate to determine the driver's specific state. When determining whether the driver is distracted, the system continuously monitors their gaze direction and calculates the duration of gaze deviation from a preset reference area (e.g., the road ahead). When this gaze deviation duration is determined to be greater than a first preset duration threshold, the driver is identified as distracted. This first preset duration threshold is determined based on actual conditions and can be set to 5 seconds, aiming to identify distracting behaviors that may lead to safety hazards.

[0041] When the blinking frequency is less than a preset frequency and the heart rate is less than a preset heart rate for a sustained second preset duration, the driver is determined to be fatigued. First, the number of blinks per unit time is calculated. If this frequency is lower than a preset frequency threshold, such as a specific percentage or absolute value of the normal blinking frequency, it initially indicates signs of fatigue. Simultaneously, a heart rate monitoring module integrated into the seat back continuously measures the driver's heart rate. If the detected heart rate is lower than a preset heart rate threshold, such as 60 beats per minute, and this low heart rate state persists for more than a second preset duration, such as 5 minutes, then fatigue is confirmed. This judgment utilizes the changing characteristics of physiological signals in a fatigued state, namely, a decrease in blinking frequency and a drop in heart rate, thereby improving the reliability of state recognition.

[0042] The driver is identified as being under stress when heart rate variability data is below a preset threshold or when the peak amplitude of the skin conductance waveform exceeds a preset amplitude threshold. Heart rate variability data is obtained by performing frequency domain and nonlinear analysis on the heart rate interval sequence collected by the heart rate monitoring module, and then using principal component analysis to reduce the dimensionality of multiple extracted feature indicators or by weighted synthesis to obtain scalar values. If the fused heart rate variability data value is below the preset threshold, it indicates that the comprehensive indicator reflecting the balance of the autonomic nervous system deviates from the normal range, usually pointing to an imbalance between sympathetic and parasympathetic nerve activity caused by stress. Simultaneously, a skin conductance sensor located in the steering wheel grip area continuously monitors skin conductance levels. If the peak amplitude of the conductance response waveform detected within a specific time window exceeds a preset amplitude threshold, it indicates a significant increase in sweat gland secretion activity caused by sympathetic nerve excitation, a physiological response under stress. If either of these conditions is met, the driver is determined to be under stress.

[0043] When the fundamental frequency (FFM) of the speech exceeds a preset FFM threshold, or the speech rate exceeds a preset speech rate threshold, the driver is determined to be in an excited state. The driver's speech signal is acquired through an in-vehicle microphone array, and after preprocessing and feature extraction, the FFM and speech rate are obtained. If the calculated average FFM exceeds a preset FFM threshold (e.g., 180Hz), or the speech rate exceeds a preset speech rate threshold (e.g., 7.5 syllables per second), the driver is determined to be in an excited state. Changes in these two acoustic characteristics are highly correlated with the degree of emotional arousal; an increased FFM and faster speech rate are usually outward manifestations of emotional excitement, agitation, or tension.

[0044] In summary, this application, through the aforementioned analysis process based on multi-dimensional biometric data, can accurately identify the driver's states of distraction, fatigue, tension, and excitement. By setting physiological parameter thresholds and duration conditions, this method transforms physiological states into judgment logic, providing state input for subsequent screen adjustment operations. This enhances the accuracy of the in-vehicle system's perception of the driver's state and the rationality of its decisions, thus laying the foundation for improving driving safety and human-computer interaction experience.

[0045] In some instances, the target is a passenger whose current state includes fatigue, tension, or excitement. Analysis of blink frequency, heart rate variability, skin conductance waveform characteristics, fundamental frequency of speech, and speech rate is used to determine the target's current state, including: When the blinking frequency is less than the preset frequency and the frequency domain energy of the skin conductance waveform characteristics is in the preset fatigue frequency band, the passenger is determined to be in a fatigued state. When the heart rate variability data is less than the preset variability threshold, or the peak amplitude of the skin conductance waveform is greater than the preset amplitude threshold, the passenger is determined to be in a state of tension. When the voice base frequency value is greater than the preset base frequency threshold, or the speech rate value is greater than the preset speech rate threshold, the passenger is determined to be in an excited state.

[0046] For example, when the target is a passenger and the current state to be identified includes fatigue, tension, or excitement, the specific state is determined by analyzing the passenger's blink frequency, heart rate variability data, skin conductance waveform characteristics, fundamental frequency of speech, and speech rate. To determine if a passenger is fatigued, their blink frequency (the number of eyelid closures per unit time) is continuously monitored. If the calculated blink frequency is less than a preset frequency threshold, it initially suggests possible signs of fatigue. Simultaneously, skin conductance sensors installed on the seat armrests continuously collect skin conductance signals. After preprocessing and Fast Fourier Transform (FFT), the energy distribution within a specific frequency band is calculated. If this energy distribution is concentrated in a preset fatigue frequency band, for example, within the range of 0.08 to 0.2 Hz, and the energy value exceeds a preset energy threshold within that band, the physiological characteristics of fatigue are further confirmed. When both conditions are met—blink frequency below a preset frequency and the frequency domain energy of the skin conductance waveform characteristics matching the preset fatigue frequency band characteristics—the passenger is determined to be fatigued.

[0047] Determining whether a passenger is in a state of stress relies on the analysis of heart rate variability (HRV) data and skin conductance waveform characteristics. HRV data is obtained by performing frequency domain and nonlinear analysis on the heart rate interval sequences collected by the heart rate monitoring module, and then using principal component analysis to reduce dimensionality or weighted synthesis of multiple extracted feature indicators to obtain scalar values. If the fused HRV data value is lower than a preset variability threshold, it indicates that the comprehensive indicator reflecting the balance of the autonomic nervous system deviates from the normal range, typically pointing to an imbalance between sympathetic and parasympathetic nerve activity caused by stress. Simultaneously, skin conductance sensors located on the seat armrests continuously monitor skin conductance levels. If the peak amplitude of the conductance response waveform detected within a specific time window exceeds a preset amplitude threshold, it indicates a significant increase in sweat gland secretion activity caused by sympathetic nerve excitation. If either the HRV data is lower than the preset variability threshold or the peak amplitude of the skin conductance waveform characteristics exceeds the preset amplitude threshold, the passenger is determined to be in a state of stress.

[0048] The determination of a passenger's excitement level is achieved by analyzing their speech characteristics. Passenger speech signals are acquired via an in-vehicle microphone array. After preprocessing including noise reduction and pre-emphasis based on Wiener filtering, the fundamental frequency is detected using the autocorrelation function method. The fundamental frequency trajectory is then smoothed using median filtering and parabolic interpolation to obtain a stable sequence of fundamental frequency values. Simultaneously, a dual-threshold endpoint detection algorithm based on short-time energy and zero-crossing rate is used to identify valid speech segments. The speech rate is calculated by dividing the total number of syllables within each segment by the total duration of the speech segment. If the calculated average fundamental frequency value exceeds a preset fundamental frequency threshold (e.g., 180Hz), or the calculated speech rate value exceeds a preset speech rate threshold (e.g., 7.5 syllables per second), the passenger is determined to be in an excited state.

[0049] In summary, this application, through the aforementioned analysis of passengers' blink frequency, heart rate variability, skin conductance, and voice characteristics, can effectively distinguish between passengers' fatigue, tension, and excitement states. This recognition process utilizes the specific change patterns of different physiological and behavioral signals under different states. By setting judgment conditions and thresholds, continuous biometric data is transformed into discrete state classifications, providing state input for subsequent adjustments to the content and display attributes of the passenger's screen area. This improves passenger comfort while also optimizing the overall cabin's interactive experience.

[0050] In some instances, adjustments are made to the target screen area based on the current state, including: Based on the target object, determine the target screen area to be adjusted; Based on the current state and the target screen area, determine the corresponding display adjustment strategy; Based on the display adjustment strategy, the display attributes of the target screen area are adjusted.

[0051] For example, the process of adjusting a target screen area based on the current state first involves determining the target screen area to be adjusted based on the identity of the target object. In the preset split-screen layout of the in-vehicle integrated screen, the screen is divided into multiple logical areas, such as the driver's main display area corresponding to the driver's primary interaction, the passenger entertainment area corresponding to the front passenger's entertainment needs, and the public information area displaying basic vehicle information or public content. Based on the preset mapping relationship between the target object and the screen area, for example, the driver's state is mainly associated with the adjustment of the driver's main display area and the public information area, and the passenger's state is mainly associated with the adjustment of the front passenger entertainment area, the specific screen area affected by the current state, i.e., the target screen area, is determined.

[0052] After identifying the target screen area, a pre-defined adjustment rule base is queried based on the identified current state and the target screen area to determine the corresponding display adjustment strategy. This rule base defines the adjustment measures to be taken under different combinations of states and screen areas. For example, when the target is the driver and their current state is identified as distracted, the display adjustment strategy for the passenger entertainment area includes reducing its display area ratio and lowering the display brightness; the strategy for the driver's main display area includes increasing its display area ratio and enhancing the display of navigation information and safety warnings. The display adjustment strategy specifically defines which display attributes of the screen area need to be adjusted and the target parameters for adjustment.

[0053] Based on the determined display adjustment strategy, specific adjustments are made to the display attributes of the target screen area. This operation is driven by the adaptive rendering engine and is achieved by calling the underlying graphics interface to modify the layout and rendering parameters of the target screen area. For example, reducing the display area of ​​the passenger-side entertainment zone is accomplished by calculating the new area coordinates and dimensions and calling the view manager for layout rearrangement; reducing the brightness of this area is achieved by adjusting the color matrix of the corresponding display buffer or directly sending a dimming command to the backlight control module; enhancing the display of safety information in the driver's main display area involves increasing the rendering size of specific UI elements (such as navigation arrows), applying high-contrast color configurations (such as rendering warning information in red), or adding dynamic visual effects (such as making specific warning icons flash). All these adjustments change the visual output of the screen according to the specific parameters and action commands defined in the display adjustment strategy.

[0054] In summary, this application achieves intelligent adaptation between screen display and user state through the aforementioned method of adjusting the screen area based on the target object's state. It optimizes the allocation of screen resources and the presentation of information according to the needs of different users in specific states. For example, it automatically weakens the visual interference of entertainment content and strengthens safety information guidance when the driver is distracted, thereby effectively improving driving safety; simultaneously, it provides soothing content to improve the passenger experience when passengers are tense or fatigued. This adaptive screen control mechanism enhances the accuracy and intelligence of human-computer interaction, enabling the screen system to proactively perceive the user's state based on multi-dimensional physiological data and automatically execute corresponding display optimization operations based on preset strategies, ultimately achieving a synergistic improvement in safety and comfort in the complex cabin environment.

[0055] In some instances, it also includes: When there are multiple target objects, the adjustment priority of each target object is determined based on its type; Based on the current state, adjust the target screen area, including: Based on the current state and adjustment priority of each target object, the target screen area is adjusted.

[0056] For example, when multiple target objects exist in the vehicle environment, their respective adjustment priorities are determined based on the type of each target object. This priority determination process follows a preset priority rule that clearly defines the relative priority order between different types of target objects. For instance, in an in-vehicle application scenario, the preset rule sets the driver's adjustment priority higher than that of passengers. By identifying the seat position information corresponding to each target object and combining the mapping relationship between object type and seat position, its type is determined, and a corresponding adjustment priority value is assigned according to the preset rule.

[0057] After determining the adjustment priority of each target object, adjustments are made to the target screen area based on the current state and adjustment priority of each target object. Specifically, the current state of all identified target objects is obtained. Then, based on a preset mapping between screen areas and target object types, one or more screen areas to be adjusted associated with the state of each target object are determined. Next, based on the current state and adjustment priority of all relevant target objects, a preset conflict resolution strategy library is consulted to decide the final adjustment operation to be performed on each screen area. This strategy library defines how to merge requests based on priority when the states of multiple target objects simultaneously trigger adjustment requests for different or the same screen area, and these requests have potential conflicts. For example, the strategy stipulates that when the state of a high-priority target object triggers an adjustment request for a certain screen area, this request will override the potentially conflicting requests of lower-priority target objects for the same area; or, for non-conflicting adjustment requests for different screen areas, they can be executed in parallel. Based on this decision, specific screen adjustment instructions are generated and executed.

[0058] In summary, the embodiments of this application ensure that the allocation of screen resources and the adjustment of display strategies follow preset principles in complex in-vehicle environments with multiple users. By determining priorities, decisions can be made when multiple user status signals are input simultaneously, avoiding screen display chaos or frequent switching caused by command conflicts. For example, this mechanism ensures that, under any circumstances, the driver's status, which is closely related to driving safety, always prioritizes the critical information areas of the screen, thereby improving passenger comfort while always putting driving safety first, enhancing practicality and reliability in real-world complex scenarios.

[0059] Please see Figure 2 The diagram below illustrates the structure of a screen control device according to an embodiment of this application, including: Data acquisition unit 21 is used to acquire biometric data of the target object; The state recognition unit 22 is used to determine the current state of the target object based on biometric data and a preset state recognition model. The screen control unit 23 is used to perform adjustment operations on the target screen area based on the current state.

[0060] Please see Figure 3 This application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements the steps of a screen control method.

[0061] Since the electronic device described in this embodiment is a device used to implement a screen control device in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this application falls within the scope of protection of this application.

[0062] In practice, when the computer program 311 is executed by the processor, it can implement any of the embodiments corresponding to the first aspect.

[0063] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0064] Those skilled in the art will understand that embodiments of this application can provide methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media containing computer-readable program code.

[0065] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0066] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0067] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0068] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to perform... Figure 1 The flowchart of a screen control method in the corresponding embodiment.

[0069] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, computer instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any usable medium that a computer can store or a data storage device such as a server or data center that integrates one or more usable media. The usable medium may be a magnetic medium, an optical medium, or a semiconductor medium, etc.

[0070] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Furthermore, the mutual couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in the form of hardware and / or software functional units.

[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, magnetic disks, or optical disks.

[0075] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0076] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications that fall outside the scope of this specification.

[0077] Obviously, those skilled in the art can make various modifications to this specification without departing from its spirit and scope. Therefore, this specification also intends to include any modifications that fall within the scope of the claims and their equivalents.

Claims

1. A screen control method, characterized in that, include: Obtain biometric data of the target object; Based on the biometric data, the current state of the target object is determined; Based on the current state, the target screen area is adjusted.

2. The method according to claim 1, characterized in that, The biometric data includes eye-tracking data, heart rate monitoring data, skin conductance data, and voice sampling data.

3. The method according to claim 2, characterized in that, Determining the current state of the target object based on the biometric data includes: Based on the eye-tracking data, the duration of gaze deviation and blink frequency of the target object are determined; Based on the heart rate monitoring data, the heart rate value and heart rate variability data of the target object are determined; Based on the skin conductance data, the skin conductance waveform characteristics of the target object are determined; Based on the speech sampling data, the fundamental frequency value and speech rate value of the target object are determined; The current state of the target object is determined by analyzing the duration of gaze deviation, blink frequency, heart rate value, heart rate variability data, skin conductance waveform characteristics, speech fundamental frequency value, and speech rate value.

4. The method according to claim 3, characterized in that, The target object is the driver, and the current state includes a distracted state, a fatigued state, a tense state, or an excited state. The process of analyzing the duration of gaze deviation, blink frequency, heart rate value, heart rate variability data, skin conductance waveform characteristics, voice fundamental frequency value, and speech rate value to determine the current state of the target object includes: When the duration of the gaze deviation exceeds a first preset duration, the driver is determined to be in a distracted state. When the blinking frequency is less than a preset frequency and the heart rate is less than a preset heart rate for a second preset duration, the driver is determined to be in a state of fatigue. When the heart rate variability data is less than a preset variability threshold, or the peak amplitude of the skin conductance waveform is greater than a preset amplitude threshold, the driver is determined to be in a state of tension. When the fundamental frequency value of the speech is greater than a preset fundamental frequency threshold, or the speech rate value is greater than a preset speech rate threshold, the driver is determined to be in an excited state.

5. The method according to claim 4, characterized in that, The target object is a passenger, and the current state includes a fatigued state, a tense state, or an excited state. The current state of the target object is determined by analyzing the blink frequency, heart rate variability data, skin conductance waveform characteristics, voice fundamental frequency value, and speech rate value, including: When the blinking frequency is less than the preset frequency, and the frequency domain energy of the skin conductance waveform characteristics is in the preset fatigue frequency band, the passenger is determined to be in a fatigued state. When the heart rate variability data is less than the preset variability threshold, or the peak amplitude of the skin conductance waveform feature is greater than the preset amplitude threshold, the passenger is determined to be in a state of tension. When the voice base frequency value is greater than the preset base frequency threshold, or the speech rate value is greater than the preset speech rate threshold, the passenger is determined to be in an excited state.

6. The method according to claim 1, characterized in that, The adjustment operation of the target screen area based on the current state includes: Based on the target object, determine the target screen area to be adjusted; Based on the current state and the target screen area, determine the corresponding display adjustment strategy; Based on the aforementioned display adjustment strategy, the display attributes of the target screen area are adjusted.

7. The method according to claim 1, characterized in that, Also includes: When there are multiple target objects, the adjustment priority of each target object is determined based on its type; The adjustment operation of the target screen area based on the current state includes: Based on the current state of each target object and the adjustment priority, the target screen area is adjusted.

8. A screen control device, characterized in that, include: The data acquisition unit is used to acquire the biometric data of the target object; A state recognition unit is used to determine the current state of the target object based on the biometric data and a preset state recognition model. The screen control unit is used to perform adjustment operations on the target screen area based on the current state.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program stored in the memory, implements the steps of the screen control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the screen control method as described in any one of claims 1 to 7.

Images