Self-adaptive HUD adjustment method based on man-machine flight mission light environment

Abstract

The invention provides a light environment adaptive HUD adjusting method based on a man-machine flight mission, and belongs to the technical field of flight display control. Light intensity, color temperature and reflectivity data outside a cabin are collected, and eye movement signals, electroencephalogram signals and heart rate variability signals of a pilot are synchronously obtained; after registration, constructing a synchronous time sequence data set containing the light environment and the physiological state, and based on external light parameters, utilizing a gradient lifting decision tree model to identify the current light environment and obtaining a brightness reference interval; according to the invention, the HUD brightness and contrast are dynamically calculated and compensated in advance through real-time acquisition of external light parameters and a classification result of a light environment recognition model, so that the HUD can maintain clear symbols and stable contrast under various light conditions such as cloud crossing, backlight, strong reflection and night voyage; the problem of over-brightness or over-darkness caused by adjustment lag of a traditional fixed brightness curve is avoided, and information readability in a flight task is improved.

Description

Technical Field

[0001] This invention relates to the field of flight display control technology, and in particular to a method for adaptive HUD adjustment based on the lighting environment during human-machine flight missions. Background Technology

[0002] By overlaying flight speed, altitude, attitude, trajectory, and guidance information onto the forward field of view, HUD enables pilots to acquire key flight parameters while maintaining their line of sight. In modern mission scenarios, aircraft need to continuously experience various lighting environments, including low-angle lighting at dawn and dusk, highly reflective sea surfaces, cloud crossings, high-contrast illumination at high altitudes, and complex conditions such as night approach. The readability and stability of the HUD display are directly related to flight safety. Traditional HUD adjustment methods mainly employ fixed brightness curves, manual adjustment by the pilot, or passive compensation modes relying on a single ambient light sensor. However, under conditions such as cloud cover boundaries, reflections from city lights, strong reflections from the sea surface, and sudden changes in illumination angle caused by changes in heading, HUD brightness and contrast often exhibit response lag, insufficient compensation, or over-adjustment, resulting in symbols that are too bright, too dark, or have reduced contrast. In some cases, symbols may even be submerged in the background, affecting the pilot's speed of acquiring critical information.

[0003] Meanwhile, the visual load and physiological stress of pilots vary significantly in different flight phases. For example, during landing, low-visibility approach, and backlight cruise, the pilot's concentration, visual scanning frequency, and physiological stress will change. However, the traditional HUD display adjustment mechanism does not take the pilot's own state into account in the adjustment logic, making it impossible for the HUD display content to dynamically adapt to the pilot's actual cognitive load, thus increasing the operational burden. To address this, an adaptive HUD adjustment method based on the lighting environment during human-machine flight missions is proposed. Summary of the Invention

[0004] In view of this, the present invention provides an adaptive HUD adjustment method based on the light environment of human-machine flight missions to solve or alleviate the technical problems existing in the prior art, and at least provides a beneficial option.

[0005] The technical solution of this invention is implemented as follows: A method for adaptive HUD adjustment based on the light environment of a human-machine flight mission, comprising the following steps: S1. During the flight mission, external light sensors in the cockpit are used to collect data on light intensity, color temperature and reflectivity to form external light parameters; at the same time, physiological monitoring equipment is used to collect pilot eye movement signals, electroencephalogram signals and heart rate variability signals to form physiological signals; and flight parameters are collected simultaneously, including heading deviation and altitude deviation. External light parameters, physiological signals and flight parameters are registered and interpolated according to a unified timestamp to form a synchronous time series dataset. S2. Input the external light parameters into the gradient boosting decision tree model for light environment recognition to obtain the light environment category and the corresponding brightness reference range. During the model training phase, a loss function with a regularization term is used, and the model parameters are determined through cross-validation. S3. Extract multi-dimensional features based on physiological signals and flight parameters, including: Extract pupil diameter change rate, saccade frequency in region of interest, peak saccade angular velocity, fixation duration, and scan rate from eye movement signals; Extract the alpha amplitude and the theta / β ratio of the prefrontal cortex from the electroencephalogram (EEG) signal; Extract the root mean square (RMSSD) of the difference between adjacent normal heartbeats and the low-frequency power to high-frequency power ratio (LF / HF) from the heart rate variability signal; Directly use the heading deviation and altitude deviation from the flight parameters; Calculate the cognitive load index using the formula:

[0006] Cognitive load index; The main feature weights correspond to the eye-tracking features. With flight parameters The weighted sum; These are the secondary feature weights, corresponding to EEG features. With heart rate variability characteristics The weighted sum, and satisfying These are the normalized means of each feature in the corresponding dimension; S4. Using the ambient light recognition result and cognitive load index as joint inputs, calculate the brightness adjustment value and contrast adjustment value of the HUD. The brightness adjustment value L is calculated using the formula...

[0007] Confirmed, among which External light intensity, and This is the proportionality coefficient; When the rate of change of external light intensity exceeds a preset threshold, brightness compensation is performed in advance. When the cognitive load index is higher than a set threshold, the brightness of non-critical information areas is automatically reduced and the displayed content is simplified.

[0008] More preferably, the sampling frequency of the external light parameters is not less than 50Hz, the sampling rate of the physiological signal is not less than 256Hz, and the data synchronization error is controlled within 10 milliseconds.

[0009] More preferably, the gradient boosting decision tree model used to perform light environment recognition in S2 employs five-fold cross-validation during training. The parameter tuning variables of the gradient boosting decision tree model include the learning rate, tree depth, and number of weak classifiers, and the optimal parameter combination is determined by maximizing the accuracy of the validation set.

[0010] More preferably, the weight coefficients of the cognitive load index C are determined by a combination of linear regression analysis and principal component analysis, wherein eye movement features account for the main proportion in the overall weight, while electroencephalogram features and heart rate variability features account for the secondary proportion.

[0011] Further preferably, when the cognitive load index exceeds a preset threshold, the brightness of non-critical areas is automatically reduced and non-critical display content is decreased to alleviate the visual interaction load on the pilot.

[0012] Further preferred, multi-scale image enhancement processing is performed on the adaptively adjusted HUD display to improve the contrast and visual clarity of key flight information in backlight, night flight, or low visibility environments.

[0013] In a further preferred embodiment, after the HUD display parameters are adjusted, the local areas with insufficient brightness are boosted based on the current brightness distribution of the screen, while the overly bright areas are suppressed to maintain the overall brightness balance of the screen.

[0014] Further preferred methods include prioritizing brightness adjustment based on external light intensity during the cruise phase and prioritizing adjustment based on cognitive load index during the landing phase.

[0015] Further preferably, the HUD display parameters are updated within a period of no more than 50 milliseconds, and a sliding average method is used to smooth the brightness and contrast adjustments of consecutive frames.

[0016] Further optimized, the brightness advance compensation is based on the rate of change of light. Predict light trends and update the function according to brightness:

[0017] Perform smooth adjustment, where Indicates the rate of change of light. Indicates the rate of change of cognitive load. and To adjust the weighting coefficients.

[0018] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions: I. This invention dynamically calculates and compensates for the brightness and contrast of the HUD by real-time acquisition of external light parameters and classification results of the light environment recognition model. This enables the HUD to maintain clear symbols and stable contrast under various lighting conditions such as cloud cover, backlight, strong reflection, and night flight, avoiding the problem of excessive brightness or darkness caused by the lag in the adjustment of the traditional fixed brightness curve, and improving the readability of information in flight missions.

[0019] Second, this invention utilizes the cognitive load index, composed of eye movement features, electroencephalogram features, and heart rate variability, to reflect the pilot's immediate stress. When the cognitive load increases, it automatically reduces the brightness of non-critical information and simplifies the interface display, allowing visual resources to be focused on key parameters such as speed, altitude, and attitude, thereby reducing visual interference and improving operational efficiency and safety during high-task phases.

[0020] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of the steps of the present invention. Detailed Implementation

[0023] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0024] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] like Figure 1 As shown, this embodiment of the invention provides a method for adaptive HUD adjustment based on the lighting environment during human-machine flight missions, comprising the following steps: During the flight mission, step S1 is executed, which involves collecting and synchronizing external light parameters and physiological signals. In this embodiment, parameter acquisition and data synchronization are performed as follows: During flight mission execution, the acquisition and time synchronization of external light parameters and physiological signals are first completed. This embodiment uses a light sensor installed at the front of the cockpit to construct an external light acquisition module. This module continuously outputs light intensity, color temperature, and the percentage of reflectance from the cockpit glass. Light intensity reflects the ambient brightness level, color temperature determines the type of light source, and reflectance identifies the glare effect caused by light inside the cockpit. These three parameters constitute the external light parameters in this invention. Meanwhile, to accurately characterize the visual load and attention distribution of pilots under different lighting conditions, this embodiment uses an EEG acquisition device to obtain EEG signals from the pilot's forehead area, and obtains heart rate variability (HRV) parameters through a heart rate monitoring belt. A head-mounted eye tracker captures real-time changes in fixation point, fixation duration, and saccade counts. Eye movement, EEG, and HRV signals together constitute the physiological signals in this invention. In addition, flight parameters, including heading deviation and altitude deviation, are collected simultaneously. The aforementioned external light parameters, physiological signals, and flight parameters are all sampled at a fixed frequency and registered using a unified GPS timestamp. To address the differences in sampling frequencies between different signals, a linear interpolation method is used to align the sampling points, ultimately generating a synchronized time-series dataset containing light, physiological characteristics, and flight parameters.

[0026] After completing data acquisition and registration, step S2 is executed to identify the lighting environment using a gradient boosting decision tree model. This embodiment establishes a lighting environment recognition model, using the collected light intensity, color temperature, and reflectivity as feature inputs. The lighting environment recognition model is constructed using a gradient boosting framework, generating weak decision trees through multiple iterations, ultimately forming a classification model with strong fitting ability. During the model training phase, this embodiment uses collected data on different lighting scenes (including midday, bright backlight, sunrise, sunset, night, foggy days, etc.) as training set samples. This invention introduces a loss function with a regularization term into the objective function, limiting model complexity by penalizing the number of leaf nodes and node weights in a single tree. During model training, a five-fold cross-validation method is used to perform grid search on the parameters based on tree depth, number of weak classifiers, and learning rate, so that the recognition accuracy on the validation set reaches the optimal level and the model's generalization ability is ensured. The trained model takes real-time light parameters as input and outputs the light environment category, such as "strong light and backlight", "low visibility" and "low brightness at night", while also outputting the corresponding brightness reference range.

[0027] Step S3 is used to calculate the pilot's cognitive load index, which is the psychological and visual load of the pilot under different lighting and mission conditions. Multi-dimensional features are extracted based on physiological signals and flight parameters, including: Extract pupil diameter change rate, saccade frequency in region of interest, peak saccade angular velocity, fixation duration, and scan rate from eye movement signals; Extract the alpha amplitude and the theta / β ratio of the prefrontal cortex from the electroencephalogram (EEG) signal; Extract the root mean square (RMSSD) of the difference between adjacent normal heartbeats and the low-frequency power to high-frequency power ratio (LF / HF) from the heart rate variability signal; Directly use the heading deviation and altitude deviation from the flight parameters; Calculate the cognitive load index using the formula:

[0028] In this embodiment, a targeted processing method is used for physiological signals during the feature extraction stage: Frequency domain analysis was used to extract the energy ratio of theta waves to beta waves (i.e., the theta / beta ratio) from the electroencephalogram (EEG) signals. This ratio can intuitively reflect the pilot's level of attention concentration. The root mean square difference (RMSSD) between adjacent heartbeats was extracted from the heart rate variability signal to accurately measure the pilot's level of tension. The fixation duration, fixation frequency, and saccade frequency of the area of ​​interest (AOI) were statistically analyzed from the eye movement signals. These features can effectively reflect the level of visual load. Each weight coefficient was determined using linear regression analysis and principal component analysis, allowing EEG characteristics, heart rate characteristics, and eye movement characteristics to participate in the calculation according to their contribution to cognitive load. Among them, eye movement characteristics account for the main proportion in the overall weight (value range 0.4-0.6), while EEG characteristics and heart rate variability characteristics are accounted for the secondary proportion (value range 0.1-0.2 respectively). The weights of flight parameters are adapted according to the actual flight scenario requirements (value range 0.1-0.2). B, H, E, and F are the mean values ​​of each feature under the corresponding dimension after normalization. The cognitive load index obtained in this way can truly reflect the overall cognitive state of the pilot under the current flight mission and lighting conditions. When the pilot stares at the speed or altitude area for a long time, or when the pilot shows characteristics such as pupil dilation and reduced saccades under backlight conditions, the cognitive load index will increase, providing a reliable physiological basis for HUD adjustment. Once the ambient light recognition result and cognitive load index are calculated, step S4 is executed to adaptively adjust the HUD brightness and contrast parameters. The brightness adjustment value in this embodiment uses the following formula:

[0029] External light intensity This directly affects the visibility of the HUD in bright light environments; Cognitive Load Index This reflects the pilot's current visual and psychological stress, which can be addressed by adjusting parameters. and To achieve a suitable brightness balance between lighting conditions and the pilot's physiological state, this invention also employs pre-compensation for brightness, i.e., when the rate of change of external light intensity... If the threshold is exceeded (such as when the aircraft makes a rapid turn or passes through clouds), a compensation value is generated in advance based on the changing trend, so that the HUD brightness enters a transition state before the light changes abruptly, avoiding instantaneous overexposure or underexposure of the HUD image due to sudden changes in external light. When the cognitive load index is higher than the preset threshold, this embodiment automatically reduces the brightness of non-critical information areas and temporarily reduces non-critical display content, so that core parameters such as speed, altitude, and attitude are kept in optimal visibility, thereby reducing the visual processing pressure on the pilot and improving the stability of mission execution. After the HUD display parameters are adjusted, further image enhancement processing is performed on the HUD image to maintain the clarity of key flight information under different lighting conditions. This embodiment uses two types of image enhancement methods: One type of algorithm is multi-scale Retinex and histogram equalization, which is mainly used to improve local contrast in backlight or low brightness environments, so that key text and symbols remain stable in bright and dark backgrounds. Another type is the brightness distribution analysis method, which boosts the brightness of local areas with insufficient brightness in the image and suppresses the brightness of areas with excessive brightness, so as to keep the overall visual comfort of the image stable. These two processing methods can be used in combination. The image enhancement process is performed after the HUD brightness and contrast are adjusted. It is not part of the main adjustment algorithm, but is to improve the actual display effect. Furthermore, during the cruise phase, the stability of external light is high and the cognitive load of the pilot is relatively low. Therefore, this embodiment prioritizes using the intensity of external light as the basis for brightness adjustment. During the landing phase, since the pilot needs to process multiple pieces of information simultaneously, the weight of the cognitive load index increases. Therefore, during this phase, the brightness and display content are adjusted based on the cognitive load index to make the information presentation more in line with the pilot's operational needs. To ensure the continuity and stability of the HUD display, this embodiment sets the brightness and contrast update cycle to no more than 50 milliseconds, and uses a moving average method to smooth the brightness adjustment of consecutive frames to avoid flickering due to rapid brightness changes. During the brightness pre-compensation process, this embodiment adjusts the brightness based on the rate of light change. Predict light trends and update the brightness weights using the following formula:

[0030] in and The weighting coefficients enable this function to respond to changes in brightness in response to sudden changes in external light, while avoiding visual instability caused by overcompensation.

[0031] The embodiments of the present invention constitute a complete method through steps such as external light parameter acquisition, pilot physiological signal analysis, light environment recognition, cognitive load calculation, and adaptive adjustment of HUD display parameters. This method enables real-time optimized display of the HUD under different light environments and flight mission conditions, ensuring that critical flight information always has good clarity and readability.

[0032] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for adaptive HUD adjustment based on lighting environment during human-machine flight missions, characterized in that, The method comprises the following steps: S1, during the flight task execution, collecting light intensity, color temperature and reflectivity data by using the outside-cabin light sensor to form the outside light parameter; simultaneously collecting the pilot's eye movement signal, electroencephalogram signal and heart rate variability signal by using the physiological monitoring device to form the physiological signal; and synchronously collecting the flight parameters, the flight parameters including the heading deviation and the height deviation; The outside light parameter, the physiological signal and the flight parameter are registered and interpolated according to a unified time stamp to form a synchronous time series data set; S2, inputting the outside light parameter into a gradient boosting decision tree model for light environment recognition to obtain the light environment category and the corresponding brightness reference interval, and using a loss function with a regularization term in the model training stage and determining the model parameters through cross-validation; S3, extracting multi-dimensional features from the physiological signal and the flight parameter, including: extracting the pupil diameter change rate, the interest area saccade frequency, the saccade angular velocity peak value, the fixation duration and the scan video rate from the eye movement signal; extracting the frontal alpha wave amplitude and the theta / beta ratio from the electroencephalogram signal; extracting the root mean square of the adjacent normal heartbeat interval difference RMSSD, the low frequency power and the high frequency power ratio LF / HF from the heart rate variability signal; directly taking the heading deviation and the height deviation in the flight parameter; calculating the cognitive load index according to the formula: is a cognitive load index; is a primary feature weight corresponding to an eye movement feature is a weighted sum of flight parameters; is a weighted sum of flight parameters; is a secondary feature weight corresponding to an electroencephalogram feature is a weighted sum of heart rate variability features; is a weighted sum of heart rate variability features; are normalized mean values of the respective features under the corresponding dimension, respectively; S4, taking the light environment recognition result and the cognitive load index as joint inputs to calculate the brightness adjustment value and the contrast adjustment value of the HUD, the brightness adjustment value L being calculated according to the formula determined, wherein is the intensity of the external light, and is a proportionality factor; When the outside light intensity change rate exceeds the preset threshold, the brightness is compensated in advance, and when the cognitive load index is higher than the set threshold, the brightness of the non-key information area is automatically reduced and the display content is simplified.

2. The method of claim 1, wherein, The sampling frequency of the outside light parameter is not less than 50Hz, the sampling rate of the physiological signal is not less than 256Hz, and the data synchronization error is controlled within 10 milliseconds.

3. The method of claim 1, wherein the method further comprises: The gradient boosting decision tree model for performing light environment recognition in S2 adopts five-fold cross-validation during training, the gradient boosting decision tree model parameter adjustment variables include the learning rate, the tree depth and the number of weak classifiers, and the optimal parameter combination is determined by maximizing the validation set accuracy.

4. The method of claim 1, wherein, The weight coefficient of the cognitive load index C is determined by the method of linear regression analysis combined with principal component analysis, wherein the eye movement features account for a major proportion in the comprehensive weight, and the electroencephalogram features and the heart rate variability features account for a minor proportion.

5. The method of claim 1, wherein, When the cognitive load index exceeds the preset threshold, the brightness of the non-key area is automatically reduced and the non-key display content is reduced to reduce the pilot's visual interaction load.

6. The method of claim 1, wherein, The HUD display picture after adaptive adjustment is subjected to multi-scale image enhancement processing to improve the contrast and visual clarity of the key flight information in the backlight, night flight or low-visibility environment.

7. The method of claim 1, wherein the method further comprises: After the HUD display parameter adjustment is completed, the local area with insufficient brightness is subjected to lifting processing and the over-bright area is subjected to suppression processing based on the current picture brightness distribution to maintain the overall brightness balance of the picture.

8. The method of claim 1, wherein, During the cruising stage, the brightness adjustment is preferentially performed according to the outside light intensity, and during the landing stage, the adjustment is preferentially performed according to the cognitive load index.

9. The method of claim 1, wherein, The HUD display parameter update period is no more than 50 milliseconds, and the brightness and contrast adjustment amount of consecutive frames is smoothed in a sliding average manner.

10. The method of claim 1, wherein, In brightness advance compensation according to light rate of change Predict light trend and update brightness according to brightness update function: performing a smoothing adjustment, wherein represents a rate of change of light, represents a rate of change of cognitive load, with is an adjustment weight coefficient.

Images