Visual angle display compensation method based on dynamic adjustment

Abstract

The invention relates to the technical field of display, and discloses a visual angle display compensation method and system based on dynamic adjustment, and the method comprises the steps: obtaining and preprocessing a head posture, an aircraft state and external environment data; a pilot viewpoint is determined, and original display content is generated; and based on the viewpoint, the original content and the display model, real-time calculation is carried out, a geometric transformation matrix is applied to carry out perspective projection and distortion correction on the content, and compensated content is generated and presented. The system comprises a head attitude tracking module, an aircraft state acquisition module, an external environment perception module, a data preprocessing module, a viewpoint calculation module, a content generation module, a compensation processing module and a large-view-field display module. By the adoption of the technical scheme, accurate perspective consistency of the display content and the external environment can be achieved when the head dynamically changes, parallax is overcome, visual fatigue and cognitive loads are reduced, and information accuracy, immersion and usability are improved.

Description

Technical Field

[0001] This invention belongs to the field of display technology, specifically relating to a viewing angle compensation method based on dynamic adjustment. Background Technology

[0002] With the rapid development of aviation technology and the continuous improvement of aircraft performance, modern aircraft are equipped with an increasing number of sophisticated instruments and advanced weapon systems. Pilots face the daunting challenge of processing massive amounts of data and operating various devices in real time during flight and combat missions. To address this complex situation, airborne head-up displays (HUDs) have emerged as an important human-machine interface system.

[0003] The airborne head-up display projects crucial flight and combat data onto a display medium in front of the pilot via an optical system, placing the image at infinity relative to the human eye's focal length. All critical information is displayed within the pilot's forward field of view, seamlessly integrating the displayed information with the external environment and greatly helping the pilot concentrate on flight and combat operations. This design avoids the pilot frequently switching between various instruments below and the forward field of view, effectively reducing pilot fatigue and shortening reaction time.

[0004] However, existing airborne head-up displays (HUDs) still have many limitations that urgently need to be addressed. First, the pilot's field of vision is limited to a specific, narrow area, resulting in insufficient situational awareness of targets in a wide battlefield environment. Second, to achieve target tracking and aiming, the aircraft must frequently adjust its attitude to move the target into the limited field of view, which greatly restricts operational flexibility and efficiency. Third, existing HUDs typically display images in monochrome, failing to provide rich and realistic color information, thus affecting the pilot's accurate judgment and situational awareness of complex external environments. These shortcomings severely restrict the modern air combat's requirements for wide field of view, full-color display, and flexible control, necessitating a technological solution that can effectively address these problems. Summary of the Invention

[0005] This invention aims to address the problem that existing large field-of-view display systems cannot accurately compensate for the viewing angle of the displayed content when the pilot's head posture changes dynamically. This results in parallax and inaccurate fusion between the displayed information and the external environment, thus affecting the pilot's situational awareness and operational precision. Existing airborne head-up displays have a limited field of view and only display in monochrome, forcing pilots to adjust the aircraft's attitude to track targets. Although new large field-of-view display technologies achieve full-color display and 360-degree image fusion, expanding the pilot's field of view and enhancing situational awareness, their core problem lies in the fact that when the pilot's head changes position or posture relative to the cockpit display area, the perspective relationship between the synthetic information presented on the display (such as flight data, target indication, etc.) and the actual external scene will no longer match the pilot's actual viewpoint, resulting in visual bias and discomfort, reducing the accuracy of information interpretation and operational efficiency.

[0006] To address the aforementioned technical problems, this invention provides a dynamic adjustment-based viewpoint display compensation method and system. This invention dynamically calculates the pilot's current viewpoint parameters by accurately sensing the pilot's head position and attitude in real time, combined with the aircraft's own state and external environmental perception data. Based on these viewpoint parameters, the original display content, and the physical geometric model of the large field-of-view display, a geometric transformation matrix is ​​generated and applied in real time to perform perspective projection and distortion correction on the displayed content. This ensures that regardless of how the pilot's head moves, the information presented on the display always maintains precise perspective consistency with the real external world under their current viewpoint, achieving seamless and accurate integration of the displayed content with the external environment.

[0007] This invention provides a dynamic adjustment-based viewpoint display compensation method, which includes:

[0008] Acquire real-time head posture data of the pilot, which includes the three-dimensional position and three-dimensional posture of the pilot's head in the aircraft coordinate system;

[0009] Acquire real-time status data of the aircraft, including the aircraft's position, attitude, velocity, and acceleration;

[0010] Acquire external environment perception data, which includes panoramic images of the aircraft's external environment and the target's three-dimensional position and velocity;

[0011] Data preprocessing includes time synchronization, data alignment, and noise filtering of the pilot's real-time head posture data, the aircraft's real-time status data, and the external environment perception data.

[0012] Based on the preprocessed real-time head posture data of the pilot, the real-time status data of the aircraft, and the geometric parameters of the large field of view display, the three-dimensional position and orientation of the pilot's current viewpoint in the aircraft coordinate system are determined.

[0013] The original display content is generated based on the preprocessed real-time status data of the aircraft, the preprocessed external environment perception data, and the flight mission planning information.

[0014] Based on the pilot's current viewpoint, the original display content, and the physical geometric model of the large field of view display, a geometric transformation matrix is ​​calculated and applied in real time to perform perspective projection and distortion correction on the original display content, generating compensated display content.

[0015] The compensated display content is then presented on a large field-of-view display.

[0016] In a preferred embodiment of the present invention, the acquisition of the pilot's real-time head posture data specifically includes:

[0017] The system uses an array of at least three infrared cameras located inside the cockpit to capture infrared images of markers on the pilot's helmet in real time.

[0018] The pixel coordinates of the marker points are extracted from the infrared image using an image processing algorithm;

[0019] Using multi-point positioning and triangulation algorithms, based on the pixel coordinates of the marked points and the known spatial position and calibration parameters of the infrared camera array, the six-degree-of-freedom position and attitude of the pilot's head in the aircraft coordinate system are calculated.

[0020] Furthermore, the sampling frequency of the infrared camera array is higher than 150Hz to ensure the real-time and continuous nature of head posture data.

[0021] In a preferred embodiment of the present invention, the acquisition of real-time aircraft status data specifically includes:

[0022] The inertial navigation system acquires the aircraft's real-time attitude angles, angular velocity, acceleration, and angular acceleration data.

[0023] The real-time three-dimensional position coordinates and velocity data of the aircraft are obtained through a GPS receiver.

[0024] The air pressure, temperature, humidity, wind speed, and wind direction data of the airspace where the aircraft is located are obtained through atmospheric data sensors.

[0025] By using a data fusion algorithm, the aforementioned sensor data is fused to obtain high-precision real-time status data of the aircraft.

[0026] In a preferred embodiment of the present invention, the acquisition of external environment perception data specifically includes:

[0027] Multiple high-resolution full-color cameras mounted on the exterior of the aircraft capture real-time images of the environment surrounding the aircraft.

[0028] The environmental images are stitched together in real time using an image stitching algorithm to generate a 360-degree panoramic environmental image.

[0029] The space distribution, relative position, and relative velocity data of external targets are acquired in real time by radar and lidar sensors installed on the outside of the aircraft.

[0030] External targets are identified, located, and tracked from radar and lidar data using target detection and tracking algorithms.

[0031] In a preferred embodiment of the present invention, the data preprocessing specifically includes:

[0032] All collected data are timestamped and synchronized, and data from different sensor sampling frequencies are unified to a preset sampling frequency, such as 120Hz.

[0033] Data alignment is performed on the time-synchronized data to ensure accurate time relationships between different data streams;

[0034] The aligned data is then subjected to noise filtering to remove random errors and interference from the sensor acquisition process.

[0035] In a preferred embodiment of the present invention, determining the three-dimensional position and orientation of the pilot's current viewpoint in the aircraft coordinate system specifically includes:

[0036] The real-time head posture data of the pilot is converted into the position vector and line-of-sight direction vector of the pilot's viewpoint in the aircraft coordinate system;

[0037] Based on the installation position and geometric parameters of the large field-of-view display within the aircraft cockpit, the distance and relative angle from the pilot's viewpoint to the display surface are calculated.

[0038] In a preferred embodiment of the present invention, generating the original display content specifically includes:

[0039] Based on the external environment perception data, a background environment image and external target indication information are generated;

[0040] Based on the real-time status data of the aircraft, flight instrument data, navigation path information, and flight warning information are generated;

[0041] Based on flight mission planning information, generate mission objectives, route planning, and threat area indication information;

[0042] The above information is superimposed and synthesized according to preset priorities and layout rules to form a two-dimensional or three-dimensional original display content data stream.

[0043] In a preferred embodiment of the present invention, the real-time calculation and application of the geometric transformation matrix to perform perspective projection and distortion correction on the original display content to generate compensated display content specifically includes:

[0044] Based on the pilot's current viewpoint, the three-dimensional spatial position of the original displayed content, and the physical geometric model of the large field-of-view display, calculate the perspective projection matrix corresponding to the current viewpoint;

[0045] Calculate the distortion correction matrix based on the optical distortion characteristics of the large field-of-view display;

[0046] The original display content is geometrically transformed in real time using the perspective projection matrix and the distortion correction matrix, and projected from the original space onto the surface of the large field of view display to eliminate parallax and geometric distortion.

[0047] Furthermore, the frequency of real-time calculation and application of the geometric transformation matrix is ​​consistent with the frequency of head posture data acquisition to ensure the real-time nature of the compensation.

[0048] According to another aspect of the present invention, a viewing angle compensation system based on dynamic adjustment is provided, comprising:

[0049] The pilot head attitude tracking module is used to acquire real-time head attitude data of the pilot, which includes the three-dimensional position and three-dimensional attitude of the pilot's head in the aircraft coordinate system.

[0050] The aircraft status data acquisition module is used to acquire real-time status data of the aircraft, including the aircraft's position, attitude, velocity, and acceleration.

[0051] An external environment perception module is used to acquire external environment perception data, which includes panoramic images of the aircraft's external environment and the three-dimensional position and velocity of the target.

[0052] The data preprocessing module is used to perform time synchronization, data alignment, and noise filtering on the pilot's real-time head posture data, the aircraft's real-time status data, and the external environment perception data;

[0053] The viewpoint calculation module is used to determine the three-dimensional position and orientation of the pilot's current viewpoint in the aircraft coordinate system based on the preprocessed real-time head posture data of the pilot, the real-time status data of the aircraft, and the geometric parameters of the large field of view display.

[0054] The display content generation module is used to generate original display content based on the preprocessed real-time status data of the aircraft, the preprocessed external environment perception data, and flight mission planning information.

[0055] The compensation processing module is used to calculate and apply a geometric transformation matrix in real time based on the pilot's current viewpoint, the original display content, and the physical geometric model of the large field of view display, to perform perspective projection and distortion correction on the original display content and generate compensated display content.

[0056] A large field-of-view display module is used to present the compensated display content on a large field-of-view display.

[0057] In a preferred embodiment of the present invention, the pilot head attitude tracking module includes:

[0058] At least three infrared camera arrays are installed inside the cockpit to capture infrared images of markers on the pilot's helmet in real time;

[0059] An image processing unit is used to extract the pixel coordinates of the marker points from the infrared image;

[0060] The attitude calculation unit is used to calculate the six-degree-of-freedom position and attitude of the pilot's head in the aircraft coordinate system based on the pixel coordinates of the marked points and the known spatial position and calibration parameters of the infrared camera array, using multi-point positioning and triangulation algorithms.

[0061] Furthermore, the sampling frequency of the infrared camera array is higher than 150Hz.

[0062] In a preferred embodiment of the present invention, the aircraft status data acquisition module includes:

[0063] Inertial navigation systems are used to acquire real-time attitude angles, angular velocities, accelerations, and angular acceleration data of aircraft.

[0064] A global positioning system receiver is used to acquire the real-time three-dimensional position coordinates and velocity data of the aircraft;

[0065] Atmospheric data sensors are used to acquire data on air pressure, temperature, humidity, wind speed, and wind direction in the airspace where the aircraft is located.

[0066] The data fusion unit is used to fuse the aforementioned sensor data to obtain high-precision real-time status data of the aircraft.

[0067] In a preferred embodiment of the present invention, the external environment sensing module includes:

[0068] Multiple high-resolution full-color cameras are mounted on the exterior of the aircraft to capture real-time images of the environment surrounding the aircraft.

[0069] An image stitching unit is used to stitch the environmental images in real time to generate a 360-degree panoramic environmental image;

[0070] Radar sensors and lidar sensors are installed on the outside of the aircraft to acquire real-time data on the spatial distribution, relative position, and relative velocity of external targets.

[0071] The target detection and tracking unit is used to identify, locate and track external targets from the radar and lidar data.

[0072] In a preferred embodiment of the present invention, the data preprocessing module includes:

[0073] The time synchronization unit is used to timestamp and synchronize all collected data, unifying the data from different sensor sampling frequencies to a preset sampling frequency.

[0074] A data alignment unit is used to align the time-synchronized data.

[0075] A noise filtering unit is used to perform noise filtering on the aligned data.

[0076] In a preferred embodiment of the present invention, the viewpoint calculation module specifically includes:

[0077] The attitude conversion unit is used to convert the pilot's real-time head attitude data into the pilot's viewpoint position vector and line-of-sight direction vector in the aircraft coordinate system.

[0078] The distance and angle calculation unit is used to calculate the distance and relative angle from the pilot's viewpoint to the display surface by combining the installation position and geometric parameters of the large field-of-view display in the aircraft cockpit.

[0079] In a preferred embodiment of the present invention, the display content generation module specifically includes:

[0080] The background environment generation unit is used to generate a background environment image and external target indication information based on the external environment perception data.

[0081] The flight information generation unit is used to generate flight instrument data, navigation path information, and flight warning information based on the real-time status data of the aircraft.

[0082] The mission information generation unit is used to generate mission objectives, route planning, and threat area indication information based on flight mission planning information.

[0083] The information synthesis unit is used to superimpose and synthesize the above information according to preset priority and layout rules to form a two-dimensional or three-dimensional original display content data stream.

[0084] In a preferred embodiment of the present invention, the compensation processing module specifically includes:

[0085] The perspective projection matrix calculation unit is used to calculate the perspective projection matrix corresponding to the current viewpoint based on the pilot's current viewpoint, the three-dimensional spatial position of the original display content, and the physical geometric model of the large field of view display.

[0086] The distortion correction matrix calculation unit is used to calculate the distortion correction matrix based on the optical distortion characteristics of the large field-of-view display.

[0087] The geometric transformation unit is used to perform real-time geometric transformation on the original display content through the perspective projection matrix and the distortion correction matrix, projecting it from the original space onto the surface of the large field of view display to eliminate parallax and geometric distortion.

[0088] Furthermore, the processing frequency of the compensation processing module is consistent with the acquisition frequency of the pilot head attitude tracking module.

[0089] As a preferred embodiment of the present invention, the large field-of-view display module adopts a technology that combines organic light-emitting diode array or laser projection with a curved screen, which has high resolution, high brightness, high refresh rate and wide field of view.

[0090] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0091] 1. This invention provides a dynamic adjustment-based viewpoint display compensation method and system, which ensures that the content displayed on the large field-of-view display maintains precise perspective consistency with the external environment even when the pilot's head posture changes dynamically within the cockpit. By introducing real-time fusion of high-precision head posture tracking, aircraft status perception, and external environment perception data, this invention can accurately acquire the pilot's instantaneous viewpoint parameters and accordingly perform real-time geometric transformation and perspective projection correction on the displayed content.

[0092] 2. This invention solves the problems of limited field of view and monochrome display in traditional head-up displays. More importantly, it overcomes the core technical bottleneck of existing large-field-of-view full-color display technology, which suffers from inaccurate integration of displayed information with the external environment and parallax when the pilot's head moves. This eliminates the need for pilots to adjust their focus or viewing angle between the display and the external environment when observing the screen and the scenery, reducing visual fatigue and cognitive load.

[0093] 3. This invention improves the accuracy, immersion, and usability of displayed information, especially in highly dynamic flight environments, ensuring pilots' precise perception of the battlefield situation and accurate target aiming. Through dynamic compensation, virtual display information (such as aiming crosshairs, target indications, and flight path predictions) can be positioned correctly in three-dimensional space from any viewpoint of the pilot, just like objects in the real world, thereby enhancing aiming flexibility and combat efficiency. Attached Figure Description

[0094] Figure 1 This is a schematic diagram of the overall technical solution architecture of the dynamic adjustment-based viewpoint display compensation method proposed in this invention;

[0095] Figure 2 This is a flowchart illustrating the overall logical flow of the dynamic adjustment-based viewpoint compensation method proposed in this invention.

[0096] Figure 3 This is a schematic diagram of the core principle framework of the content compensation processing in this invention;

[0097] Figure 4 This is a flowchart illustrating the logical process of multi-source data acquisition and preprocessing in this invention.

[0098] Figure 5 This is a flowchart illustrating the logical process of determining pilot viewpoint parameters and generating original display content in this invention. Detailed Implementation

[0099] This invention provides a dynamically adjusted viewing angle compensation method to address the problems of limited field of view, monochromatic display, and insufficient aiming flexibility in existing head-up displays (HUDs). Embodiments of this invention integrate large field-of-view full-color display technology and multi-source data fusion technology to dynamically adjust the displayed content, thereby enhancing pilots' battlefield situational awareness and optimizing flight and combat operations. The method and system of this invention will be described in detail below with reference to the accompanying drawings. Those skilled in the art will understand that various modifications and substitutions can be made to the described embodiments without departing from the essential spirit and scope of this invention.

[0100] Please refer to Figure 1-5The method proposed in this invention operates within an airborne display system environment. This system comprises multiple core functional modules that collaboratively determine the pilot's viewpoint parameters in real time, generate the original display content, and perform dynamic viewing angle compensation. The system integrates an advanced airborne sensor array, cockpit sensors for acquiring pilot physiological and behavioral data, a high-performance data processing and fusion unit, a module dedicated to display content compensation calculations, and a large field-of-view full-color display. The entire system architecture aims to achieve high-precision, low-latency data processing and display content presentation to support pilot decision-making and operations in complex situations.

[0101] First, the method of the present invention includes the following main steps:

[0102] S1. Acquire multi-source data and perform preprocessing;

[0103] S2. Determine the pilot's viewpoint parameters and generate the original display content;

[0104] S3. Perform display content compensation processing;

[0105] S4. Output the compensated display content.

[0106] The steps described above will be explained in detail below.

[0107] S1. Acquire multi-source data and perform preprocessing.

[0108] This step is the starting point of the entire perspective display compensation method. Its core lies in acquiring comprehensive information about the flight platform, the external environment, and the pilot's own status in real time and accurately, and performing unified cleaning, synchronization, and formatting of these heterogeneous data to lay the data foundation for subsequent viewpoint determination and display content generation.

[0109] To achieve multi-source data acquisition, the airborne display system upon which this invention is based integrates a multimodal sensor array. This array includes:

[0110] First, the flight platform status sensors. This sensor suite is responsible for collecting various dynamic parameters of the aircraft. For example, it acquires the aircraft's three-dimensional attitude angles, such as roll, pitch, yaw, angular velocity, and angular acceleration, in real time through the inertial measurement unit; it acquires the aircraft's real-time three-dimensional position coordinates, velocity, and acceleration data through the global positioning system module; it acquires air pressure, temperature, humidity, and wind speed and direction data of the airspace where the aircraft is located through atmospheric data sensors; and it acquires key flight parameters such as airspeed, altitude, engine status, and fuel level through the flight control system interface. All of this data is collected at a preset sampling frequency, such as 100Hz per second, and transmitted to the data processing unit with high priority via the onboard data bus.

[0111] Second, external environment perception sensors. This sensor suite is responsible for detecting information about the external environment around the aircraft. For example, it acquires obstacle distribution, target identification and tracking data in the airspace ahead through airborne radar systems, including target distance, orientation, speed, size and type; it acquires high-precision data on terrain undulations, cloud structure and airflow disturbances through lidar systems; and it acquires images or video streams of the external scene through airborne visual sensors such as visible light cameras and infrared cameras to identify ground landmarks, aerial targets, weather conditions and lighting conditions. These sensor data streams have different data structures and update frequencies, with some, such as image data, having high data bandwidth.

[0112] Third, pilot status awareness sensors. This sensor suite is deployed inside the cockpit to acquire the pilot's physiological and behavioral status in real time. For example, an eye-tracking system acquires fine-grained eye movement parameters such as the pilot's gaze point, pupil diameter, blink frequency, and fixation area. This data can directly indicate the pilot's focus of attention and cognitive load. A head posture tracking system acquires the pilot's three-dimensional head position and posture to infer the pilot's observation direction and intention. A voice recognition module listens to and parses the pilot's voice commands, such as task switching, information retrieval, and system control, and converts them into executable control signals.

[0113] After all multi-source data has been acquired, this step proceeds to the data preprocessing stage. Preprocessing is a crucial step in ensuring data quality and consistency, and includes the following sub-steps:

[0114] S101. Time Synchronization Processing: Due to the different sampling frequencies and data transmission delays of different sensors, precise timestamp synchronization of all acquired data is necessary. This synchronization is achieved through a global clock system, aligning all data to a unified time reference. For example, this is done by setting a timestamp field for data packets and using interpolation algorithms or time series alignment techniques such as dynamic time warping to compensate for sampling phase differences between different data sources.

[0115] S102. Data Calibration and Noise Filtering: The raw sensor data undergoes necessary calibration to eliminate sensor-specific errors such as zero-point drift and nonlinear response, as well as the influence of environmental factors like temperature on sensor readings. Simultaneously, digital filtering techniques such as Kalman filtering, extended Kalman filtering, or moving average filtering are applied to remove random noise and transient interference from the data, improving data stability and accuracy. For example, for measurement noise in radar target tracking data, an adaptive Kalman filter can be used for smoothing, improving the accuracy of target position and velocity estimation.

[0116] S103. Data Format Unification and Missing Value Handling: Data from different sensors and in different formats, such as analog signals, digital signals, image frames, and text commands, are uniformly converted into standardized data structures and data types within the system, such as floating-point matrices or structured data records. For missing values ​​that may occur during data transmission, various strategies can be used to fill them, such as linear interpolation based on historical data, polynomial interpolation, or using predictive models such as Long Short-Term Memory networks to infer and fill values ​​based on contextual information.

[0117] S104. Data Association and Fusion: After cleaning and standardizing the single-source data, preliminary fusion is performed on data from different modalities that have inherent relationships. For example, the aircraft's position data is associated with terrain data in the external environment to determine the aircraft's precise position relative to the ground; the pilot's line-of-sight is matched with external image data to identify the external object the pilot is currently looking at. This stage of fusion mainly involves early feature-level fusion, aiming to generate a unified, continuous, low-noise, and highly reliable multidimensional situational awareness dataset.

[0118] S2. Determine the pilot's viewpoint parameters and generate the original display content.

[0119] The core of this step lies in accurately determining the pilot's current viewpoint position, line of sight, and head posture in three-dimensional space based on preprocessed multi-source data, and generating a raw display content that comprehensively reflects flight and combat requirements. This content covers the entire three-dimensional space perceptible to the pilot and has been rendered in full color, but has not yet undergone dynamic perspective compensation.

[0120] S201. Determination of Pilot's Viewpoint Parameters:

[0121] This step comprehensively utilizes data from in-cockpit sensors to construct an accurate model of the pilot in the cockpit coordinate system.

[0122] First, a head attitude tracking system is used to acquire the real-time three-dimensional position and orientation of the pilot's helmet in the cockpit reference frame. This system typically combines an inertial measurement unit (IMU) with external optical tracking markers to provide high-precision, low-latency head motion data. The head tracking data includes the translation vector of the head relative to the cockpit center, such as meters, and the rotation matrix, such as Euler angles, with an update frequency typically higher than 100 Hz.

[0123] Secondly, an eye-tracking system is used to obtain the pilot's gaze vector. This system illuminates the pilot's eyes with an infrared light source and uses a miniature camera to capture the pupil and corneal reflection points, thereby calculating the eye's rotation angle and ultimately deriving the gaze direction vector. This vector is typically referenced to a head-centric coordinate system, and its update frequency can reach up to 200Hz.

[0124] By fusing head posture data with eye movement data, the pilot's true viewpoint, such as the center position of the eyeballs and the coordinates of the line of sight's point of contact in the external environment, can be accurately calculated. For example, the viewpoint position can be defined as the offset of the head's center point, and the direction of the line of sight can be obtained by multiplying the head posture matrix and the eye rotation matrix. The viewpoint parameters are the basis for the subsequent dynamic adjustment of the displayed content, and their accuracy directly affects the naturalness and immersion of the compensation effect.

[0125] S202. Generation of original display content:

[0126] While determining the pilot's viewpoint parameters, this sub-step utilizes flight platform status data, external environment perception data, and current mission planning information to generate raw, full-color, uncompensated display content covering the entire 360° field of view. This content includes the following key components:

[0127] First, flight information display: This includes the aircraft's real-time attitude, such as airspeed, altitude, vertical speed, angle of attack, sideslip angle, geographic coordinates, heading, wind speed and direction, engine performance parameters, and fuel consumption. This information is displayed in instrument panel, digital, or graphical form, such as altimeter, airspeed indicator, and attitude indicator, at specific locations in three-dimensional space, usually directly in front of the pilot or dynamically arranged according to mission importance.

[0128] Second, mission information display: This includes detailed planning of the current flight mission, such as flight path charts, target waypoints, target identification and lock-on information, attack range, no-fly zone warnings, and threat indications such as enemy aircraft positions, missile attack directions, and friendly aircraft positions. This information is presented in the form of symbols, boundary lines, path indicators, or color-coded areas overlaid on the external scene.

[0129] Third, situational awareness imagery: Combining data from airborne radar, infrared, and visible light cameras, among other external environmental perception sensors, a high-resolution 360° panoramic fused image is generated as a realistic representation of the external world. This image undergoes multispectral fusion and enhancement processing, such as pixel-level fusion of infrared and visible light images, to provide clear visual information under complex weather or nighttime conditions. The image data is projected onto a virtual spherical or cylindrical display surface to simulate the real external environment.

[0130] Fourth, alarms and alerts: These include critical safety alerts such as system malfunction warnings, dangerous proximity warnings, low fuel warnings, and stall warnings. These alerts are typically presented as visual symbols with highlighted colors, flashing animations, or specific sound effects, and are prioritized in the display space according to their urgency.

[0131] When generating the original display content, the system performs a unified spatial coordinate system transformation on all information and places it in a shared 3D virtual scene for subsequent compensation processing. The center of this 3D virtual scene is usually set at the pilot's viewpoint, and all objects are positioned based on a global coordinate system.

[0132] S3. Perform display content compensation processing.

[0133] This step is the core of the invention, aiming to dynamically and adaptively transform, occlusion-handling, and render the original displayed content based on the pilot's real-time viewpoint parameters and environmental context. This ensures the accuracy, readability, and immersive experience of the displayed information from different viewing angles. This compensation process eliminates problems such as parallax, distortion, and limited field of view inherent in traditional head-up displays, truly achieving an organic integration of displayed information and the external environment.

[0134] S301. Geometric projection compensation based on viewpoint parameters:

[0135] This sub-step performs precise perspective projection transformation on all virtual objects in the original display content based on the pilot's real-time 3D viewpoint position and line of sight.

[0136] Each 3D point in the original display content, such as a vertex on a virtual dashboard, has a fixed world coordinate in the 3D virtual scene. The pilot's viewpoint can be defined as a camera model, which contains a position vector, for example... and a direction matrix, for example Based on these parameters, points in world coordinates can be projected onto the pilot's two-dimensional observation plane, such as the surface of a display screen. The projection transformation can be expressed by the following formula:

[0137] ;

[0138] in, Represents the three-dimensional world coordinates in the original displayed content; This represents the view matrix from the world coordinate system to the pilot's viewpoint coordinate system, determined by the pilot's viewpoint position and attitude. This represents the perspective projection matrix that projects three-dimensional viewpoint coordinates onto a two-dimensional display plane. The parameters of this matrix, such as the field of view, near clipping plane, and far clipping plane, are configured according to the physical characteristics of a large field of view display. This represents the final two-dimensional pixel coordinates on the display.

[0139] Through the above transformation, regardless of how the pilot's head moves or how their gaze shifts, the displayed content can be reprojected in real time relative to their current viewpoint, thereby maintaining the spatial consistency of the displayed information with the external world and eliminating the offset and distortion of the displayed content caused by changes in viewpoint.

[0140] S302. Environmental and Display Content Obstruction Management:

[0141] This step is designed to address situations where, when a pilot is observing the external environment, the displayed content may be obstructed by the aircraft cockpit structure, such as the instrument panel, fuselage components, or external obstacles such as the wings, engines, or even other displayed content itself.

[0142] The system builds a 3D geometric model that accurately represents the internal geometry of the cockpit and the critical external structures of the aircraft. Before rendering the displayed content, the system performs a depth test or occlusion query algorithm. For each virtual information element to be displayed, the system determines whether it is occluded by any actual physical structures or higher-priority virtual information in the pilot's line-of-sight path.

[0143] If an information element is partially or completely obscured, the system will adopt the following compensation strategies based on preset priority rules and the degree of obscuration:

[0144] First, adjust the transparency: For non-critical information that is slightly obscured, you can appropriately reduce its transparency so that it appears semi-transparent behind the obscuring object, so as to indicate its existence without interfering with the main observation.

[0145] Second, position offset: For critical information, if its current position is obscured, the system will intelligently calculate an optimal offset position and dynamically move it to a blank area visible to the pilot, while maintaining its logical association with the original target or original position, such as through indicator lines or arrows.

[0146] Third, information simplification or scaling: When display space is limited or information is dense, the system can automatically simplify the displayed content, such as only displaying key values ​​while hiding secondary text, or dynamically scaling the information to make it clearly visible within a limited space.

[0147] Fourth, visual enhancement: For obscured but extremely important targets, such as enemy aircraft, the system can generate a visual indicator, such as a highlighted arrow or a small thumbnail, on the obstruction, such as the edge of the cockpit strut or a blank area of ​​the display, to indicate the location and distance of the obscured target.

[0148] S303. Optimization of dynamic rendering and display effects:

[0149] After completing geometric projection and occlusion processing, this sub-step performs final rendering and visual optimization of the compensated display content to adapt to the pilot's physiological characteristics and current environmental conditions.

[0150] First, adaptive adjustment of color and brightness: Based on the intensity of external ambient light, such as daytime, nighttime, dusk, and the pilot's visual adaptation state, the overall brightness, contrast, and color saturation of the displayed content are dynamically adjusted. For example, in bright light environments, the display brightness is increased to improve visibility; during night flights, the display is adjusted to a low-brightness, high-contrast mode to protect the pilot's night vision.

[0151] Second, anti-aliasing and sharpening: Advanced rendering techniques such as multisampling anti-aliasing and morphological anti-aliasing are applied to eliminate jagged edges on displayed content and improve the smoothness of graphics. Simultaneously, some key information can be sharpened to make it clearer and more prominent, such as enhancing the edges of the aiming crosshairs and target tracking boxes.

[0152] Third, focus area enhancement: Based on the pilot's real-time gaze location, the displayed information within the current gaze area is locally enhanced. For example, the detail resolution, brightness, or color vibrancy of that area is increased, while information in the peripheral areas of the gaze is appropriately weakened to guide the pilot's attention and reduce cognitive load.

[0153] Fourth, motion blur and jitter compensation: High-speed moving aircraft or rapidly changing external scenes may cause motion blur in the displayed content. The system can apply motion compensation algorithms, such as predictive motion estimation, to reduce image blur. Simultaneously, for minor vibrations or jitters that may occur during flight, the system will fine-tune the displayed image using image stabilization algorithms to ensure the stability and readability of the displayed content.

[0154] S4. Output the compensated display content.

[0155] This step is the end of the entire methodology chain, responsible for presenting the finely compensated full-color, wide-field-of-view display content to the pilot in a high refresh rate and low latency manner.

[0156] The specific output process includes the following steps:

[0157] S401. Rendering Data Stream Transmission: The compensation calculation unit transmits the processed 2D image data or rendering command stream to the large field-of-view full-color display via a high-speed data interface such as Fibre Channel or a dedicated display bus. The transmission protocol ensures data integrity and transmission efficiency to avoid introducing additional latency during the output stage.

[0158] S402. Display Characteristic Matching: Before data is transmitted to the display, the system performs a final adaptation adjustment on the rendered data based on the specific physical characteristics of the connected large field-of-view display, such as resolution, color depth, refresh rate, curvature parameters, and field of view. This includes pixel format conversion, color space mapping (e.g., mapping from the rendering pipeline's internal color space to the display's native color space), and geometric correction to compensate for the display's own physical distortions. For example, for curved displays, an anti-distortion mapping is applied to counteract the display's curvature effect, ensuring that the image appears flat or has correct perspective.

[0159] S403. High Refresh Rate Display: The large field-of-view full-color display updates the screen in real time with a high refresh rate, such as a minimum of 60Hz, typically reaching 120Hz or even higher. The high refresh rate ensures that pilots perceive continuous, smooth images during rapid head movements or intense flight maneuvers, effectively avoiding screen tearing and stuttering, thus maintaining immersion and timely operation.

[0160] S404. Real-time Feedback and Calibration: The display system also includes a closed-loop feedback mechanism. Internal optical sensors or external calibration equipment periodically monitor the actual output of the displayed image, such as brightness uniformity, color accuracy, and geometric precision. This monitoring data is sent back to the data processing unit for further fine-tuning of the compensation algorithm parameters, such as adjusting the brightness curve or color lookup table, to ensure optimal display performance over long-term use. This feedback mechanism adaptively addresses performance drift caused by monitor aging or environmental changes, guaranteeing consistently high-quality display content.

[0161] Through the coordinated execution of the above steps, this invention achieves dynamically adjusted viewpoint display compensation, completely revolutionizing the way pilots acquire and process information. Pilots can flexibly view targets and acquire information in a 360° panoramic field of view without rotating the aircraft, simply by moving their eyes or head, with the displayed content in full color, greatly enhancing situational awareness and combat efficiency. For example, when the pilot's gaze shifts from the front to a side target, the system can smoothly transition and reposition relevant target information, such as aiming symbols and distance data, from the front view to the side display area in real time. Simultaneously, it dynamically adjusts the displayed content based on the pilot's gaze point, such as enhancing the clarity of target symbols or adding auxiliary information, ensuring the continuity and comprehensibility of information. This allows pilots to interact with the flight environment more naturally and intuitively, ultimately achieving a higher level of flight safety and mission success rate.

[0162] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0163] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamic adjustment-based viewpoint display compensation method, characterized in that, include: Real-time acquisition of pilot head posture data; Real-time acquisition of aircraft status data; Real-time acquisition of external environment perception data; The pilot's head posture data, the aircraft status data, and the external environment perception data are preprocessed to obtain a dataset after time synchronization, data alignment, and noise filtering. Based on the preprocessed pilot head posture data, the aircraft status data, and the geometric parameters of the large field of view display, the three-dimensional position and orientation of the pilot's current viewpoint in the aircraft coordinate system are determined. Based on the preprocessed aircraft status data, the preprocessed external environment perception data, and flight mission planning information, the original display content is generated; Based on the pilot's current viewpoint, the original display content, and the physical geometric model of the large field of view display, a geometric transformation matrix is ​​calculated and applied in real time to perform perspective projection and distortion correction on the original display content, generating compensated display content. The compensated display content is then presented on a large field-of-view display.

2. The viewing angle compensation method based on dynamic adjustment according to claim 1, characterized in that, The real-time acquisition of pilot head posture data includes: The infrared images of marked points on the pilot's helmet are captured in real time by an array of at least three infrared cameras installed inside the cockpit, the sampling frequency of which is higher than 150Hz. The pixel coordinates of the marker points are extracted from the infrared image using an image processing algorithm; Using multi-point positioning and triangulation algorithms, based on the pixel coordinates of the marked points and the known spatial position and calibration parameters of the infrared camera array, the six-degree-of-freedom position and attitude of the pilot's head in the aircraft coordinate system are calculated.

3. The viewing angle compensation method based on dynamic adjustment according to claim 2, characterized in that, The real-time acquisition of aircraft status data includes: The inertial navigation system acquires the aircraft's real-time attitude angles, angular velocity, acceleration, and angular acceleration data. The real-time three-dimensional position coordinates and velocity data of the aircraft are obtained through a GPS receiver. The air pressure, temperature, humidity, wind speed, and wind direction data of the airspace where the aircraft is located are obtained through atmospheric data sensors. By using a data fusion algorithm, the inertial navigation system data, the global positioning system receiver data, and the atmospheric data sensor data are fused together to obtain high-precision real-time status data of the aircraft.

4. The viewing angle compensation method based on dynamic adjustment according to claim 3, characterized in that, The real-time acquisition of external environment perception data includes: Multiple high-resolution full-color cameras mounted on the exterior of the aircraft capture real-time images of the environment surrounding the aircraft. The environmental images are stitched together in real time using an image stitching algorithm to generate a 360-degree panoramic environmental image. The space distribution, relative position, and relative velocity data of external targets are acquired in real time by radar and lidar sensors installed on the outside of the aircraft. External targets are identified, located, and tracked from the data of the radar sensor and the lidar sensor using a target detection and tracking algorithm.

5. The viewing angle compensation method based on dynamic adjustment according to claim 4, characterized in that, The preprocessing of the pilot's head posture data, the aircraft status data, and the external environment perception data includes: All collected data are timestamped to unify the data from different sensor sampling frequencies to the preset sampling frequency; Data alignment is performed on the time-synchronized data to ensure accurate time relationships between different data streams; The aligned data is then subjected to noise filtering to remove random errors and interference from the sensor acquisition process.

6. The viewing angle compensation method based on dynamic adjustment according to claim 5, characterized in that, Determining the pilot's current viewpoint's three-dimensional position and orientation in the aircraft coordinate system includes: The pilot's head attitude data is converted into the pilot's viewpoint position vector and line-of-sight direction vector in the aircraft coordinate system; Based on the installation position and geometric parameters of the large field-of-view display within the aircraft cockpit, the distance and relative angle from the pilot's viewpoint to the display surface are calculated.

7. The viewing angle compensation method based on dynamic adjustment according to claim 6, characterized in that, The generation of the original display content includes: Based on the external environment perception data, a background environment image and external target indication information are generated; Based on the aircraft status data, flight instrument data, navigation path information, and flight warning information are generated; Based on flight mission planning information, generate mission objectives, route planning, and threat area indication information; The above information is superimposed and synthesized according to preset priorities and layout rules to form a two-dimensional or three-dimensional original display content data stream.

8. The viewing angle compensation method based on dynamic adjustment according to claim 7, characterized in that, The real-time calculation and application of the geometric transformation matrix to perform perspective projection and distortion correction on the original display content, generating compensated display content, includes: Based on the pilot's current viewpoint, the three-dimensional spatial position of the original displayed content, and the physical geometric model of the large field-of-view display, calculate the perspective projection matrix corresponding to the current viewpoint; Calculate the distortion correction matrix based on the optical distortion characteristics of the large field-of-view display; The original display content is geometrically transformed in real time using the perspective projection matrix and the distortion correction matrix, projecting it from the original space onto the surface of the large field-of-view display to eliminate parallax and geometric distortion. The calculation and application frequency of the geometric transformation matrix is ​​consistent with the frequency of the pilot's head attitude data acquisition.

9. A viewing angle compensation system based on dynamic adjustment, characterized in that, include: The pilot head attitude tracking module is used to acquire pilot head attitude data in real time. The head attitude data includes the three-dimensional position and three-dimensional attitude of the pilot's head in the aircraft coordinate system. The aircraft status data acquisition module is used to acquire aircraft status data in real time, including the aircraft's position, attitude, velocity, and acceleration. The external environment perception module is used to acquire external environment perception data in real time, including panoramic images of the aircraft's external environment and the three-dimensional position and velocity of the target. The data preprocessing module is used to perform time synchronization, data alignment, and noise filtering on the pilot's head posture data, the aircraft status data, and the external environment perception data. The viewpoint calculation module is used to determine the three-dimensional position and orientation of the pilot's current viewpoint in the aircraft coordinate system based on the preprocessed pilot head posture data, the aircraft status data, and the geometric parameters of the large field of view display. The display content generation module is used to generate original display content based on the preprocessed aircraft status data, the external environment perception data, and the flight mission planning information. The compensation processing module is used to calculate and apply a geometric transformation matrix in real time based on the pilot's current viewpoint, the original display content, and the physical geometric model of the large field of view display, to perform perspective projection and distortion correction on the original display content and generate compensated display content. A large field-of-view display module is used to present the compensated display content on a large field-of-view display.

10. A viewing angle compensation system based on dynamic adjustment according to claim 9, characterized in that, The pilot head attitude tracking module includes: At least three infrared camera arrays, located inside the cockpit, are used to capture infrared images of marked points on the pilot's helmet in real time, and the sampling frequency of the infrared camera arrays is higher than 150Hz; An image processing unit is used to extract the pixel coordinates of the marker points from the infrared image; The attitude calculation unit is used to calculate the six-degree-of-freedom position and attitude of the pilot's head in the aircraft coordinate system based on the pixel coordinates of the marked points and the known spatial position and calibration parameters of the infrared camera array, using multi-point positioning and triangulation algorithms.

Images