Airborne 3D light field display system based on integrated imaging technology

Abstract

The invention relates to the technical field of three-dimensional light field display, in particular to an airborne 3D light field display system based on an integrated imaging technology, which is a system used for carrying out real-time three-dimensional display on flight state and environment information in an aircraft cockpit. The system comprises a terrain database module, an aircraft attitude database module, a flight data acquisition module, a data processing module, a mini LED display screen, a diaphragm array, a lens array and a real-time rendering module. The system can visually display the three-dimensional attitude and the surrounding environment of an aircraft, remarkably shortens the information processing time of a pilot, improves the key information screening capability and the space loss recovery efficiency, can still provide clear display under the condition of strong ambient light, is suitable for scenes such as cruise, take-off and landing, low visibility and the like, and effectively improves the flight safety.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional light field display technology, and in particular to an airborne 3D light field display system based on integrated imaging technology, which is used in an aircraft cockpit to display flight status and environmental information in real time. Background Art

[0002] Modern aircraft cockpits are equipped with a variety of display devices to convey various flight information to pilots. 1. Currently, instrument systems based on two-dimensional displays are widely used. This traditional display method carries a limited amount of information, and the massive amount of flight data generated every second during flight is difficult to efficiently present within a limited space. Pilots need to observe multiple, scattered two-dimensional displays to obtain complete flight information, which not only increases the complexity of the cockpit layout but also increases the cognitive burden on pilots to filter effective information. Statistics from the U.S. National Transportation Safety Board (NTSB) show that between 2000 and 2020, improperly designed information loss accounted for 12% of accidents. 2. Traditional two-dimensional display technology requires pilots to convert abstract data into an understanding of the aircraft's position and attitude in space. This conversion process consumes a significant amount of cognitive resources. According to the latest research data from the U.S. Federal Aviation Administration (FAA), human factors account for 60% to 80% of aviation accidents in recent years, with improper information processing being one of the main causes. 3. Spatial disorientation is a major technical issue affecting flight safety. According to SKYbrary aviation safety data, spatial disorientation is more likely to occur when the horizon is not visible, such as at night or in instrument meteorological conditions (IMC). The risk of spatial disorientation is further increased when flight instruments malfunction, workload is high, or crew resource management fails. Traditional abstract instrument data requires pilots to perform complex cognitive transformations to understand the aircraft's spatial attitude, a process that is more prone to errors under stress. Therefore, in special circumstances such as inclement weather, low visibility, or spatial disorientation, pilots urgently need display technologies that can intuitively present the aircraft's attitude and surrounding environment, improving decision-making speed and reducing the risk of terrain misinterpretation.

[0003] Aircraft display environments also face the serious challenge of interference from strong ambient light. According to a special study conducted between 2008 and 2015 by Dr. Adrian Chorley of the UK Civil Aviation Authority (CAA), long-wave ultraviolet A (UVA) poses the greatest risk to pilots' eyes, as a higher percentage of UVA rays can penetrate the cockpit. Long-term exposure to solar radiation (particularly UV and blue light components) is a risk factor for cataracts and age-related retinal degeneration. A survey of professional pilots in the UK revealed that approximately 25% rarely or never use sunglasses, primarily due to the lens depth interfering with instrument readability. This indicates that traditional displays have low light efficiency, making it difficult to clearly view displayed content in strong light conditions, severely impacting pilots' ability to access critical information.

[0004] As a next-generation information display technology, three-dimensional (3D) display technology can present data in a more intuitive, stereoscopic manner, allowing viewers to view 3D images without any auxiliary equipment. Integrated imaging light field 3D display technology can simultaneously provide all physiological depth cues, including focus adjustment, motion parallax, binocular convergence, and binocular parallax, thus avoiding stereoscopic visual fatigue. However, these technologies currently face challenges in specific applications in airborne environments. First, airborne environments place extremely high demands on the real-time performance of display systems. Conventional real-time rendering methods are computationally complex and cannot meet the stringent display latency requirements of flight environments. Second, there is a lack of effective solutions for the combined display of flight data and spatial position information. Third, traditional displays have low light efficiency and cannot meet the real-world requirements of strong ambient light conditions. Therefore, a 3D light field display system that can intuitively display aircraft attitude and surroundings in real time and with high light efficiency is needed to meet the needs of efficient and intuitive presentation of aircraft cockpit information. Summary of the Invention

[0005] The present invention aims to provide an airborne 3D light field display system based on integrated imaging technology. This system can directly display the aircraft's three-dimensional environment and the aircraft's three-dimensional real-time attitude. High light utilization is achieved through a light-collecting display design, and clear display is possible even in strong light environments. This helps pilots quickly establish accurate spatial perception, reduces information processing time, and improves the ability to screen key information. This system allows pilots to intuitively understand the aircraft's status, particularly in situations of spatial disorientation or high-stress environments, thereby enabling them to effectively control the aircraft in a more timely and accurate manner.

[0006] The present invention is achieved through the following technical solutions:

[0007] The terrain database module is used to store pre-established three-dimensional terrain data, including detailed three-dimensional information such as airports, routes and surrounding terrain.

[0008] The aircraft attitude database module contains the aircraft three-dimensional model view data at various attitude angles (mainly including azimuth, pitch angle, roll angle, flight altitude, etc.), which are pre-collected and encoded and stored in micro-image array format.

[0009] The flight data acquisition module is used to collect the aircraft's position, attitude, altitude, speed and other flight parameters in real time.

[0010] The data processing module quickly retrieves the corresponding terrain data and aircraft attitude view from the database based on the real-time flight data of the aircraft, and performs synthesis processing.

[0011] The mini LED display screen is an array composed of multiple mini LED light source units. Each mini LED light source unit is a scattered light source used to load the corresponding sub-image in the primitive image array.

[0012] The aperture array is used to limit the beam diameter entering each lens, suppressing aberrations and display crosstalk. Each aperture unit corresponds to a mini LED light source unit and a lens unit.

[0013] The lens array, together with the mini LED display, forms the first-level integrated imaging display structure, which can convert scattered light into a more collimated output angle and directly form a viewing area. The distance between each lens unit and the corresponding mini LED light source unit is equal to the focal length of the lens.

[0014] The real-time rendering module is used to directly map spatial coordinates into 3D light field pixel encoding coordinates, reducing rendering complexity.

[0015] The present invention establishes three key one-to-one correspondences: each mini LED light source unit corresponds to an EIA sub-image, each EIA sub-image corresponds to an aperture unit, and each aperture unit corresponds to a lens unit in a lens array, forming

[0016] The working principle of the present invention is as follows: the system pre-establishes a detailed terrain map database and a three-dimensional model database of various aircraft postures; during the flight, the system retrieves the corresponding terrain data and aircraft posture model from the database based on the aircraft's real-time position, posture and other flight data, and synthesizes them through the data processing module; the real-time rendering module directly maps the spatial coordinates into 3D light field pixel encoding coordinates, greatly reducing the rendering complexity; the scattered light emitted by the mini LED display is limited by the aperture array and then becomes a more collimated exit angle through the lens array, realizing a light-collecting display and greatly improving light utilization; the simplified single-stage optical structure directly forms a viewing area, presenting a high-brightness three-dimensional image.

[0017] The present invention has the following gain effects:

[0018] Shortened pilot reaction time: The intuitive presentation of the aircraft's three-dimensional posture and surrounding environment reduces the pilot's process of converting abstract data into spatial cognition, significantly shortening the time from observation to reaction.

[0019] Improved ability to filter key information: 3D displays can more effectively highlight important information, reducing the burden on pilots to filter key information from multiple 2D displays in different spatial distributions amidst massive amounts of data.

[0020] Adaptability to strong light environments: The light-collecting display design focuses light that would otherwise be scattered at other angles directly in the pilot's viewing direction, significantly improving light utilization and providing clear display even in strong ambient light conditions such as strong sunlight or passing through clouds.

[0021] Improved system reliability: The simplified single-stage optical structure reduces the number of optical components, reduces system complexity, and improves reliability and stability in airborne environments.

[0022] Enhanced spatial disorientation recovery: In the event of spatial disorientation, extreme weather or limited vision, the system provides a three-dimensional intuitive display to help pilots quickly regain awareness of the aircraft's position and attitude.

[0023] Improved flight safety: By providing more intuitive flight information, the probability of human misjudgment and incorrect operation is reduced, thereby improving overall flight safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the overall architecture of the system of the present invention.

[0025] Figure 2 It is a cross-sectional schematic diagram of the core display structure of the system of the present invention.

[0026] Figure 3 This is a diagram showing the working principle of the aperture array to suppress aberrations.

[0027] Figure 4 This is a diagram showing the working principle of the lens array light collection display.

[0028] Figure 5 It is a schematic diagram of the system's display effect in different flight scenarios. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] Example 1: Overall system architecture. Figure 1 As shown, the airborne 3D light field display system based on integrated imaging technology of the present invention includes a terrain database module 101, an aircraft attitude database module 102, a flight data acquisition module 103, a data processing module 104, a mini LED display screen 105, an aperture array 106, a lens array 107, and a real-time rendering module 108.

[0031] Example 2: Core display structure and working principle. Figure 2As shown, the core display structure of the present invention is as follows from back to front: a mini LED display screen 105, which is composed of multiple mini LED light source units, each mini LED light source unit is a scattered light source, responsible for loading the corresponding EIA sub-image in the primitive image array; an aperture array 106, which corresponds to the lens array 107, and the aperture unit aperture is smaller than the lens unit aperture, and is used to limit the aperture of the light beam entering the lens, suppress aberrations and display crosstalk; the lens array 107 uses a lens unit with a medium focal length (5mm) and a large relative aperture. The distance from the mini LED display screen 105 is equal to the focal length of the lens, which converts the scattered light into a more collimated exit angle, and forms an integrated imaging display structure with the mini LED display screen, directly forming a viewing area and displaying a three-dimensional image.

[0032] Working Principle: Each mini LED light source unit in the mini LED display 105 emits scattered light. The light is then restricted by the aperture array 106 and passed through the lens array 107. Because the distance between the mini LED and the lens is equal to the focal length, the scattered light is converted to a more collimated emission angle, directly forming the viewing area for the three-dimensional image. This light-collecting design concentrates light that would otherwise be scattered at various angles directly in the pilot's viewing direction, significantly improving light utilization. The simplified single-stage optical structure reduces light loss and improves system efficiency and reliability.

[0033] Example 3: Implementation of Three One-to-One Correspondences. This invention establishes three key one-to-one correspondences: First, each mini LED light source unit corresponds to an EIA sub-image, ensuring that each light source unit displays specific primitive image content; second, each EIA sub-image corresponds to an aperture element in aperture array 106, ensuring accurate light beam confinement; and third, each aperture element corresponds to a lens element in lens array 107, achieving precise optical imaging. The entire system forms a complete "mini LED → EIA → aperture → lens" correspondence chain, ensuring accurate light field reconstruction and consistent display effects.

[0034] Example 4: Method for implementing aperture array to suppress aberration. Figure 3 As shown, the aperture units in aperture array 106 correspond to the lens units in lens array 107, and are used to limit the aperture of the light beam passing through each lens. The aperture unit's aperture is smaller than the lens unit's aperture. When the mini LED light source unit emits scattered light in all directions, only the central beam passing through the aperture can be imaged by the lens, while the peripheral beams are blocked by the aperture. This helps reduce the image point's diffuse spot, improves image quality, and suppresses aberrations when the viewing angle changes.

[0035] The aperture array 106 is fabricated by using a precision photolithography process to create opaque areas on an optically transparent substrate. The aperture unit has a diameter of 1.2 mm, which is smaller than the lens unit's diameter (1.5 mm). The distance between the aperture array and the lens array is precisely calculated and adjusted to ensure optimal light field reconstruction.

[0036] Example 5: Method for realizing light collection display by lens array. Figure 4 As shown, the present invention adopts a single-stage lens array structure, and the lens array 107 and the mini LED display screen 105 constitute an integrated imaging display structure, which converts scattered light into more collimated output light and directly forms a viewing area.

[0037] The lens array operating parameters are as follows: the focal length f of the lens array is 5mm, the lateral dimensions of the lens unit are 1.5mm, and the longitudinal dimensions are 1.5mm. The distance from the mini LED display to the lens array is equal to f, and the viewing distance d is set to 500mm, meeting the viewing requirements of an airborne display environment. The simplified single-stage structure reduces the number of optical components, improves light transmission efficiency, and reduces system complexity.

[0038] Example 6: Implementation method of real-time rendering module. The real-time rendering module uses the rectangular coordinate mapping method to map the spatial coordinates (X, Y, Z) to 3D light field pixel encoding coordinates (u, v, k), where u represents the uth column, v represents the vth row, and k represents the kth sub-pixel. The mapping relationship is: n = N / P fmod[(3xu+3xvxtan(θ)+k),P], where P is the grating pitch and the grating inclination is θ; fmod is the modulo function; N is the number of viewpoints, and n represents the viewpoint number. This direct mapping method avoids the intensive viewpoint rendering process required by traditional rendering methods, greatly reducing rendering time and computational complexity.

[0039] Example 7: Method for synthesizing aircraft attitude and terrain images. The present invention adopts a special method for synthesizing aircraft attitude and terrain images, which greatly reduces the rendering complexity and calculation requirements. The aircraft attitude database pre-records the three-dimensional model images of the aircraft at various attitude angles (such as azimuth, pitch angle, roll angle). The aircraft part in these images is a normal pixel, while the background part uses a special value that sets the RGB value to a non-display grayscale, such as the annotation method of the RGB value (256, 256, 256). Since the maximum RGB value of an 8-bit display system is (255, 255, 255), (256, 256, 256) is actually a special value that cannot be displayed by a display, and is specifically used to identify the area that needs to be replaced by the terrain image. During the image synthesis process, the system first retrieves the terrain image of the current flight position from the terrain database, and then retrieves the aircraft image that best matches the current attitude from the aircraft attitude database. During synthesis, each pixel position is evaluated: if the RGB value at that location in the aircraft image is a special non-display grayscale value of (256, 256, 256), the corresponding pixel value in the terrain image is directly used; if it is not a special non-display grayscale value, the original pixel value in the aircraft image is retained. This synthesis method is essentially a selective filling process between two matrices, eliminating the need for complex rendering calculations and requiring only simple pixel replacement operations, significantly improving the system's real-time performance.

[0040] Example 8: Application of the system in normal flight and extreme conditions. Figure 5 As shown, this system clearly and intuitively presents the aircraft's attitude and surroundings during normal flight, enabling pilots to understand flight status without complex cognitive transitions. In extreme conditions (such as spatial disorientation, low visibility, or partial system failure), the system can help pilots quickly regain situational awareness, enhancing flight safety. The light-collecting display design ensures clear visibility, especially under strong ambient light conditions such as sunlight or passing through cloud cover.

[0041] Specific application scenarios include: during the cruising phase, the system intuitively displays the aircraft's altitude and attitude relative to the terrain, while showing the route and terrain ahead; during the take-off and landing phases, the system provides detailed airport environment and runway information, and intuitively displays the relative position and attitude of the aircraft and the runway; in low visibility conditions, the system provides clear three-dimensional situational awareness based on pre-stored precise three-dimensional terrain and aircraft attitude data; in the event of spatial disorientation, the system can immediately provide intuitive three-dimensional situational awareness to help pilots quickly regain a correct understanding of the aircraft's position and attitude; in strong light environments, the light-collecting display ensures clear display of key information.

[0042] The present invention integrates a variety of advanced technologies, including mini LED light-collecting display, aperture array aberration suppression, simplified single-stage optical structure, real-time rendering optimization and efficient aircraft attitude and terrain image synthesis method, to solve the various limitations of traditional integrated imaging 3D display technology, especially the display clarity problems under high system complexity and strong ambient light conditions. It provides the aircraft cockpit with a more intuitive, clear, real-time, high-brightness and high-reliability three-dimensional information presentation system, greatly improving flight safety.

Claims

1. An airborne 3D light field display system based on integrated imaging technology, characterized in that: include: A terrain database module is used to store pre-established three-dimensional terrain data; The aircraft attitude database module is used to store the three-dimensional model and attitude information of the aircraft in different flight states; the flight data acquisition module is used to collect the aircraft's position, attitude, altitude, and speed flight parameters in real time; the data processing module is used to retrieve the corresponding data from the terrain database and the aircraft attitude database based on the real-time flight data, and perform synthesis processing; the mini LED display is used to load the corresponding sub-image in the primitive image array; the aperture array is used to limit the aperture of the light beam entering each lens, suppress aberrations and display crosstalk, and each aperture unit corresponds to a mini LED light source unit and a lens unit one-to-one; the lens array and the mini LED display constitute the first-level integrated imaging display structure, which converts scattered light into a more collimated output angle and directly forms a viewing area. The distance between each lens unit and the corresponding mini LED light source unit is equal to the focal length of the lens; A real-time rendering module is used to directly map spatial coordinates into 3D light field pixel-encoded coordinates.

2. The airborne 3D light field display system based on integrated imaging technology according to claim 1, characterized in that: Three one-to-one correspondences are established: each mini LED light source unit corresponds to an EIA sub-image, each EIA sub-image corresponds to an aperture unit, and each aperture unit corresponds to a lens unit in the lens array, forming a complete "mini LED → EIA → aperture → lens" chain structure.

3. The airborne 3D light field display system based on integrated imaging technology according to claim 1, characterized in that: The real-time rendering module uses a rectangular coordinate mapping method to map the spatial coordinates (X, Y, Z) into 3D light field pixel encoding coordinates (u, v, k), where u represents the uth column, v represents the vth row, and k represents the kth sub-pixel; the mapping relationship is: n = N / P fmod[(3xu+3xvxtan(θ)+k),P], where P is the grating pitch and the grating inclination angle is θ; fmod is the modulo function; N is the number of viewpoints, and n represents the viewpoint number.

4. The airborne 3D light field display system based on integrated imaging technology according to claim 1, characterized in that: In the aircraft attitude database module, the aircraft 3D model view is normal pixels, and the background part uses special values ​​that set the RGB value to a non-display grayscale to identify the area that needs to be replaced by the terrain image.

5. The airborne 3D light field display system based on integrated imaging technology according to claim 1, characterized in that: When the data processing module performs image synthesis, it judges each pixel position: if the RGB value of the position in the aircraft image is a special value that does not display grayscale, the pixel value of the corresponding position in the terrain image is directly used; if it is not a special value that does not display grayscale, the original pixel value in the aircraft image is retained.

6. The airborne 3D light field display system based on integrated imaging technology according to claim 1, characterized in that: The aperture array forms a light-proof area on an optically transparent substrate through a precision photolithography process. The aperture of the aperture unit is smaller than the aperture of the lens unit, which is used to limit the aperture of the light beam passing through each lens and reduce the diffuse spot of the image point.

7. The airborne 3D light field display system based on integrated imaging technology according to claim 1, characterized in that: The system uses a light-collecting display design to concentrate light that would have been scattered at other angles into the pilot's viewing direction, significantly improving light utilization and providing clear display even under strong ambient light conditions.

Images