A pilot ground vision training device considering environmental and physiological factors

Abstract

The present invention is applicable to the field of pilot training technology and provides a ground-based vision training device for pilots that takes environmental and physiological factors into account. The device uses a computer system to rapidly simulate human visual characteristics, including: establishing a light source and brightness detector; obtaining a two-dimensional distributed brightness matrix of the scene; obtaining ambient brightness information of the external scene as perceived by the human eye, which affects optical aberrations by affecting pupil diameter; establishing an optical model of the human eye and obtaining a modulation transfer function; simulating the observation effect of the external scene after passing through the model in the frequency domain, using a GPU for accelerated processing; using an eye tracker to determine the coordinates of the pilot's gaze point, simulating the blur and depth of field characteristics of the peripheral vision; and finally performing image fusion to obtain a simulated image of the external scene after passing through the human eye. The present invention can improve the effectiveness of flight training or the accuracy of new aircraft cockpit evaluations, and provide pilots with a more realistic and intuitive flight visual experience during ground training.

Description

Technical Field

[0001] The invention belongs to the technical field of pilot training, and in particular relates to a ground vision training device for pilots taking environmental and physiological factors into consideration. Background Art

[0002] During ground-based physical cockpit training for pilots, or during evaluation and testing of new aircraft cockpits, accurately simulating the pilot's external visual perception under various flight conditions on a projection screen is a crucial step. However, existing simulation technologies face multiple challenges in this area. First, dynamic changes in external environmental factors during flight, such as the angle of sunlight and ambient light brightness, place extremely high demands on the generation of simulated images. Second, as the pilot's gaze distance changes, the human eye model itself undergoes corresponding adjustments, leading to changes in its perceptual characteristics. This variability further exacerbates the difficulty of generating simulated images, as it must ensure that the images accurately reflect the human eye's true perception under varying conditions. Furthermore, existing simulation technologies face bottlenecks in processing speed and real-time performance. To achieve highly realistic simulation of the external visual scene, a method that can rapidly generate the human eye model and perform image processing is required. However, existing algorithms often introduce time delays during processing, which not only affects the real-time performance of the simulation but also reduces the contrast fidelity between multiple visual images. To address this issue, the present invention proposes a ground-based pilot vision training device that takes environmental and physiological factors into account. Summary of the Invention

[0003] The purpose of the present invention is to provide a ground vision training device for pilots that takes environmental and physiological factors into consideration, in order to solve the problems raised in the above-mentioned background technology.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A ground-based pilot vision training device that takes environmental and physiological factors into account includes a cockpit module, a camera, a human eye visual display system, an eye tracker, and a computer system. The cockpit module is configured to simulate a real cockpit layout. The camera is configured to capture external visual images and transmit them to the computer system. The human eye visual display system is composed of three display panels and is configured to simulate the real scene observed by the human eye. The eye tracker is configured to detect the pilot's gaze point in real time and transmit the gaze point coordinates to the computer system in real time. The computer system is configured to rapidly simulate human eye visual characteristics and transmit the output image to the human eye visual display system in real time.

[0006] The specific steps of the computer system for rapid simulation of human visual characteristics are as follows:

[0007] Step 1: Establish a scene model for the pilot when driving;

[0008] The light source and brightness detector were established using Speos software. A two-dimensional distributed brightness matrix of the scene was obtained through large-scale ray tracing and inverse simulation. This two-dimensional distributed brightness matrix was used to obtain the ambient brightness information of the external scene perceived by the human eye. Ambient brightness affects optical aberrations by influencing pupil diameter.

[0009] Step 2: Establish an optical model of the human eye;

[0010] Zemax software was used to establish the human eye optical model and obtain the modulation transfer function at different field angles and different defocus object distances;

[0011] Step 3: Use GPU acceleration in the frequency domain to simulate the observation effect of the external scene after passing through the human eye;

[0012] In the frequency domain, the product of the external scene image and the modulation transfer function is used to simulate the observation effect of the external scene after passing through the human eye optical model, and the GPU is used to accelerate the image processing;

[0013] Step 4: Use the gaze point position information from the eye tracker to simulate the peripheral vision blur and depth of field characteristics of the human eye in real time;

[0014] An eye tracker is used to determine the coordinates of the pilot's gaze point, and the peripheral vision blur characteristics are simulated by sampling the field of view angles. The depth of field characteristics are simulated by sampling different depth information inside and outside the cockpit. After superimposing the depth of field and peripheral vision blur characteristics, the images between adjacent field of view angles are fused to obtain a simulated image of the external scene after passing through the human eye.

[0015] Furthermore, the specific process of step 1 is as follows:

[0016] The light source and brightness detector are established using Speos software, where the light source is the superposition of the sunlight, the ambient light outside the cabin, and the light inside the cabin; the sun's position is determined by the azimuth angle. and altitude angle Sure;

[0017] In a certain position of the sun The simulation is carried out under the following conditions to obtain the two-dimensional distribution brightness matrix including the brightness of the sunlight source, the brightness of the external ambient light source and the brightness of the cabin lighting received by the pilot's eyes. , use the following formula to convert the two-dimensional distribution into the average value L and average the L As the ambient brightness of the external view;

[0018] ;

[0019] in, m and nare the number of rows and columns of the two-dimensional distribution brightness matrix respectively; and Respectively represent the abscissa and ordinate of the two-dimensional distribution brightness matrix;

[0020] The ambient brightness of the external scene will affect pupil size and thus cause changes in optical aberrations. The change pattern is as follows:

[0021] ;

[0022] in, is the ambient brightness of the external scene; is the pupil diameter.

[0023] Furthermore, the specific process of step 2 is as follows:

[0024] Select the Arizona model eye and enter the corresponding optical parameters; establish the human eye constraints at close distances:

[0025] ;

[0026] The adjustment ability of the human eye is 10 D ,When the object distance is less than 0.1m, the deformation ability of the lens reaches its limit;

[0027] Focusing is achieved by adjusting the lens parameters by changing the object distance according to the following formula:

[0028] ;

[0029] in, is the radius of curvature of the anterior surface of the lens; is the radius of curvature of the posterior surface of the lens; is the aspheric coefficient of the anterior lens surface; is the aspheric coefficient of the posterior lens surface; is the refractive index of the lens; The distance from the posterior surface of the cornea to the anterior surface of the lens; is the thickness of the lens; is the object distance; is the object distance of the human eye optical model;

[0030] The wavelength, pupil diameter, and maximum half-field angle were set in Zemax software; the wavelengths included 700 nm red light, 546.1 nm green light, and 435.8 nm blue light, and the maximum half-field angle was 30°.

[0031] Through the FFT MTF function in Zemax software, the normalized modulation transfer function of the incoherent imaging system is obtained: ,in , is the spatial frequency; the modulation transfer functions of wavelengths of 700nm, 546.1nm and 435.8nm are defined as 、 and ; Define the modulation transfer function of field angle and defocus object distance as ,in is the field of view, is the out-of-focus object distance.

[0032] Furthermore, the specific process of step 3 is as follows:

[0033] Step 31: Get the exterior view image , and define its The three channels are 、 、 ;

[0034] Step 32: Place the exterior view image The three channels are converted to the frequency domain through discrete Fourier transform, and we get 、 、 ;

[0035] ;

[0036] ;

[0037] ;

[0038] in, for Frequency domain image of the channel; for Frequency domain image of the channel; for Frequency domain image of the channel; is the width of the image; is the height of the image; x and y are the coordinate values ​​of the pixels in the spatial domain image; and The coordinate values ​​of the pixel points of the frequency domain image; is an imaginary unit;

[0039] Step 33: Compare the converted frequency domain image with the modulation transfer function at three wavelengths 、 、 Perform the product in the frequency domain:

[0040] ;

[0041] ;

[0042] ;

[0043] in, for Frequency domain image of channel output; for Frequency domain image of channel output; for Frequency domain image of channel output;

[0044] Step 34: Frequency domain image after multiplication Perform a two-dimensional inverse discrete Fourier transform and convert back to the spatial domain:

[0045] ;

[0046] ;

[0047] ;

[0048] in, for Spatial domain image of channel output; for Spatial domain image of channel output; for Spatial domain image of channel output;

[0049] Will The matrices of the three channels are combined to obtain , The output image with degraded image quality after passing through the human eye optical model;

[0050] Similarly, use Represents the output image at a certain field of view angle and a certain defocus object distance;

[0051] The GPU is used to perform the calculations in steps 31 to 34 .

[0052] Furthermore, the specific process of step 4 is as follows:

[0053] Assuming that the human eye is looking horizontally at the image plane, the pixel width of the image is , the real-time coordinates of the gaze point obtained by the eye tracker are , then the pixel to be processed To the gaze point distance By Euclidean distance get:

[0054] ;

[0055] Assume that the distance from the eye to the gaze point is ,but:

[0056] ;

[0057] in, is the maximum half field of view angle, that is ;

[0058] set up is the field of view of the pixel point, then and The relationship is:

[0059] ;

[0060] ;

[0061] According to the positions of different devices inside and outside the aircraft cabin, the depth information of the external view picture is set to , the out-of-focus object distance at each pixel for:

[0062] ;

[0063] The depth information is sampled non-uniformly, and different defocus distances are used for pixels at different depths to simulate the depth of field characteristics:

[0064] ;

[0065] Perform non-uniform sampling of the field of view angle to simulate the blurring characteristics of the peripheral vision:

[0066] ;

[0067] in, is the sampling matrix of the field of view;

[0068] The output image after superimposing the depth of field and afterglow blur characteristics is:

[0069] ;

[0070] Perform image fusion on images between adjacent field of view angles, that is, fuse the pixels within the current field of view angle with the pixels within the next field of view angle according to a certain weight:

[0071] ;

[0072] in, The pixels after image fusion reflect the real scene observed by the pilot with the characteristics of depth of field and afterimage blur. is the pixel under the current field of view; is the pixel in the next field of view angle; is the weight, expressed as:

[0073] ;

[0074] in, are the left and right boundaries where different field of view angles connect. is a pixel in the image fusion area.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] 1. By using Speos software to establish a light source and brightness detector, this invention accurately simulates the dynamic changes in sunlight angle and cockpit ambient light brightness during flight, and adjusts the human eye optical model accordingly to ensure that the simulated image can accurately reflect the impact of these external environmental factors on human eye perception.

[0077] 2. This invention adopts a dynamically adjusted human eye optical model and uses Zemax software to establish the modulation transfer function at different field of view angles and different defocus object distances. The model is adjusted in real time according to the pilot's gaze distance, thereby ensuring that the simulated image can accurately reflect the real perception of the human eye at different gaze distances.

[0078] 3. The present invention uses GPU acceleration technology to efficiently simulate the observation effect of external scenes after passing through the human eye in the frequency domain, greatly reducing the time delay of algorithm calculation, realizing high-speed human eye model generation and image processing, and ensuring the real-time and smoothness of the simulation.

[0079] 4. By comprehensively considering external environmental factors, the physiological characteristics of the human eye, and the pilot's gaze behavior, the present invention generates simulated images that are not only more realistic, but also improves the contrast authenticity between multiple visual images, providing pilots with a more intuitive and realistic flight visual experience.

[0080] In summary, the present invention can not only improve the effect of flight training or the evaluation accuracy of new aircraft cockpits, but also provide pilots with a more realistic and intuitive flight visual experience during ground training. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 Schematic diagram of the device of the present invention.

[0082] Figure 2 It is a structural diagram of the device of the present invention.

[0083] Figure 3 Flowchart for rapid simulation of human visual characteristics for computer systems.

[0084] Figure 4 This is the curve of the ambient brightness of the external scene changing with the solar altitude angle.

[0085] Figure 5 This is an exterior view picture.

[0086] Figure 6 It is a simulated image of the external scene after passing through the human eye when the gaze point is at the center.

[0087] In the figure: cockpit module 1, camera 2, human eye visual display system 3, eye tracker 4, computer system 5, display 6. DETAILED DESCRIPTION

[0088] In order to have a clearer understanding of the technical features, objectives and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below, but it should not be understood as limiting the scope of implementation of the present invention.

[0089] One embodiment of the present invention provides a ground vision training device for pilots that takes environmental and physiological factors into consideration. Figure 1 This is a schematic diagram of the ground vision training device. Figure 2 This is a structural diagram of a ground vision training device. The device includes a cockpit module 1, a camera 2, a human visual display system 3, an eye tracker 4, and a computer system 5. The cockpit module 1 simulates a real cockpit layout and includes control devices such as a joystick, thrust lever, and various switches. The camera 2 captures images of the exterior scene and transmits them to the computer system 5. The human visual display system 3, composed of three high-resolution displays 6, simulates the real scene observed by the human eye. The eye tracker 4 detects the pilot's gaze point in real time and transmits the gaze point coordinates to the computer system 5. The computer system 5 rapidly simulates human visual characteristics and transmits the output image to the human visual display system 3 in real time.

[0090] The specific steps of the computer system 5 for rapid simulation of human visual characteristics are as follows ( Figure 3 shown):

[0091] Step 1: Establish a scene model for the pilot when driving;

[0092] The light source and brightness detector were constructed using Speos software. The light source was a combination of sunlight, ambient light outside the cabin, and light inside the cabin. A two-dimensional distributed brightness matrix of the scene was obtained through large-scale ray tracing and inverse simulation. This two-dimensional distributed brightness matrix was used to obtain ambient brightness information of the external scene as perceived by the human eye. Ambient brightness affects optical aberrations by influencing pupil diameter.

[0093] Step 2: Establish an optical model of the human eye;

[0094] Zemax software was used to establish the human eye optical model and obtain the modulation transfer function at different field angles and different defocus object distances;

[0095] Step 3: Use GPU acceleration in the frequency domain to simulate the observation effect of the external scene after passing through the human eye;

[0096] In the frequency domain, the product of the external scene image and the modulation transfer function is used to simulate the observation effect of the external scene after passing through the human eye optical model, and the GPU is used to accelerate the image processing;

[0097] Step 4: Using the gaze point position information from the eye tracker 4, the blur characteristics and depth of field characteristics of the human eye are simulated in real time;

[0098] The pilot's gaze point coordinates are determined using an eye tracker 4. The peripheral vision blur characteristic is simulated by sampling the field of view angles, and the depth of field characteristic is simulated by sampling different depth information inside and outside the cockpit. After superimposing the depth of field and peripheral vision blur characteristics, the images between adjacent field of view angles are fused to obtain a simulated image of the external scene after passing through the human eye.

[0099] The following is a detailed description of the steps for rapid simulation of human visual characteristics:

[0100] Step 1: Establish a scene model for the pilot when driving;

[0101] Use Speos software to create a light source and brightness detector. The light source is a combination of sunlight, external ambient light, and internal light sources. The sunlight source requires a specific orientation, while the external ambient light source requires its position relative to the ground plane and true north. The internal light source is provided by the cabin's lighting fixtures. Finally, reverse simulation is performed to obtain the scene's two-dimensional distributed brightness matrix.

[0102] The sun's position is determined by the azimuth angle and altitude angle The azimuth angle is set from the north direction of the external scene as the starting point (ie 0°), and increases to 360° by rotating clockwise. The altitude angle is the angle between the incident angle of the sun and the horizon. If the sun is above the horizon, that is, the daytime scene, then ; If the sun is below the horizon, that is, at night, then By setting the sun's position, we can model different sunlight sources in the morning, midday, and evening, and create daytime scenes and nighttime low-light scenes.

[0103] In a certain position of the sun The simulation is carried out under the following conditions to obtain the two-dimensional distribution brightness matrix including the brightness of the sunlight source, the brightness of the external ambient light source and the brightness of the cabin lighting received by the pilot's eyes. To improve simulation efficiency, the two-dimensional distribution is converted into an average value using formula 1. and average the As the ambient brightness of the external view;

[0104] ;

[0105] in, m and n are the number of rows and columns of the two-dimensional distribution brightness matrix respectively; and They represent the horizontal and vertical coordinates of the two-dimensional distribution brightness matrix respectively. Figure 4 is the azimuth When the height angle is 90°, The simulated curve after interpolation in the range of (-180°, 180°).

[0106] The ambient brightness of the external scene will affect pupil size and thus cause changes in optical aberrations. The change pattern is as follows:

[0107] ;

[0108] in, is the ambient brightness of the external scene, in units of ; is the pupil diameter, in mm. The darker the environment, the larger the pupil, the greater the higher-order aberrations of the human eye, and the greater the luminous flux on the retina; the brighter the environment, the smaller the pupil, the aberrations around the cornea and lens will also decrease, thereby improving vision, but the luminous flux on the retina will decrease. According to formula 2 and Figure 4 We can then find out the effect of the sun's position on pupil size.

[0109] Step 2: Establish an optical model of the human eye;

[0110] The present invention establishes an equivalent optical model through a series of optical parameters such as the equivalent curvature radius of the human eye lens, aspheric coefficient, refractive surface spacing, refractive index, Abbe number, retinal curvature radius, etc., laying the foundation for the subsequent solution of the modulation transfer function of the human eye and the simulation of the image. There are many human eye models, including but not limited to the Arizona model eye, Gullstrand model eye, Legrand model eye, Navarro model eye, Liou model eye, Walker model eye, etc. The rapid simulation of human eye visual characteristics performed by the present invention is applicable to multiple human eye models. The parameters of the Arizona model eye are introduced as an example below. Table 1 shows the parameters of the Arizona model eye:

[0111] Table 1 Parameters of the Arizona model eye

[0112]

[0113] In order to better conform to the real observation effect of the human eye, the present invention establishes a human eye constraint condition at close distance. The existing human eye model does not include this correction term, see formula 3, that is, the adjustment ability of the human eye is 10 D When the object distance is less than 0.1m, the deformation capacity of the lens reaches its limit. According to Equation 4, focusing can be achieved by adjusting the lens parameters by changing the object distance.

[0114] ;

[0115] ;

[0116] in, is the radius of curvature of the anterior surface of the lens; is the radius of curvature of the posterior surface of the lens; is the aspheric coefficient of the anterior lens surface; is the aspheric coefficient of the posterior lens surface; is the refractive index of the lens; The distance from the posterior surface of the cornea to the anterior surface of the lens; is the thickness of the lens; is the object distance; the unit is m; The object distance for the optical model of the human eye.

[0117] Create a human eye optical model in Zemax software. Set the wavelengths to 700nm (red light), 546.1nm (green light), and 435.8nm (blue light). Adjust the simulated wavelength range to suit other scenarios. Set the pupil diameter as in Step 1 and adjust it in real time with the sun's position. Set the maximum half-field of view to 30°.

[0118] Through the FFT MTF function in Zemax software, the normalized modulation transfer function of the incoherent imaging system can be obtained: ,in , is the spatial frequency, in cycles / mm. The modulation transfer function reflects the imaging capability of the human eye optical model. The modulation transfer function with a wavelength of 700nm is defined as , the modulation transfer function with a wavelength of 546.1nm is defined as , the modulation transfer function with a wavelength of 435.8nm is defined as .

[0119] The modulation transfer function of the human eye optical model is related to the field of view angle and the defocus distance, and is defined as , field of view It reflects that the image quality of the retina gradually declines from the axis to the off-axis, that is, the peripheral vision of the human eye is blurred; the distance from the out-of-focus object is It reflects that if an object is not in the focal plane, a blurred spot of light will be formed on the retina, which is the depth of field characteristic of the human eye.

[0120] Step 3: Use GPU acceleration in the frequency domain to simulate the observation effect of the external scene after passing through the human eye;

[0121] This method uses the modulation transfer function (MTF) of the human eye's optical model to simulate the imaging effect of an external scene image after it passes through the human eye's optical model. Zemax's image processing uses point spread functions (PSFs) for spatial convolution, which is slow. However, this method uses MTF multiplication in the frequency domain and utilizes GPU acceleration, achieving near-real-time processing speeds.

[0122] The specific method is as follows:

[0123] (1) Obtaining external view images , and define its The three channels are 、 、 .

[0124] (2) The exterior view picture The three channels are converted to the frequency domain through discrete Fourier transform, and we get 、 、 ;

[0125] ;

[0126] ;

[0127] ;

[0128] in, for Frequency domain image of the channel; for Frequency domain image of the channel; for Frequency domain image of the channel; is the width of the image; is the height of the image; x and y are the coordinate values ​​of the pixels in the spatial domain image; and The coordinate values ​​of the pixel points of the frequency domain image; is an imaginary unit;

[0129] (3) Compare the converted frequency domain image with the modulation transfer function at three wavelengths 、 、 Perform the product in the frequency domain:

[0130] ;

[0131] ;

[0132] ;

[0133] in, for Frequency domain image of channel output; for Frequency domain image of channel output; for Frequency domain image of channel output;

[0134] (4) Then multiply the frequency domain image Perform a two-dimensional inverse discrete Fourier transform and convert back to the spatial domain:

[0135] ;

[0136] ;

[0137] ;

[0138] in, for Spatial domain image of channel output; for Spatial domain image of channel output; for Spatial domain image of channel output;

[0139] Will The matrices of the three channels are combined to obtain , The output image with degraded image quality after passing through the human eye optical model;

[0140] Similarly, in the same way, the modulation transfer function under the wavelength in the above process is replaced by , you can get The present invention uses Represents the output image at a certain field of view angle and a certain defocus object distance.

[0141] The above process uses the GPU for calculations. The GPU has a large number of computing cores and can handle multiple threads simultaneously, thereby increasing processing speed. The external view image is uploaded to the GPU, where operations such as Fourier transforms and frequency domain multiplications are performed. During matrix multiplication, the GPU assigns each matrix element to a different core, performing parallel operations to increase processing speed. The Fourier transform involves a large number of complex multiplication and addition operations, which the GPU can accelerate at the hardware level.

[0142] Step 4: Using the gaze point position information from the eye tracker 4, the blur characteristics and depth of field characteristics of the human eye are simulated in real time;

[0143] The present invention uses an eye tracker 4 to monitor and analyze the trajectory of eye movement and obtain the position coordinates of the pilot's gaze. Assuming that the human eye is gazing horizontally at the image plane, the pixel width of the image is M , the real-time coordinates of the gaze point obtained by the eye tracker 4 are, , then the pixel to be processed To the gaze point distance By Euclidean distance get:

[0144] ;

[0145] Assume that the distance from the eye to the gaze point is ,but:

[0146] ;

[0147] in, is the maximum half field of view angle, that is ;

[0148] set up is the field of view of the pixel point, then and The relationship is:

[0149] ;

[0150] ;

[0151] The above formula can be used to associate the pixel position information of the image with the field of view angle of the human eye optical model.

[0152] The cockpit of an aircraft has a small space, with compact arrangement of the instrument panel, control console, front window and side windows. The position information of different instrument panels and control consoles is also different, which creates a strong sense of space. If the pilot observes the central instrument panel, the control console closer to the pilot will be blurred. At the same time, the pilot also needs to switch between the vast external scene and the internal space. If the pilot observes the external scene through the front windshield, the instrument panel will be blurred due to the sense of depth; if the pilot observes the instrument panel inside the cockpit, the external scene will also be blurred. According to the position of different equipment inside and outside the aircraft cabin, the depth information of the external view picture is set to ,like Then the pixel is located behind the gaze point; if , then the pixel is in front of the gaze point; if , then the pixel is coplanar with the gaze point. The out-of-focus object distance at each pixel for:

[0153] ;

[0154] The depth information is sampled non-uniformly, and different defocus distances are used for pixels at different depths to simulate the depth of field characteristics:

[0155] ;

[0156] The field of view is sampled non-uniformly. Based on the characteristics of cones and rods in the human eye, the sampling is denser at the gaze point and sparser at the edges, thus simulating the blurred characteristics of peripheral vision:

[0157] ;

[0158] in, is the sampling matrix of the field of view;

[0159] The output image after superimposing the depth of field and afterglow blur characteristics is:

[0160] ;

[0161] Since discrete sampling of the field of view angle causes image blockiness, the present invention performs image fusion on images between adjacent field of view angles, that is, the pixels within the current field of view angle are fused with the pixels within the next field of view angle according to a certain weight:

[0162] ;

[0163] in, is the pixel after image fusion, It reflects the real scene with depth of field and blurred characteristics of the afterglow that reflects the pilot's observation; is the pixel under the current field of view; is the pixel in the next field of view angle; is the weight, expressed as:

[0164] ;

[0165] in, are the left and right boundaries where different field of view angles connect. is a pixel in the image fusion area.

[0166] Figure 5 This is an exterior view picture. Figure 6 This is a simulated image of the external scene after the human eye when the gaze point is at the center. It can be observed that the image is more blurred at the edges than at the center. The specific simulation process is as follows: First, the pupil diameter is determined based on the sun's position. Then, based on the human eye optical model, a series of optical modulation transfer functions are derived for different fields of view. Then, the external scene image is multiplied by the modulation transfer function in the frequency domain to obtain a series of blurred images. Finally, the eye tracker 4 is used to determine the gaze point is at the center. After superimposing the depth of field and the blur characteristics of the peripheral vision, the images between adjacent field angles are fused to obtain a simulated image of the external scene after the human eye.

[0167] The above are only preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, several variations and improvements can be made without departing from the concept of the present invention. These should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A ground vision training device for pilots taking into account environmental and physiological factors, characterized in that: The system comprises a cockpit module, a camera, a human eye visual display system, an eye tracker, and a computer system; the cockpit module is used to simulate a real cockpit layout; the camera is used to capture external scene images and send them to the computer system; the human eye visual display system is composed of three display panels, which are used to simulate the real scene observed by the human eye; the eye tracker is used to detect the pilot's gaze point in real time and send the gaze point position coordinates to the computer system in real time; the computer system is used to quickly simulate the human eye visual characteristics and send the output image to the human eye visual display system in real time; The specific steps of the computer system for rapid simulation of human visual characteristics are as follows: Step 1: Establish a scene model for the pilot when driving; The light source and brightness detector were established using Speos software. A two-dimensional distributed brightness matrix of the scene was obtained through large-scale ray tracing and inverse simulation. This two-dimensional distributed brightness matrix was used to obtain the ambient brightness information of the external scene perceived by the human eye. Ambient brightness affects optical aberrations by influencing pupil diameter. Step 2: Establish an optical model of the human eye; Zemax software was used to establish the human eye optical model and obtain the modulation transfer function at different field angles and different defocus object distances; Step 3: Use GPU acceleration in the frequency domain to simulate the observation effect of the external scene after passing through the human eye; In the frequency domain, the product of the external scene image and the modulation transfer function is used to simulate the observation effect of the external scene after passing through the human eye optical model, and the GPU is used to accelerate the image processing; Step 4: Use the gaze point position information from the eye tracker to simulate the peripheral vision blur and depth of field characteristics of the human eye in real time; An eye tracker is used to determine the coordinates of the pilot's gaze point, and the peripheral vision blur characteristics are simulated by sampling the field of view angles. The depth of field characteristics are simulated by sampling different depth information inside and outside the cockpit. After superimposing the depth of field and peripheral vision blur characteristics, the images between adjacent field of view angles are fused to obtain a simulated image of the external scene after passing through the human eye.

2. The pilot ground vision training device considering environmental and physiological factors according to claim 1, characterized in that: The specific process of step 1 is as follows: The light source and brightness detector are established using Speos software, where the light source is the superposition of the sunlight, the ambient light outside the cabin, and the light inside the cabin; the sun's position is determined by the azimuth angle. and altitude angle Sure; In a certain position of the sun The simulation is carried out under the following conditions to obtain the two-dimensional distribution brightness matrix including the brightness of the sunlight source, the brightness of the external ambient light source and the brightness of the cabin lighting received by the pilot's eyes. , use the following formula to convert the two-dimensional distribution into the average value L and average the L As the ambient brightness of the external view; ； in, m and n are the number of rows and columns of the two-dimensional distribution brightness matrix respectively; and Respectively represent the abscissa and ordinate of the two-dimensional distribution brightness matrix; The ambient brightness of the external scene will affect pupil size and thus cause changes in optical aberrations. The change pattern is as follows: ； in, is the ambient brightness of the external scene; is the pupil diameter.

3. The pilot ground vision training device considering environmental and physiological factors according to claim 2, characterized in that: The specific process of step 2 is as follows: Select the Arizona model eye and enter the corresponding optical parameters; establish the human eye constraints at close distances: ； The adjustment ability of the human eye is 10 D ,When the object distance is less than 0.1m, the deformation ability of the lens reaches its limit; Focusing is achieved by adjusting the lens parameters by changing the object distance according to the following formula: ； in, is the radius of curvature of the anterior surface of the lens; is the radius of curvature of the posterior surface of the lens; is the aspheric coefficient of the anterior lens surface; is the aspheric coefficient of the posterior lens surface; is the refractive index of the lens; The distance from the posterior surface of the cornea to the anterior surface of the lens; is the thickness of the lens; is the object distance; is the object distance of the human eye optical model; The wavelength, pupil diameter, and maximum half-field angle were set in Zemax software; the wavelengths included 700 nm red light, 546.1 nm green light, and 435.8 nm blue light, and the maximum half-field angle was 30°. Through the FFT MTF function in Zemax software, the normalized modulation transfer function of the incoherent imaging system is obtained: ,in , is the spatial frequency; the modulation transfer functions of wavelengths of 700nm, 546.1nm and 435.8nm are defined as 、 and ; Define the modulation transfer function of field angle and defocus object distance as ,in is the field of view, is the out-of-focus object distance.

4. The pilot ground vision training device considering environmental and physiological factors according to claim 3, characterized in that: The specific process of step 3 is as follows: Step 31: Get the exterior view image , and define its The three channels are 、 、 ; Step 32: Place the exterior view image The three channels are converted to the frequency domain through discrete Fourier transform, and we get 、 、 ; ； ； ； in, for Frequency domain image of the channel; for Frequency domain image of the channel; for Frequency domain image of the channel; is the width of the image; is the height of the image; x and y are the coordinate values ​​of the pixels in the spatial domain image; and The coordinate values ​​of the pixel points of the frequency domain image; is an imaginary unit; Step 33: Compare the converted frequency domain image with the modulation transfer function at three wavelengths 、 、 Perform the product in the frequency domain: ； ； ； in, for Frequency domain image of channel output; for Frequency domain image of channel output; for Frequency domain image of channel output; Step 34: Frequency domain image after multiplication Perform a two-dimensional inverse discrete Fourier transform and convert back to the spatial domain: ； ； ； in, for Spatial domain image of channel output; for Spatial domain image of channel output; for Spatial domain image of channel output; Will The matrices of the three channels are combined to obtain , The output image with degraded image quality after passing through the human eye optical model; Similarly, use Represents the output image at a certain field of view angle and a certain defocus object distance; The GPU is used to perform the calculations in steps 31 to 34 .

5. The pilot ground vision training device considering environmental and physiological factors according to claim 4, characterized in that: The specific process of step 4 is as follows: Assuming that the human eye is looking horizontally at the image plane, the pixel width of the image is , the real-time coordinates of the gaze point obtained by the eye tracker are , then the pixel to be processed To the gaze point distance By Euclidean distance get: ； Assume that the distance from the eye to the gaze point is ,but: ； in, is the maximum half field of view angle, that is ; set up is the field of view of the pixel point, then and The relationship is: ； ； According to the positions of different devices inside and outside the aircraft cabin, the depth information of the external view picture is set to , the out-of-focus object distance at each pixel for: ； The depth information is sampled non-uniformly, and different defocus distances are used for pixels at different depths to simulate the depth of field characteristics: ； Perform non-uniform sampling of the field of view angle to simulate the blurring characteristics of the peripheral vision: ； in, is the sampling matrix of the field of view; The output image after superimposing the depth of field and afterglow blur characteristics is: ； Perform image fusion on images between adjacent field of view angles, that is, fuse the pixels within the current field of view angle with the pixels within the next field of view angle according to a certain weight: ； in, The pixels after image fusion reflect the real scene observed by the pilot with the characteristics of depth of field and afterimage blur. is the pixel under the current field of view; is the pixel in the next field of view angle; is the weight, expressed as: ； in, are the left and right boundaries where different field of view angles connect. is a pixel in the image fusion area.

Images