A flight simulation mixed reality display system

Abstract

A flight simulation mixed reality display system comprises a physical simulation cockpit, a head position measurement system, a virtual scene generation system and a see-through VR glasses; the head position measurement system is used to collect the head posture position coordinates of a trainee, and comprises a six-degree-of-freedom inertial module and a three-degree-of-freedom infrared measurement module. The virtual scene generation system corresponds the cockpit in the virtual scene established by the virtual scene generation system to the three-dimensional digital model of the cockpit, and corresponds the head position of the trainee in the virtual scene established by the virtual scene generation system to the head posture position coordinates; the see-through VR glasses reflects the virtual scene on the virtual scene display screen through a concave semi-transparent half-mirror. The real scene of the illuminated physical simulation cockpit is transmitted through the concave semi-transparent half-mirror, so as to realize the superposition of the virtual scene and the real scene. The above structure can effectively solve the problem of the flight simulation mixed reality display system in the prior art, poor training effect and the inability to truly simulate the flight training effect.

Description

Technical Field

[0001] The present invention relates to the technical field of flight simulators, and in particular to a flight simulation mixed reality display system. Background Art

[0002] A flight simulator is a machine used to simulate aircraft flight. Using mixed reality for flight simulation training, flight simulators seamlessly integrate the trainee's view of the actual cockpit with the virtual scene outside the cockpit. The physical cockpit is placed on the ground. When the trainee sits inside and uses a mixed reality device to view the cockpit, they can see the actual cockpit equipment and themselves. However, they cannot see the actual environment outside the cockpit, such as the interior fixtures, walls, roof, and ground. By presenting a virtual scene to the trainee, the trainee can create the illusion of being in the cockpit, overlooking the ground, or even on the runway.

[0003] However, the flight simulators in the prior art all achieve the above functions through VR glasses. In the prior art, the real scene of the real world is captured by the camera on the front of the VR glasses and converted into a continuous video image signal a. At the same time, the scene of the virtual world is generated by a computer, and the video image signal b is output through the display output port. Afterwards, the above-mentioned video image signal a and video image signal b are subjected to a dedicated algorithm, and the video image signal a is cropped and replaced with the corresponding part in the virtual world image b, and the final virtual-reality mixed scene image c is formed by fusion. The virtual-reality mixed scene image c is displayed to the trainees through VR glasses. The above-mentioned mixed reality system can achieve good results in training, education, entertainment and other industries.

[0004] However, when this device is applied to pilot training, the numerous small characters and line symbols present in the cockpit present insufficient resolution after being processed through the aforementioned video image signal processing method, resulting in pilots being unable to accurately read cockpit instruments or signage. This problem arises from the fact that, with current technology, cameras are used to observe the real world instead of the human eye, and are limited by factors such as the camera's dynamic focus, resolution, adaptability to ambient brightness, and video transmission bandwidth, making it impossible to achieve the same level of accuracy as the human eye. Consequently, after being processed through existing flight simulators, the instruments and signage inside the physical cockpit appear blurry to pilots, and in severe cases, even simulated training is affected. Furthermore, regardless of the algorithm used, fusing the two video image signals a and b requires significant computing power from the device, making it difficult to meet the requirements of flight training even with expensive and heavy dedicated hardware.

[0005] To clearly see fine characters and line symbols in the cockpit, the prior art patent document JP2000029589A discloses an information display device. Specifically, it discloses that a transflective mirror, which reflects the image on the display and presents it to the viewer, is positioned in a predetermined position relative to the display. Furthermore, a light source capable of adjusting the amount of light directed at an observation object located on the opposite side of the transflective mirror from the viewer is positioned on the opposite side of the transflective mirror. The viewer can then see, in the same space, the image information of the image reflected on the display, which is inverted left and right by the transflective mirror, and the viewer's hand or other physical object, illuminated by the light source and transmitted through the transflective mirror, superimposed on each other. This demonstrates that the prior art described in this patent document effectively enables the operator to use bright colors to make real objects appear to have the same or higher brightness as virtual objects. At the same time, using dark colors allows the viewer to more consistently observe the desired superposition of real objects and images.

[0006] However, these technologies, when used in pilot training, require accurate head and eye position information to be provided to the computer imaging system, enabling timely updates to the virtual scene generation system. Furthermore, they also require a wider range of head movement for trainees, enhancing training realism and ensuring effectiveness.

[0007] Therefore, those skilled in the art need a new flight simulation mixed reality display system to solve the problem that existing mixed reality flight simulators have poor training effects and cannot truly simulate flight training effects. Summary of the Invention

[0008] The present invention aims to provide a flight simulation mixed reality display system to solve the problem that the flight simulation mixed reality display system in the prior art has poor training effect and cannot truly simulate the flight training effect. To this end, the present invention provides a flight simulation mixed reality display system, comprising:

[0009] A physical simulated cockpit is set in a dark environment and is visible to the trainees; a three-dimensional digital model of the cockpit is generated based on the frame and outer contour dimension information of the physical simulated cockpit;

[0010] A head position measurement system for collecting the trainee's head posture position coordinates; the head position measurement system includes: a six-degree-of-freedom inertial module and a three-degree-of-freedom infrared measurement module for correcting the accumulated error of the six-degree-of-freedom inertial module; the six-degree-of-freedom inertial module is disposed on the helmet of the see-through VR glasses; the three-degree-of-freedom infrared measurement module is disposed directly opposite the trainee's seat;

[0011] a virtual scene generation system that acquires the three-dimensional digital model of the cockpit and the head posture position coordinates and creates a virtual scene; aligns the cockpit in the virtual scene created by the virtual scene generation system with the three-dimensional digital model of the cockpit; and aligns the trainee's head position in the virtual scene created by the virtual scene generation system with the head posture position coordinates;

[0012] The see-through VR glasses include: a display screen and a concave semi-transparent and semi-reflective mirror; the display screen is used to display the virtual scene established by the virtual scene generation system; the display screen and the concave semi-transparent and semi-reflective mirror are arranged opposite to each other, and the content of the display screen is reflected onto the concave semi-transparent and semi-reflective mirror so that the trainee can see the virtual scene; and the concave semi-transparent and semi-reflective mirror is arranged opposite to the physical simulation cockpit, and the real scene of the illuminated physical simulation cockpit is transmitted through the concave semi-transparent and semi-reflective mirror so that the trainee can see the real scene; the virtual scene and the real scene overlap.

[0013] Optionally, the flight simulation mixed reality display system further includes: a light-shielding outer shell and a lighting mechanism, wherein the light-shielding outer shell is made of a non-reflective material and is used to block external light, and the physical simulation cockpit is disposed within the light-shielding outer shell;

[0014] The lighting mechanism is arranged opposite to the physical simulated cockpit and is used to illuminate the physical simulated cockpit; and the lighting mechanism is arranged at a position that is invisible to the trainees.

[0015] Optionally, the three-degree-of-freedom infrared measurement module is arranged on the inner cavity wall of the light-shielding shell and is facing the trainee's seat. When the six-degree-of-freedom inertial module and the three-degree-of-freedom infrared measurement module are used in combination, when the trainee's head deviates from the infrared measurement range of the three-degree-of-freedom infrared measurement module, the integration algorithm of the six-degree-of-freedom inertial module is used to obtain the trainee's head posture information.

[0016] Optionally, the six-degree-of-freedom inertial module utilizes the spatial attitude maintenance characteristics of the gyroscope to measure in real time the three-axis acceleration of the helmet position of the see-through VR glasses and the change in the attitude Euler angle relative to the reference position, and calculates the speed of the three axes based on the integration of the three-axis acceleration;

[0017] The three-axis acceleration is the ax axis, the ay axis and the az axis; the attitude Euler angle includes: pitch angle, tilt angle and heading angle;

[0018] Wherein, each time the six-degree-of-freedom inertial module is powered on, the module is ensured to be in a stationary state to facilitate the initial alignment of the internal computing module.

[0019] Optionally, the three-degree-of-freedom infrared measurement module includes: a camera and multi-source positioning software;

[0020] The helmet of the operator to be located is provided with three strong reflective points. The light emitted by the lighting mechanism hits the strong reflective points and is reflected, entering the camera. The camera is provided with an infrared filter to reduce interference from external light.

[0021] The multi-source positioning software runs and reads the video signal of the camera. The multi-source positioning software calculates the operator's head posture and position information based on the changes in the relative relationship of the image formed by the reflective points in the video screen; the head posture and position information is used for the three-degree-of-freedom correction of the three-degree-of-freedom infrared measurement module.

[0022] Optionally, the multi-source positioning software is opentrack software;

[0023] When powered on, the six-degree-of-freedom inertial module measures the trainee's head rotation posture, linear acceleration and speed of spatial movement in real time, and sends them to the virtual scene generation system to measure the trainee's head posture angle in real time, and obtains the trainee's head posture information through an integration algorithm to update the virtual scene; the integration algorithm is: original head position + speed × time interval = new head position.

[0024] The acquisition cycle of the above-mentioned three-degree-of-freedom infrared measurement module is 30 Hz, and the position measurement error does not increase over time; the acquisition cycle of the six-degree-of-freedom inertial module is 100 Hz, and the position measurement error accumulates and amplifies over time. The present invention uses a comprehensive algorithm to achieve a fusion of the advantages of the two: the head position of the person being measured = original position + speed × time interval, where the speed is output by the inertial module, the time interval is fixed at 0.01 seconds, and its error has been processed by the built-in inertial module. The value of the calculated original position is recorded as p0, which has two different sources: the result of the last integral calculation, recorded as p01, with an update frequency of 100 Hz; the measurement result of the three-degree-of-freedom infrared measurement module, recorded as p02, with an update frequency of 30 Hz. In order to eliminate cumulative errors and ensure an update rate of 100 Hz, the value of p0 is determined as follows: the time interval variable dt is set, initialized to 0, added by 1 each time calculation, and reset to 0 when it is greater than or equal to 2. When dt < 2, p0 = p01, otherwise p0 = p01 + (p02 - p01) / N, where N > = 1. The larger N is, the smoother the system is, and the closer N is to 1, the smaller the system error is. It is generally recommended to use 1 to 2.

[0025] Optionally, the lighting mechanism is a white light source, which is arranged above and behind the trainee's seat back.

[0026] Optionally, the cockpit three-dimensional digital model is a black model structure, and the position of the cockpit three-dimensional digital model in the virtual scene corresponds to the position of the physical simulation cockpit in real time, so as to fill the image information of the cockpit three-dimensional digital model in the virtual scene through the physical simulation cockpit.

[0027] The technical scheme of the present application has the following advantages:

[0028] 1. The flight simulation mixed reality display system provided by the present application comprises:

[0029] The physical simulation cockpit is arranged in a dark environment, and the physical simulation cockpit is in a visible state for the trainee; a cockpit three-dimensional digital model is generated based on the frame and the outer contour size information of the physical simulation cockpit;

[0030] A head position measurement system is used to collect the head posture position coordinates of the trainee; the head position measurement system comprises a six-degree-of-freedom inertial module and a three-degree-of-freedom infrared measurement module used to correct the cumulative error of the six-degree-of-freedom inertial module; the six-degree-of-freedom inertial module is arranged on the helmet of the see-through VR glasses; the three-degree-of-freedom infrared measurement module is arranged at a position opposite to the seat of the trainee;

[0031] A virtual scene generation system is used to obtain the cockpit three-dimensional digital model and the head posture position coordinates, and to establish a virtual scene; the cockpit in the virtual scene established by the virtual scene generation system corresponds to the cockpit three-dimensional digital model; and the head position of the trainee in the virtual scene established by the virtual scene generation system corresponds to the head posture position coordinates;

[0032] The see-through VR glasses comprise a display screen and a concave semi-transparent half mirror; the display screen is used to display the virtual scene established by the virtual scene generation system; the display screen and the concave semi-transparent half mirror are arranged opposite to each other, so that the content of the display screen is reflected on the concave semi-transparent half mirror, so that the trainee can see the virtual scene; and the concave semi-transparent half mirror is arranged opposite to the physical simulation cockpit, so that the real scene of the physical simulation cockpit illuminated can be seen through the concave semi-transparent half mirror, so that the trainee can see the real scene; the virtual scene and the real scene overlap.

[0033] In the present application, a physical simulation cabin is built in a dark environment, and a cabin three-dimensional digital model is generated according to the frame and outer contour size information of the cabin. Then, the head posture position coordinates of the trainee are collected in real time by a head position measurement system. A virtual scene is established by a virtual scene generation system, so that the cabin in the virtual scene corresponds to the cabin three-dimensional digital model, and the head position of the trainee in the virtual scene corresponds to the head posture position coordinates. The cabin three-dimensional digital model in the virtual scene is a black model structure, and the position of the cabin three-dimensional digital model in the virtual scene corresponds to the position of the physical simulation cabin in real time. The virtual scene on the display screen is reflected on the concave semi-transparent half-mirror, so that the trainee can see the virtual scene. At the same time, the concave semi-transparent half-mirror is arranged opposite to the physical simulation cabin, the real scene of the physical simulation cabin overlaps with the cabin three-dimensional digital model in the virtual scene, and the trainee can see the reflected virtual scene image and the actual scene image on the opposite side of the lens at the same time. Through the concave semi-transparent half-mirror, the preliminary mixing of the virtual scene and the real scene can be effectively realized, without the need to use the algorithm in the prior art to fuse the virtual scene image and the real scene image, so that the trainee can see the image after the fusion of the virtual scene and the real scene, and the problem that the flight simulation mixed reality display system in the prior art is limited by the processing capacity of the equipment and is difficult to accurately and low-costly process the small characters and line symbols in the aircraft cabin is effectively solved.

[0034] Moreover, the head posture position coordinates of the trainee are collected by the cooperation of the six-degree-of-freedom inertial module and the three-degree-of-freedom infrared measurement module. When the head of the trainee deviates from the infrared measurement range of the three-degree-of-freedom infrared measurement module, the integral algorithm of the six-degree-of-freedom inertial module is used to obtain the head posture information of the trainee. Through the above structure, the measurement accuracy of the head position measurement system is improved, and at the same time, a larger head movement range is provided for the trainee, so as to improve the training authenticity and ensure the training effect.

[0035] 2. The flight simulation mixed reality display system provided by the present application further comprises a light-shielding outer shell and an illumination mechanism, the light-shielding outer shell is made of non-reflective material and is used to shield external light, and the physical simulation cabin is arranged in the light-shielding outer shell.

[0036] The illumination mechanism is arranged opposite to the physical simulation cabin and is used to illuminate the physical simulation cabin, and the illumination mechanism is arranged at a position that cannot be seen by the trainee.

[0037] In the present invention, the light-shielding outer shell must be large enough to accommodate the trainee, the simulated cockpit, and the seats. It can be spherical or other shaped, with its interior covered in a special black coating or wrapping to minimize light reflection. This light-shielding outer shell effectively ensures that the pure black 3D digital cockpit model in the virtual scene is replaced by a visible simulated cockpit, allowing trainees to clearly see the characters and symbols on the simulated cockpit through the concave semi-transparent and semi-reflective mirror.

[0038] Furthermore, in the present invention, the brightness and illumination range of the aforementioned lighting mechanism are determined to ensure that the characters on the equipment and instrument displays within the cockpit can be clearly seen through the aforementioned see-through VR glasses. The aforementioned light source illuminates the actual simulated cockpit, and the inner wall of the light-shielding outer shell always appears black to the trainee.

[0039] 3. In the flight simulation mixed reality display system provided by the present invention, the three-dimensional digital model of the cockpit is a black model structure, and the position of the three-dimensional digital model of the cockpit in the virtual scene corresponds in real time to the position of the physical simulated cockpit, so that the image information of the three-dimensional digital model of the cockpit in the virtual scene is filled in by the physical simulated cockpit.

[0040] In this invention, the portion of the cockpit that trainees see in the virtual scene always overlaps with the real-world simulated cockpit. Furthermore, the three-dimensional digital model of the cockpit in the virtual scene is pure black, and the inner wall of the light-shielding outer shell is non-reflective. Through the concave semi-transparent and semi-reflective mirrors, trainees can only see the simulated cockpit and the virtual scene image attached to it. When the image reflected from the virtual scene and the image projected from the simulated cockpit simultaneously enter the human eye, it appears as if the real cockpit has been integrated into the virtual scene, without interfering with each other.

[0041] 4. The flight simulation mixed reality display system provided by this invention uses a six-degree-of-freedom inertial module to collect inertial measurement information and calculate position coordinates, which results in time-accumulated errors. Therefore, this invention uses an infrared measurement module to measure the trainee's head position and simultaneously correct for this accumulated error.

[0042] In the present invention, infrared measurements are used instead of inertial signal integration to determine the trainee's position when the trainee's head is near its normal position in the seat. This is the case most of the time. Inertial signal integration information is only used when the trainee's head moves outside the infrared measurement range, which typically lasts only a few seconds. In actual use, significant skeletal deformation can occur when the trainee's head moves outside the infrared measurement range. This only occurs when the trainee briefly observes an extreme position. The trainee's extreme position lasts only a short time before returning to normal, so the time required to calculate position using inertial signal integration information is short, and the aforementioned accumulated error is minimal. Furthermore, given the actual cockpit space and the volume of the flight helmet, the spatial range of the pilot's head is further limited, resulting in fewer opportunities for inertial signal integration to measure position. Therefore, the head position measurement system of this application maximizes the advantages of both positioning methods while ensuring sufficient accuracy. Furthermore, the head position measurement scheme described in this application also has the advantages of not involving visible light, eliminating blind spots, occupying a compact footprint, and being less susceptible to environmental influences. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A schematic diagram of the structure of the flight simulation mixed reality display system provided by the present invention;

[0045] Figure 2 A schematic diagram of the working principle of the see-through VR glasses provided by the present invention;

[0046] Figure 3 A schematic diagram of the three-axis acceleration directions measured by the six-degree-of-freedom inertial module provided by the present invention;

[0047] Description of reference numerals:

[0048] 1- Physical simulation cockpit; 2- Trainees; 3- Display screen; 4- Concave semi-transparent and semi-reflective mirror; 5- Light-shielding outer shell; 6- Lighting mechanism; 7- Virtual scene generation system; 8- Head position measurement system; 9- Transparent VR glasses. DETAILED DESCRIPTION

[0049] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0051] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0052] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0053] Example 1

[0054] A flight simulation mixed reality display system is described. Figure 1 As shown, it includes:

[0055] A physical simulated cockpit 1 is provided in a dark environment and is visible to the trainee 2; a three-dimensional digital model of the cockpit is generated based on the frame and outer dimensions of the physical simulated cockpit 1;

[0056] The light-shielding outer shell 5 is made of a non-reflective material and is used to block external light. The physical simulation cockpit 1 is disposed within the light-shielding outer shell 5. Of course, this embodiment does not specifically limit the shape and light-shielding method of the light-shielding outer shell 5. In other embodiments, the shape of the light-shielding outer shell 5 can be spherical or other shapes, and the inner wall can be covered with a special black paint or a wrapping layer to minimize light reflection.

[0057] The lighting mechanism 6 is disposed opposite the physical simulated cockpit 1 and is used to illuminate the physical simulated cockpit 1. Furthermore, the lighting mechanism 6 is a white light source and is disposed above and behind the seatback of the trainee 2, so that the trainee 2 cannot see the light source. Of course, this embodiment does not specifically limit the selection of the light source and the location of the lighting mechanism 6. In other embodiments, the lighting mechanism 6 may also be disposed above the trainee 2's head or in other locations that are not visible to the trainee 2. Furthermore, there is no specific limitation on whether the specific light source is white light.

[0058] The head position measurement system 8 is used to collect the position coordinates of the trainee's 2 head posture. When the head position measurement system 8 is working properly, the pure black three-dimensional digital model of the cockpit in the virtual scene seen by the trainee 2 always coincides with the physical simulated cockpit 1 in the real world, regardless of how the trainee's 2 head changes position and posture.

[0059] The virtual scene generation system 7 obtains the three-dimensional digital model of the cockpit and the head posture position coordinates and creates a virtual scene; the cockpit in the virtual scene created by the virtual scene generation system 7 is aligned with the three-dimensional digital model of the cockpit; and the head position of the trainee 2 in the virtual scene created by the virtual scene generation system 7 is aligned with the head posture position coordinates. Furthermore, the three-dimensional digital model of the cockpit is a black model structure, and the position of the three-dimensional digital model of the cockpit in the virtual scene corresponds in real time to the position of the physical simulated cockpit 1, so that the image information of the three-dimensional digital model of the cockpit in the virtual scene is supplemented by the physical simulated cockpit 1. In this embodiment, the portion of the cockpit seen by the trainee 2 in the virtual scene always overlaps with the physical simulated cockpit 1 in the real world. Furthermore, the three-dimensional digital model of the cockpit in the virtual scene is pure black, and the inner wall of the light-shielding outer shell 5 is made of a non-reflective material. Therefore, the three-dimensional digital model of the cockpit in the virtual scene image can be effectively replaced with the physical simulated cockpit 1.

[0060] Through-type VR glasses 9, such as Figure 2 As shown, it includes: a display screen 3 and a concave semi-transparent and semi-reflective mirror 4; the display screen 3 is used to display the virtual scene established by the virtual scene generation system 7; the display screen 3 and the concave semi-transparent and semi-reflective mirror 4 are arranged opposite to each other, and the content of the display screen 3 is reflected onto the concave semi-transparent and semi-reflective mirror 4 so that the trainee 2 can see the virtual scene; and the concave semi-transparent and semi-reflective mirror 4 is arranged opposite to the physical simulation cockpit 1, and the real scene of the illuminated physical simulation cockpit 1 is transmitted through the concave semi-transparent and semi-reflective mirror 4 so that the trainee 2 can see the real scene; the virtual scene and the real scene overlap.

[0061] In the present application, because the cockpit part in the virtual scene seen by the trainee 2 always coincides with the real-world physical simulation cockpit 1. At the same time, the cockpit three-dimensional digital model in the virtual scene is a pure black, light-shielded shell body 5 shell inner wall is not reflective, the trainee 2 through the concave half-reflective mirror 4 can only see the physical simulation cockpit 1 and the virtual scene image attached to the physical simulation cockpit 1. The virtual scene image reflected image and the physical simulation cockpit 1 projected image enter the human eye at the same time, and it looks like the real cockpit is integrated into the virtual scene and they do not interfere with each other. Through the above-mentioned through-type VR glasses 9 in the light-shielded shell body 5, the trainee 2's actual field of view can only have the physical simulation cockpit 1 that can be illuminated by the light source and can reflect the light, that is, it is dark except the cockpit. The physical simulation cockpit 1 reaches the human eye through the concave half-reflective mirror 4 of the above-mentioned through-type VR glasses 9. The computer-generated virtual scene picture is reflected by the concave half-reflective mirror 4 and also enters the human eye. However, because the cockpit three-dimensional digital model in the virtual scene picture is pure black, which is equivalent to no corresponding light, it looks like it is cut off, and only the rest of the virtual scene picture can be normally reflected into the human eye. The cockpit three-dimensional digital model is replaced by the image of the physical simulation cockpit 1, and the above-mentioned reflected and projected two pictures enter the human eye at the same time, and it looks like the real cockpit is integrated into the virtual scene and they do not interfere with each other.

[0062] In the present application, in order to maximize the advantages of the six-degree-of-freedom inertial module and the three-degree-of-freedom infrared measurement module, so as to ensure sufficient trainee 2 head measurement accuracy. The above-mentioned head position measurement system 8 includes: a six-degree-of-freedom inertial module and a three-degree-of-freedom infrared measurement module;

[0063] The six-degree-of-freedom inertial module is arranged on the helmet of the through-type VR glasses 9. In the powered state: the six-degree-of-freedom inertial module measures the head 3 attitude angle, spatial movement linear acceleration and speed of the trainee 2 in real time, and sends it to the virtual scene generation system 7 to measure the head attitude angle of the trainee 2 in real time, and obtains the head position information of the trainee 2 through the integral algorithm to update the virtual scene. The integral algorithm is: head original position + speed x time interval = head new position.

[0064] The head position measurement system 8 also includes: a three-degree-of-freedom infrared measurement module; the three-degree-of-freedom infrared measurement module is arranged on the inner cavity wall of the light-shielded shell body 5 and is opposite to the position of the seat of the trainee 2; the three-degree-of-freedom infrared measurement module is used to correct the cumulative error of the six-degree-of-freedom inertial module.

[0065] The six-degree-of-freedom inertial module and the three-degree-of-freedom infrared measurement module are used in cooperation in the embodiment, and when the head of the trainee 2 deviates from the infrared measurement range of the three-degree-of-freedom infrared measurement module, the integral algorithm of the six-degree-of-freedom inertial module is used to obtain the head pose information of the trainee 2.

[0066] When the head of the trainee 2 on the seat is near the normal position, the infrared measurement value is used to replace the integral of the inertial signal to obtain the position of the trainee 2, and the above-mentioned case is the case for most of the time. Only when the head of the trainee 2 deviates from the infrared measurement range, the inertial integral information is used, and the above-mentioned case usually only lasts for a few seconds. In actual use, when the head of the trainee 2 deviates from the infrared measurement range, the limb skeleton is also greatly deformed, and the above-mentioned case only occurs when the trainee 2 briefly observes the limit position. The duration of the trainee 2 in the limit position is very short and the trainee 2 returns to the normal position, so the time of using the inertial integral information to calculate the position is very short, and the accumulated error is not too large. At the same time, considering the actual cockpit space and the volume of the flight helmet, the space movement range of the head of the pilot is more limited, so the opportunity of using the inertial integral measurement position is less. Therefore, the head position measurement system 8 in the application maximizes the advantages of the two positioning methods, while ensuring sufficient accuracy. Moreover, the above-mentioned head position measurement scheme in the application also has the advantages of not involving visible light, no dead angle, compact space occupation, and less environmental influence.

[0067] The three-degree-of-freedom infrared measurement module is arranged on the inner cavity wall of the light-shielding outer shell 5 and faces the seat of the trainee 2, and when the head of the trainee 2 deviates from the infrared measurement range of the three-degree-of-freedom infrared measurement module, the integral algorithm of the six-degree-of-freedom inertial module is used to obtain the head pose information of the trainee 2.

[0068] In the embodiment, as shown in Figure 3 The six-degree-of-freedom inertial module uses the spatial pose maintaining characteristics of the gyroscope to measure the three-axis acceleration of the helmet position of the see-through VR glasses 9 and the change of the pose Euler angle relative to the reference position in real time, and calculates the speed of the three axes according to the integral of the three-axis acceleration; wherein the three-axis acceleration is ax axis, ay axis and az axis; the pose Euler angle includes pitch angle, roll angle and heading angle. The six-degree-of-freedom inertial module ensures that the module is in a static state at each power-on, so as to facilitate the initial alignment of the internal calculation module.

[0069] In the embodiment, the three-degree-of-freedom infrared measurement module includes a camera and a multi-source positioning software, and the multi-source positioning software is an opentrack software.

[0070] The operator to be positioned wears a helmet on which three strong reflective points are arranged, the emitted light emitted by the lighting mechanism is reflected on the strong reflective points and enters the camera; the camera is provided with an infrared filter to reduce the interference of external light;

[0071] The opentrack software runs and reads the video signal of the camera, and the multi-source positioning software calculates the head posture and position information of the operator according to the relative relationship change of the image formed by the reflective points in the video picture; the head posture and position information are used for three-dimensional correction of the three-dimensional infrared measurement module.

[0072] In addition, in the embodiment, the acquisition cycle of the three-dimensional infrared measurement module is 30HZ, and the position measurement error does not increase with time; the acquisition cycle of the six-dimensional inertial module is 100HZ, and the position measurement error is amplified with time accumulation. The present application realizes the fusion of the advantages of the two by using a comprehensive algorithm: the head position of the measured person = original position + velocity x time interval, wherein the velocity is output by the inertial module at 100HZ, the time interval is fixed at 0.01 seconds, and the error has been processed by the inertial module. The calculated value of the original position is denoted as p0, which has two different sources: the result of the last integral calculation, denoted as p01, with an update frequency of 100HZ; the measurement result of the three-dimensional infrared measurement module, denoted as p02, with an update frequency of 30HZ. In order to eliminate the cumulative error and ensure the update rate of 100HZ, the following method is adopted to determine the value of p0: set the time interval variable dt to 0, initialize it to 0, and add 1 each time, and reset it to 0 when it is greater than or equal to 2. When dt<2, p0=p01, otherwise p0=p01+(p02-p01) / N, wherein N>=1, the larger N is, the smoother the system is, and the smaller the system error is. Generally, it is recommended to take 1 to 2.

[0073] Of course, in the embodiment, the way of illuminating the physical simulation cockpit 1 is not specifically limited, and in other embodiments, the self-luminous physical simulation cockpit 1 can be used without the lighting mechanism 6, so that the trainee 2 can see the small characters and line symbols on the physical simulation cockpit 1.

[0074] Of course, in the embodiment, the specific name of the multi-source positioning software is not specifically limited, and in other embodiments, the multi-source positioning software can also be other positioning software.

[0075] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A flight simulation mixed reality display system, characterized in that: include: A physical simulated cockpit (1) is arranged in a dark environment, wherein the physical simulated cockpit (1) is visible to a trainee (2); a three-dimensional digital model of the cockpit is generated based on the frame and outer contour dimension information of the physical simulated cockpit (1); A head position measurement system (8) for collecting the head posture position coordinates of the trainee (2); the head position measurement system (8) comprises: a six-degree-of-freedom inertial module, and a three-degree-of-freedom infrared measurement module for correcting the accumulated error of the six-degree-of-freedom inertial module; the six-degree-of-freedom inertial module is arranged on the helmet of the see-through VR glasses (9); the three-degree-of-freedom infrared measurement module is arranged at a position facing the seat of the trainee (2); A virtual scene generation system (7) acquires the three-dimensional digital model of the cockpit and the head posture position coordinates and establishes a virtual scene; the cockpit in the virtual scene established by the virtual scene generation system (7) is aligned with the three-dimensional digital model of the cockpit; and the head position of the trainee (2) in the virtual scene established by the virtual scene generation system (7) is aligned with the head posture position coordinates; Transmissive VR glasses (9) comprise: a display screen (3) and a concave semi-transparent and semi-reflective mirror (4); the display screen (3) is used to display the virtual scene created by the virtual scene generation system (7); the display screen (3) and the concave semi-transparent and semi-reflective mirror (4) are arranged relative to each other, and the content of the display screen (3) is reflected onto the concave semi-transparent and semi-reflective mirror (4) so ​​that the trainee (2) can see the virtual scene; and the concave semi-transparent and semi-reflective mirror (4) and the physical simulation cockpit (1) are arranged relative to each other, and the real scene of the illuminated physical simulation cockpit (1) is transmitted through the concave semi-transparent and semi-reflective mirror (4) so ​​that the trainee (2) can see the real scene; the virtual scene and the real scene overlap; The flight simulation mixed reality display system further comprises: a light-shielding outer shell (5) and a lighting mechanism (6); the light-shielding outer shell (5) is made of a non-reflective material and is used to shield external light; the physical simulation cockpit (1) is arranged in the light-shielding outer shell (5); The lighting mechanism (6) is arranged opposite to the physical simulation cockpit (1) and is used to illuminate the physical simulation cockpit (1); and the lighting mechanism (6) is arranged at a position that is invisible to the trainee (2); The three-degree-of-freedom infrared measurement module includes: a camera and multi-source positioning software; The helmet worn by the operator who needs to be located is provided with three strong reflective points, and the light emitted by the lighting mechanism (6) shines on the strong reflective points and is reflected and enters the camera; the camera is provided with an infrared filter to reduce interference from external light; The multi-source positioning software runs and reads the video signal of the camera. The multi-source positioning software calculates the operator's head posture and position information based on the changes in the relative relationship of the image formed by the reflective points in the video screen; the head posture and position information is used for the three-degree-of-freedom correction of the three-degree-of-freedom infrared measurement module.

2. The flight simulation mixed reality display system according to claim 1, characterized in that: The three-degree-of-freedom infrared measurement module is arranged on the inner wall of the light-shielding outer shell (5) and is directly opposite to the seat of the trainee (2). When the six-degree-of-freedom inertial module and the three-degree-of-freedom infrared measurement module are used in conjunction with each other, when the head of the trainee (2) deviates from the infrared measurement range of the three-degree-of-freedom infrared measurement module, the integration algorithm of the six-degree-of-freedom inertial module is used to obtain the head posture information of the trainee (2).

3. The flight simulation mixed reality display system according to claim 1 or 2, characterized in that: The six-degree-of-freedom inertial module utilizes the spatial attitude holding characteristics of the gyroscope to measure the three-axis acceleration of the helmet position of the see-through VR glasses (9) and the change of the attitude Euler angle relative to the reference position in real time, and calculates the speed of the three axes based on the integration of the three-axis acceleration; The three-axis acceleration is the ax axis, the ay axis and the az axis; the attitude Euler angle includes: pitch angle, tilt angle and heading angle; Wherein, each time the six-degree-of-freedom inertial module is powered on, the module is ensured to be in a stationary state to facilitate the initial alignment of the internal computing module.

4. The flight simulation mixed reality display system according to claim 1, characterized in that: The multi-source positioning software is opentrack software; When powered on, the six-degree-of-freedom inertial module measures the head rotation posture, linear acceleration and speed of spatial movement of the trainee (2) in real time, and sends them to the virtual scene generation system (7) to measure the head posture angle of the trainee (2) in real time, and obtains the head posture information of the trainee (2) through an integration algorithm to update the virtual scene; the integration algorithm is: original head position + speed × time interval = new head position.

5. The flight simulation mixed reality display system according to claim 1, characterized in that: The lighting mechanism (6) is a white light source and is arranged above and behind the seat back of the trainee (2).

6. The flight simulation mixed reality display system according to claim 1, characterized in that: The three-dimensional digital model of the cockpit is a black model structure, and the position of the three-dimensional digital model of the cockpit in the virtual scene corresponds in real time to the position of the physical simulated cockpit (1), so that the image information of the three-dimensional digital model of the cockpit in the virtual scene is filled in by the physical simulated cockpit (1).

Images