Flight simulator based on mixed reality

Abstract

The utility model relates to the technical field of flight simulators, and provides a flight simulator based on mixed reality, which comprises a console device, a cockpit, an interaction assembly and a mixed reality head-mounted display, and the interaction assembly is arranged in the cockpit; the mixed reality head-mounted display and the interaction assembly are respectively connected with the console device, and the console device is used for receiving and processing data information of the interaction assembly and is also used for adjusting a visual image of the mixed reality head-mounted display; the cockpit comprises a shell, a window and a seat, an opening is formed in the shell, the window is installed at the opening, and the seat is arranged in the shell; the window comprises a green curtain window, and at least the window in front of the seat is the green curtain window. According to the utility model, a mode of combining the mixed reality head-mounted display and the green screen window is adopted, a coherent and non-segmented visual experience is provided, a pilot can directly see a visual image through the green screen window, a more vivid visual environment is created for flight simulation training, immersive training is realized, and the simulation training effect is enhanced.

Description

Technical Field

[0001] The utility model relates to the technical field of flight simulators, in particular to a flight simulator based on mixed reality. Background Art

[0002] Flight simulators can simulate real flight environments and provide pilots with a safe training platform, allowing them to experience various flight conditions in a simulated environment, including those special dangerous conditions that are difficult to encounter in reality. This can not only reduce the risks in the training process, but also significantly reduce training costs and improve training efficiency.

[0003] In recent years, the rapid development of virtual reality (VR), augmented reality (AR) and mixed reality (MR) technologies has made it possible to upgrade flight simulators. These advanced technologies have been gradually introduced into flight simulators, providing pilots with a more realistic and immersive flight experience.

[0004] Traditional flight simulators usually use multiple display screens or projection systems to build a visual system, and build a green screen outside the cockpit to simulate the external environment. Although this design method can simulate the flight environment to a certain extent, the physical interaction between the user and the virtual objects is very limited. Pilots often cannot get enough immersion during the simulation, which affects their real experience.

[0005] In addition, traditional visual systems have some technical limitations. For example, the connection between multiple display screens or projection systems may cause visual discontinuity, affecting the pilot's judgment and operation. Utility Model Content

[0006] The utility model provides a flight simulator based on mixed reality, which is used to solve the problem in the prior art that a traditional green screen is built around the cockpit of the flight simulator, which reduces the pilot's sense of immersion and affects the pilot's real experience.

[0007] The utility model provides a flight simulator based on mixed reality, comprising: a console device, a cockpit, an interactive component and a mixed reality head display, wherein the interactive component is arranged in the cockpit; the mixed reality head display and the interactive component are respectively connected to the console device, and the console device is used to receive and process data information of the interactive component, and is also used to adjust the visual image of the mixed reality head display; the cockpit comprises a shell, a window and a seat, the shell is provided with an opening, the window is installed at the opening, and the seat is arranged in the shell; the window comprises a green screen window, and at least the window directly in front of the seat is the green screen window.

[0008] According to a flight simulator based on mixed reality provided by the utility model, the green screen window includes electrochromic glass and a window frame, the window frame is provided with a receiving cavity, and the electrochromic glass is installed in the receiving cavity.

[0009] According to a mixed reality-based flight simulator provided by the utility model, the green screen window also includes a lamp body, and the lamp body is arranged on a side of the window frame close to the mixed reality head display.

[0010] According to a mixed reality-based flight simulator provided by the utility model, a mounting groove is provided on a side of the window frame close to the mixed reality head display, and the lamp body is installed in the mounting groove.

[0011] According to a flight simulator based on mixed reality provided by the utility model, the electrochromic glass includes a first glass plate, a second glass plate and an electrochromic film, and the electrochromic film is arranged between the first glass plate and the second glass plate.

[0012] According to a flight simulator based on mixed reality provided by the utility model, the green screen window also includes a power supply component, and the electrochromic film and the lamp body are respectively connected to the power supply component.

[0013] According to a mixed reality-based flight simulator provided by the utility model, the console device includes a platform body, a display component, an input component and a control component, and the display component, the input component and the control component are all arranged on the platform body; the display component and the input component are electrically connected to the control component respectively.

[0014] According to a mixed reality-based flight simulator provided by the utility model, the control component includes a data processing module and an image generation module, the data processing module is electrically connected to the image generation module, the data processing module is electrically connected to the interactive component, and the image generation module is communicatively connected to the mixed reality head display; the data processing module is used to receive data information of the interactive component and process it into flight dynamics; the image generation module is used to receive the flight dynamics, generate a visual image, and input it into the mixed reality head display.

[0015] According to a flight simulator based on mixed reality provided by the utility model, the interactive components include a joystick, a control panel and a throttle control console.

[0016] According to a mixed reality-based flight simulator provided by the utility model, the mixed reality head display includes a mixed reality head-mounted device.

[0017] The mixed reality-based flight simulator provided by the utility model adopts a combination of a mixed reality head display and a green screen window to provide a coherent and non-segmented visual experience. The pilot can directly see the visual image through the green screen window, creating a more realistic visual environment for flight simulation training, realizing immersive training, and enhancing the simulation training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a structural schematic diagram of the cockpit provided by the utility model;

[0020] Figure 2 It is a structural schematic diagram of the green screen window provided by the utility model;

[0021] Figure 3 It is a structural schematic diagram of the lamp body provided by the utility model;

[0022] Figure 4 It is a schematic diagram of the structure of the mixed reality head display provided by the utility model;

[0023] Figure 5 It is a structural schematic diagram of the console device provided by the utility model;

[0024] Reference numerals:

[0025] 10. console device; 11. console body; 12. input component; 13. display component; 14. control component;

[0026] 20. Cockpit; 21. Shell; 22. Green screen window; 221. Electrochromic glass; 222. Window frame; 223. Lamp body; 23. Seat; 24. Joystick; 25. Control panel; 26. Throttle control console;

[0027] 30. Mixed reality headsets. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the utility model clearer, the technical solution of the utility model will be described clearly and completely in conjunction with the drawings in the utility model. Obviously, the described embodiments are part of the embodiments of the utility model, not all of the embodiments. Based on the embodiments in the utility model, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the utility model.

[0029] In the description of the embodiments of the present utility model, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present utility model can be understood according to specific circumstances.

[0030] In the description of the embodiments of the present invention, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present invention.

[0031] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present utility model, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0032] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0033] The disclosure below provides many different embodiments or examples for realizing different structures of the utility model. In order to simplify the disclosure of the utility model, the components and settings of specific examples are described below. Of course, they are merely examples, and the purpose is not to limit the utility model. In addition, the utility model can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. In addition, the utility model provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the applicability of other processes and / or the use of other materials.

[0034] Combine the following Figures 1 to 5 A mixed reality based flight simulator of the present invention is described.

[0035] The mixed reality-based flight simulator provided by the embodiment of the utility model comprises: a console device 10, a cockpit 20, an interactive component and a mixed reality head display 30, wherein the interactive component is arranged in the cockpit 20, and the mixed reality head display 30 is worn on the head of the driver, who is located in the cockpit 20. The mixed reality head display 30 and the interactive component are respectively connected to the console device 10, and the console device 10 is used to receive data information of the interactive component and to adjust the mixed reality head display 30. The cockpit 20 comprises a shell 21, a window and a seat 23, wherein the shell 21 is provided with an opening, the window is installed at the opening, and the seat 23 is arranged in the shell 21; the window comprises a green screen window 22, and at least the window directly in front of the seat 23 is a green screen window 22.

[0036] Mixed reality (MR) combines the characteristics of virtual reality (VR) and augmented reality (AR) technologies, not only bringing users into a completely virtual environment, but also superimposing virtual objects or information into the real world. Mixed reality provides a new way of interaction that can bring an immersive experience to users.

[0037] The cockpit 20 provided in the embodiment of the utility model adopts a 1:1 replica of a real aircraft cabin, and its appearance and interior are consistent with the cabin of a real aircraft. The interactive components used inside the cockpit 20 are consistent with the components inside the cockpit 20 of a real aircraft. The pilot can experience real operational feedback, and improve his understanding and adaptability of the aircraft's control characteristics, enhance interactivity, and enhance practical operation perception.

[0038] The interactive component is connected to the console device 10. The interactive component includes components such as a joystick 24 and a gesture recognition sensor. The user physically selects and manipulates the interactive component. The console device 10 receives and processes the data information of the interactive component, and adjusts the visual image of the mixed reality headset 30 according to the corresponding information, seamlessly combining the input of the real world and the output of the virtual world to experience flying in the virtual world.

[0039] The utility model adopts a mixed reality head display 30, which reduces the dependence on large machinery and multi-screen displays or projectors compared to training based entirely on real aircraft or training based on traditional flight simulators, and can significantly reduce costs.

[0040] Furthermore, the cockpit 20 includes a shell 21 and a window. An opening is provided on the shell 21, and the window is installed at the opening. In one embodiment, the shell 21 is formed by a plurality of windows.

[0041] like Figure 1 As shown, the seat 23 is arranged in the shell 21 and worn on the head by the driver. The driver sits on the seat 23 and manipulates the interactive components. The mixed reality headset 30 can combine the virtual world with the real world. The user can interact with the virtual objects and overlay the virtual content into the real world, and adjust the display of the virtual content according to the user's perspective.

[0042] In one embodiment, at least the window directly in front of the seat 23 is a green screen window 22 .

[0043] Green screen technology, also known as Chroma Keying, is a visual effects technology that allows video producers or photographers to remove a specific color (usually green or blue) in front of the subject and replace the background with another image or video. Using green screen technology can create effects that appear to be shot in a specific environment or scene, when in fact these effects are added during post-production.

[0044] The embodiment of the utility model sets the window directly in front of the seat 23 as a green screen window 22. When the pilot looks at any "window" of the cockpit 20, the green screen window 22 can help the mixed reality head display 30 or the software to more accurately and easily distinguish the real world (various equipment in the cockpit 20) and the area that needs to be replaced by virtual content (the flight view seen by the pilot during flight, such as clouds, land, airports and other virtual dynamic flight views.), which can reduce erroneous image coverage. At the same time, the green screen window provides a uniform background that can be easily identified and removed by the software. Therefore, when the pilot looks at the green screen window 22, a complex external flight picture can be rendered in real time to replace the green screen picture, thereby improving the video resolution and visual rendering effect, and increasing the pilot's sense of immersion.

[0045] Compared with the traditional green screen and projection at a distance, the embodiment of the utility model provides a coherent visual experience without a sense of separation, achieving a higher level of spatial and visual integration. When the pilot is performing simulated flight, he can directly see a highly realistic simulation of the outside world through the green screen window 22, which makes the pilot feel that he is really flying in the air, thereby increasing the effect of simulation training.

[0046] In one embodiment, all windows of the cockpit 20 are green screen windows 22. When the pilot is performing simulated flight, he can use any window to make the virtual image more naturally integrated into the pilot's direct line of sight, thereby enhancing the pilot's sense of immersion.

[0047] The mixed reality-based flight simulator provided by the embodiment of the utility model adopts a combination of a mixed reality head display 30 and a green screen window 22 to provide a coherent and non-segmented visual experience. The pilot can directly see the visual image through the green screen window 22, creating a more realistic visual environment for flight simulation training, realizing immersive training, and enhancing the simulation training effect.

[0048] like Figure 2 As shown, the green screen window 22 provided in the embodiment of the utility model includes an electrochromic glass 221 and a window frame 222. The window frame 222 is provided with a receiving cavity, and the electrochromic glass 221 is installed in the receiving cavity.

[0049] The electrochromic glass 221 can change the color of the coating inside the glass by applying voltage, thereby controlling the amount of light and the transparency of the glass; that is, the glass switches between transparent and opaque, and the electrochromic glass 221 in the utility model switches between transparent glass and green glass. When the pilot is in normal use, the electrochromic glass 221 remains transparent to ensure that the cockpit 20 seen by the trainee is consistent with the real cockpit 20. When performing mixed reality simulated flight, by applying a certain voltage, the electrochromic glass 221 becomes opaque glass, such as green glass.

[0050] Figure 2 and Figure 3 As shown, the green screen window 22 provided in the embodiment of the utility model further includes a lamp body 223, which is arranged on a side of the window frame 222 close to the mixed reality head display 30, that is, inside the cockpit 20. Specifically, a mounting groove is provided on a side of the window frame 222 close to the mixed reality head display 30, and the lamp body 223 is installed in the mounting groove.

[0051] When performing mixed reality simulated flight, the electrochromic glass 221 becomes opaque, and the light body 223 radiates through the glass, producing a green screen-like effect. The light body 223 can be an LED light bar, which has the characteristics of low energy consumption, long life and adjustable color.

[0052] In one embodiment, a plurality of lamp bodies 223 are provided at circumferential intervals on a side of the window frame 222 close to the mixed reality head display 30 .

[0053] The electrochromic glass 221 provided in the embodiment of the present invention comprises a first glass plate, a second glass plate and an electrochromic film, wherein the electrochromic film is arranged between the first glass plate and the second glass plate.

[0054] Specifically, a film is arranged between the electrochromic film and the first glass plate, and between the electrochromic film and the second glass plate. Preferably, the first glass plate and the second glass plate are of the same size, the electrochromic film is smaller than the first glass plate and the second glass plate, and a film is also arranged on the periphery of the electrochromic film to fill the gap between the first glass plate and the second glass plate. The film can be at least one of SGP, PVB, EVA and the like.

[0055] The green screen window 22 provided in the embodiment of the utility model further includes a power supply component, the electrochromic film and the lamp body 223 are respectively connected to the power supply component, and the power supply component is used to supply power to the electrochromic film and the lamp body 223.

[0056] like Figure 5 As shown, the console device 10 provided in the embodiment of the utility model includes a platform 11, a display component 13, an input component 12 and a control component 14, wherein the display component 13, the input component 12 and the control component 14 are all arranged on the platform 11; the display component 13 and the input component 12 are electrically connected to the control component 14 respectively.

[0057] The input device includes a keyboard, through which the instructor can create, edit and publish customized learning tasks for different students or groups of students and provide real-time feedback on task entry tracking and completion status.

[0058] The control component 14 includes an integrated database that stores various forms of learning materials such as teaching videos, simulation cases, operation manuals, interactive Q&A, etc. Users can learn through the display component 13 and learn anytime and anywhere. They can also check their learning progress and grades, and teachers can also adjust their teaching strategies based on the learning situation.

[0059] The control component 14 of the embodiment of the utility model includes a data processing module and an image generation module. The data processing module is electrically connected to the image generation module, the data processing module is electrically connected to the interactive component, and the image generation module is communicatively connected to the mixed reality head display 30; the data processing module is used to receive data information of the interactive component and process it into flight dynamics; the image generation module is used to receive flight dynamics, generate visual images, and input them into the mixed reality head display 30.

[0060] It should be noted that the visual image is generated based on the pilot's operation of the interactive components in the cockpit 20 and the flight data of the simulator. The image generation module is used to create a virtual scene and dynamically adjust the viewing angle and environment according to the flight data to ensure that it matches the pilot's operation.

[0061] like Figure 1 As shown, the interactive components in the embodiment of the utility model include a joystick 24, a control panel 25 and a throttle control console 26, wherein the control panel 25 is provided with control keys, and the throttle control console 26 is used to control the size of the throttle. It should be noted that the positions of the interactive components such as the joystick 24, the control panel 25 and the throttle control console 26 are consistent with the positions of the real aircraft cockpit 20, and can simulate the functions and performance of the equipment. For example, the joystick 24 can respond to the pilot's operating input and feedback to various instruments and displays in a manner close to that of a real aircraft. The functional restoration of the interactive components ensures the effectiveness and authenticity of the training.

[0062] The joystick 24, the control panel 25, the throttle console 26 and other interactive components are equipped with corresponding sensors to detect the user's operation and transmit the signal to the control component 14. The control component 14 receives the corresponding data information and adjusts the virtual flight visual image in the mixed reality head display 30 according to the data information.

[0063] like Figure 4 As shown, the mixed reality head display 30 in the embodiment of the present invention includes a mixed reality head-mounted device, such as a mixed reality helmet, mixed reality glasses, etc.

[0064] The embodiment of the utility model provides a flight training platform that is both realistic and flexible. The pilot can not only physically operate real aircraft equipment, but also visually experience environmental changes under various flight conditions. This simulator that combines physical elements with virtual reality technology will be of great value in flight training, emergency simulation, and flight skill improvement.

[0065] The embodiment of the utility model is based on a mixed reality flight simulator. The instructor can customize personalized learning tasks on the console device 10 according to the student's current skill level and learning progress. The tasks include basic flight principle learning, aircraft operation training, or advanced emergency handling ability training. The instructor can use various forms of teaching resources such as videos, simulated scenarios, and interactive questions and answers to assist learning tasks and promote students' in-depth understanding and mastery of knowledge points.

[0066] The console device 10 provides a wealth of learning resources such as course videos, simulation tests, flight operation manuals, etc., supporting students to learn anytime and anywhere. The console device 10 can automatically record the students' learning progress and test scores, so that students can view their learning status in real time, and teachers can adjust teaching strategies accordingly.

[0067] The cockpit 20 in the embodiment of the utility model is a 1:1 physical cockpit 20 that is consistent with the cabin of a real aircraft. The appearance and interior are consistent with the configuration of the cockpit 20 of a real aircraft (the green screen window 22 is transparent glass during the visit). The interactive components in the cabin, including the joystick 24, the control panel 25, the throttle console 26 and the switches, are consistent with the position, size and touch of the real aircraft. At the same time, the interactive components can simulate the functions and performance of the equipment. The instructor can demonstrate various operations of the aircraft in the real cockpit 20, including using the navigation system, executing take-off and landing procedures, emergency handling, etc., so that the trainees can have an intuitive understanding of flight operations.

[0068] The mixed reality-based flight simulator of the embodiment of the utility model provides an immersive training environment, and through mixed reality technology, provides a highly realistic flight environment. Trainees will experience a variety of flight conditions from early morning to night, from sunny days to storms, allowing trainees to learn how to deal with various flight situations in a safe virtual environment.

[0069] When the trainee operates the joystick 24 or button of the simulator, the corresponding action will be immediately fed back to the control component 14. Then, the control component 14 will adjust the virtual flight environment in the MR device according to the trainee's operation, such as changing the flight direction, altitude or speed, and provide real-time operation feedback. The flight simulator based on mixed reality can simulate emergency situations such as engine failure, navigation system failure, and bad weather, and train trainees in emergency response and decision-making capabilities in various non-standard situations.

[0070] After the training, the control component 14 will provide a detailed performance report based on the trainee's performance in the mixed reality-based flight simulator, including flight path, operation accuracy, emergency handling, etc., and feed the data back to the storage module of the control component 14. The instructor will conduct one-on-one evaluation and guidance based on this data.

[0071] The embodiment of the utility model uses an integrated design concept of "principle-structure-flight". By integrating learning tasks issued by the console device 10, autonomous learning, a visit to a real aircraft cockpit 20, and flight training based on a mixed reality flight simulator, it closely combines theoretical learning with practical operations, providing a new and systematic education model for flight teaching.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the utility model, rather than to limit it. Although the utility model has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the utility model.

Claims

1. A flight simulator based on mixed reality, characterized in that: include: A console device, a cockpit, an interactive component and a mixed reality head display, wherein the interactive component is arranged in the cockpit; The mixed reality head display and the interactive component are respectively connected to the console device, and the console device is used to receive and process data information of the interactive component and to adjust the visual image of the mixed reality head display; The cockpit includes a shell, a window and a seat. The shell is provided with an opening, the window is installed at the opening, and the seat is arranged in the shell; the window includes a green screen window, and at least the window directly in front of the seat is the green screen window.

2. The mixed reality based flight simulator according to claim 1, characterized in that: The green screen window comprises an electrochromic glass and a window frame. The window frame is provided with a receiving cavity, and the electrochromic glass is installed in the receiving cavity.

3. The mixed reality based flight simulator according to claim 2, characterized in that: The green screen window also includes a lamp body, which is arranged on a side of the window frame close to the mixed reality head display.

4. The mixed reality based flight simulator according to claim 3, characterized in that: A mounting groove is provided on a side of the window frame close to the mixed reality head display, and the lamp body is installed in the mounting groove.

5. The mixed reality based flight simulator according to claim 3, characterized in that: The electrochromic glass comprises a first glass plate, a second glass plate and an electrochromic film, wherein the electrochromic film is arranged between the first glass plate and the second glass plate.

6. The mixed reality based flight simulator according to claim 5, characterized in that: The green screen window also includes a power supply component, and the electrochromic film and the lamp body are respectively connected to the power supply component.

7. The mixed reality based flight simulator according to any one of claims 1 to 6, characterized in that: The console device comprises a platform body, a display component, an input component and a control component. The display component, the input component and the control component are all arranged on the platform body; the display component and the input component are electrically connected to the control component respectively.

8. The mixed reality based flight simulator according to claim 7, characterized in that: The control component includes a data processing module and an image generation module, the data processing module is electrically connected to the image generation module, the data processing module is electrically connected to the interaction component, and the image generation module is communicatively connected to the mixed reality head display; The data processing module is used to receive the data information of the interactive component and process it into flight dynamics; the image generation module is used to receive the flight dynamics, generate a visual image, and input it into the mixed reality head display.

9. The mixed reality based flight simulator according to claim 1, characterized in that: The interactive components include a joystick, a control panel and a throttle console.

10. The mixed reality based flight simulator according to claim 1, characterized in that: The mixed reality head display includes a mixed reality head-mounted device.

Images