Immersive cockpit hybrid display system for super panoramic presentation

Abstract

The invention belongs to the technical field of cockpit display and control design, and discloses an immersive cockpit mixed display system for super panoramic presentation, which is composed of a display and control processor, a multi-projector system, a transparent cockpit cover and a combined screen, integrates cockpit cover projection display and combined screen display, reduces the limitation of the physical form of a cockpit on the visual field of a pilot, and improves the visual effect of the pilot. And mixed display of the augmented reality information outside the full visual field and in the effective visual field is realized. The technical problem that screen display is difficult to adapt to a cabin structure is solved, and a mixed reality display effect and dynamic switching between augmented reality and virtual reality under a non-near-to-eye display condition are realized by combining screen display and transparent cabin cover projection.

Description

Technical Field

[0001] The present invention belongs to the technical field, and in particular relates to an immersive cockpit hybrid display system for super panoramic presentation. Background Art

[0002] In a high-speed flight environment, an immersive cockpit hybrid display system with super panoramic presentation function is an ideal interactive system for the driver to achieve omnidirectional situational awareness and ensure high-speed driving safety. To achieve a good immersive hybrid display effect, the cockpit display system must match the overall structure of the cockpit in terms of structure. At the same time, it needs to provide the driver with a clear and bright symbol or video image that can be superimposed on the real or virtual exterior scene and support a large field of view display. In addition, in order to meet the ergonomic requirements in a high-speed motion environment, the display system must also meet the performance requirements of sight distance matching and image clarity. Mixed reality helmets are one of the effective ways to meet the needs of immersive panoramic presentation in the cockpit, but as a near-eye display device, they have inherent defects such as limited field of view, 3D display dizziness, and increased head load. At the current stage, there are still many problems with long-term use. Summary of the invention

[0003] The purpose of the present invention is to meet the demand for super panoramic mixed reality display in the cockpit under high-speed flight environment, break through the inherent defects of mixed reality near-eye display devices, and realize an immersive cockpit display solution for super panoramic presentation based on combined screen display and transparent cabin cover projection, which effectively solves the display requirements of sight distance matching, clear image and compatibility with the cockpit structure.

[0004] The technical solution of the present invention is achieved in this way:

[0005] An immersive cockpit hybrid display system for super panoramic presentation, comprising:

[0006] The display control processor hosts relevant display processing software and supports the closed-loop implementation of display system functions. The core includes panoramic image generation and projection surface fusion.

[0007] The multi-projector system receives the video images after being fused and distributed by the display control processor for optical projection, and projects them onto the transparent hatch area for display;

[0008] The transparent canopy is arranged in the main visual area of ​​the cockpit, and receives the optical projection of the multi-projector system for projection fusion image display;

[0009] The combined screen receives the video image processed by the display control processor for screen display;

[0010] The transparent canopy projection display and the combined screen display are integrated to reduce the limitation of the pilot's field of vision due to the physical form of the cockpit, and to achieve a mixed display of the visual scene outside the full field of vision and the augmented reality information within the effective field of vision.

[0011] As a further solution of the present invention: the display and control processor stores full-view image generation and distribution software, projection surface fusion software, and combined screen picture adjustment software; the full-view image generation and distribution software is used to construct and generate full-view images and distribute them, the projection surface fusion software is used to fuse and process the projection pictures displayed in the transparent cabin cover area, and the combined screen picture adjustment software is used to adjust and process the screen pictures displayed in the combined screen area.

[0012] As a further solution of the present invention: the multi-projector system includes a top projector, a left projector and a right projector, the top projector is located at the top and rearward position of the seat in the cabin, the left projector and the right projector are symmetrically distributed on the left and right sides of the seat in the cabin, and the projection surface of the multi-projector system covers the entire middle area of ​​the glass cabin cover, meeting the driver's vision requirements for the upper half of the front.

[0013] As a further solution of the present invention: the transparent hatch cover includes a plexiglass hatch cover, a dimming film and a front projection film, the plexiglass hatch cover is used for structural support and transparent display, the dimming film is used to achieve transparent / closed switching display of the hatch cover, and the front projection film is used to achieve projection display with local information enhancement.

[0014] As a further solution of the present invention: the cockpit canopy is based on transparent organic glass and presents a special-shaped curved surface structure, which is divided into three areas from top to bottom: the upper area and the lower area are areas outside the horizontal field of vision and are not processed; the middle area is an area within the horizontal field of vision, and a flexible dimming film and a front projection film are attached to form an electro-liquid crystal film dimming glass, which is a projection display carrier of the transparent canopy; at the same time, the three sections of the area are closely spliced ​​through the edging structure, the transparent organic glass of the three sections of the area is fixed, and the mechanical connection with the cabin body is undertaken.

[0015] As a further solution of the present invention: in the middle area of ​​the cockpit canopy, the dimming film is attached to the outer surface of the transparent organic glass, and the front projection film is attached to the inner surface of the transparent organic glass, and is used in conjunction with the dimming film. The dimming film and the front projection film are both flexible structures.

[0016] As a further solution of the present invention: the combined screen includes a front integrated display and control screen, left and right auxiliary display and control screens, left and right external view display screens and a bottom external view display screen, which mainly covers the visual field space of the area below the pilot's front hemisphere.

[0017] As a further solution of the present invention: the bottom exterior view display screen is formed by splicing three display screens; the front integrated display control screen is used to display the main function interface information, the left auxiliary display control screen and the right auxiliary display control screen are used to display shortcut command controls and key backup information, and both support touch operations; the left exterior view display screen, the right exterior view display screen and the bottom exterior view display screen are used to display the exterior image and enhanced information from the corresponding perspective of the driver, and in a closed state, the virtual exterior view constructed by the virtual vision generation technology is used as the base for enhanced information display, and in a transparent state, the real exterior view collected by the extravehicular image sensor is used as the base for enhanced information display.

[0018] As a further solution of the present invention: the multi-projector system, the transparent cabin cover and the combined screen work together to meet the driver's display needs for the complete forward field of view. The horizontal field of view of the overall display area of ​​the cockpit is 210±10°, and the vertical field of view is 175±10°.

[0019] Beneficial effects of the present invention:

[0020] 1. The present invention is mainly aimed at high-speed flight environments and provides an immersive cockpit hybrid display system for super panoramic presentation. It breaks through the inherent defects of existing near-eye display devices such as limited field of view and 3D dizziness, solves the problem that the screen display is difficult to adapt to the cockpit structure, and provides an overall solution for cockpit super panoramic presentation.

[0021] 2. The present invention provides an immersive cockpit hybrid display system for super panoramic presentation. By combining screen display and transparent cabin cover projection, a mixed reality display effect under non-near-eye display conditions and dynamic switching between augmented reality and virtual reality are achieved.

[0022] 3. The present invention adopts a special-shaped screen splicing solution to achieve a combined screen display covering the driver's front to lower half of the field of vision. The real exterior scene display problem of the corresponding field of vision area is solved through the external image sensor of the cockpit. The virtual exterior scene display problem of the corresponding field of vision area is solved through the cockpit virtual scene generation technology, meeting the mixed reality display requirements.

[0023] 4. The present invention uses a flexible dimming film and a front projection film to attach the transparent cabin cover, which realizes the controllable switching of the transparent / closed state of the cabin's special-shaped curved cabin cover and the fusion imaging of multi-projector close-range ultra-short-focus projection, solves the adaptation problem of the hybrid display carrier and the cabin structure, and can well adapt to the streamlined structural requirements of the cabin product.

[0024] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a block diagram of the system composition of the present invention;

[0026] Figure 2 It is a schematic diagram of the layout of multiple projectors of the present invention;

[0027] Figure 3 This is a schematic diagram of a switchable glass transparent hatch cover of the present invention;

[0028] Figure 4 This is a schematic diagram of the combined screen layout of the present invention;

[0029] Figure 5 This is a schematic diagram of the cockpit display field of view of the present invention;

[0030] Figure 6 A schematic diagram of the panoramic image generation technology route of the present invention;

[0031] Figure 7 It is a schematic diagram of the projection surface fusion technology route of the present invention;

[0032] Figure 8 It is an overall schematic diagram of the present invention.

[0033] The reference numerals in the figure are: left projector 201, right projector 202, top projector 203, upper area 301, edging structure 302, middle area 303, lower area 304, dimming film 305, transparent organic glass 306, front projection film 307, front integrated display and control screen 401, left auxiliary display and control screen 402, right auxiliary display and control screen 405, left external view display screen 403, right external view display screen 406, bottom external view display screen 404. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0035] The present invention realizes an immersive cockpit display system for super panoramic presentation, which adopts an overall solution of combined screen display and transparent canopy projection, and is composed of a display control processor, a multi-projector system, a transparent canopy and a combined screen. The following is the composition of each module in the system and the specific implementation method.

[0036] (1) Examples of the present invention are as follows Figure 1 As shown in the system block diagram, in order to realize the immersive cockpit hybrid display for super panoramic presentation, the solution includes a display control processor, a multi-projector system, a transparent cabin cover and a combined screen. Among them, the display control processor is located in the equipment space behind the cabin.

[0037] (2) Figure 2As shown, the multi-projector system includes three parts: left projector 201, right projector 202 and top projector 203. The top projector is located at the top and rear of the seat in the cabin, and the left and right projectors are symmetrically distributed on the left and right sides of the seat in the cabin. The projection surface of the multi-projector system covers the entire middle area of ​​the glass cabin cover to meet the driver's vision requirements from the front to the upper half. Based on the previous optical path simulation verification, the projector-related parameters are selected: projection ratio 0.5:1, projection area 16:9. In addition, the number and position of projectors can be adjusted according to the display space layout of the cockpit object.

[0038] (3) Figure 3 As shown, the cockpit canopy is based on transparent organic glass 306, presenting a special-shaped curved surface structure, and is divided into three areas from top to bottom: the upper area 301 and the lower area 304 are outside the horizontal field of vision and are not processed; the middle area 303 is the area within the horizontal field of vision, and a flexible dimming film 305 and a front projection film 307 are attached to form an electro-liquid crystal film dimming glass, which is the projection display carrier of the transparent canopy; at the same time, the three sections of the area are closely spliced ​​through the edge structure 302, the transparent organic glass 306 of the three sections is fixed, and the mechanical connection with the cabin body is assumed. In addition, through simulated assembly, the fixing, support and opening and closing of the canopy are considered in advance, and sufficient internal circuit design space and mechanical installation and operation space are reserved for the dimming glass and the opening and closing structure.

[0039] (4) Figure 3 As shown, in the middle area of ​​the cockpit canopy, the dimming film 305 is attached to the outer surface of the transparent organic glass 306. The canopy can be made transparent or opaque by turning on and off the internal current to meet the needs of transparent / closed mixed display. In the closed state, full-range projection display can be achieved; the front projection film 307 is attached to the inner surface of the transparent organic glass 306 and is used in conjunction with the dimming film 305 to achieve the effect of reflecting the optical projection content. Under the condition of ensuring the external environment field of view, the display content of the enhanced information projection can be clearly reflected. The dimming film 305 and the front projection film 307 are both flexible structures, which ensure that they can be effectively bonded on the special-shaped curved surface, effectively reducing the structural constraints of the display carrier.

[0040] (5) Figure 4As shown, the combined screen includes a front integrated display control screen 401, a left auxiliary display control screen 402, a right auxiliary display control screen 405, a left exterior display screen 403, a right exterior display screen 406 and a bottom exterior display screen 404, which jointly cover the driver's front and lower half of the visual range, wherein the bottom exterior display screen 404 is spliced ​​by three display screens. The front integrated display control screen 401 is used to display the main function interface information, the left auxiliary display control screen 402 and the right auxiliary display control screen 405 are used to display shortcut command controls and key backup information, both of which support touch operation; the left exterior display screen 403, the right exterior display screen 406 and the bottom exterior display screen 404 are used to display the exterior image and enhanced information from the corresponding perspective of the driver, and the virtual exterior scene constructed by the virtual vision generation technology is used as the base for enhanced information display in the closed state, and the real exterior scene collected by the external image sensor is used as the base for enhanced information display in the transparent state.

[0041] (6) Figure 5 As shown in the figure, the multi-projector system, transparent cabin cover and combined screen work together to meet the driver's display needs for the complete forward field of view. The horizontal field of view of the overall display area of ​​the cockpit is 210±10° (105±5° on the left and right respectively), and the vertical field of view is 175±10° (75° on the top and 100±10° on the bottom).

[0042] (7) Figure 1 As shown, the display control processor hosts relevant display processing software to support the closed-loop implementation of the display system functions. The core software functions include panoramic image generation and projection surface fusion.

[0043] (8) Figure 6 As shown in the figure, the system realizes panoramic image generation, layering and marking based on Unity real-time virtual 3D scene. Firstly, a virtual 3D scene is constructed by combining geographic information data and other information sources, and the transparent hatch cover is placed in the virtual 3D scene. The viewpoint and the position change of the transparent hatch cover are driven based on the real-time position data of the cockpit, and the real scene map and enhanced information loading are dynamically adjusted to define the hatch cover layer (the layer where the transparent hatch cover is located), the scene layer (including the landscape and the sky, etc.) and the enhanced layer (including the enhanced information mark). Then, three virtual cameras are overlapped and placed at the head position of the average-sized pilot's head-up sitting posture, namely the panoramic camera, the enhanced camera and the hatch cover camera, and set to the panoramic image capture mode: the shooting layer of the enhanced camera only includes the enhanced layer, which provides the enhanced information image in the transparent state; the shooting layer of the panoramic camera includes the enhanced layer and the real scene layer, which provides the virtual panoramic image in the closed state; the shooting layer of the hatch cover camera only includes the hatch cover layer, which is used to mark the hatch cover display area, and the position of the hatch cover area in the picture is transmitted to the projection surface fusion software in the form of a second-order matrix, which supports the transparent hatch cover projection fusion.

[0044] (9) Figure 7 As shown in the figure, the system uses software projection fusion method to realize the close-range ultra-short-throw projection fusion of multiple projectors on special-shaped surfaces, ensuring that the mixed display content can cover the main field of view and reduce the image distortion. Specifically, it is mainly carried out through three steps: parameterized fitting of the projection surface and the image content, fusion of multiple projector surfaces, and projection image scene drawing:

[0045] Bezier surface is used to perform parameterized fitting of the projection surface, determine the projection range of each projector and the fusion area of ​​the overlapping parts, create a rasterized subdivision grid, and obtain the mapping relationship from the input image to the projection surface;

[0046] Perform a gradual brightness transition process on the overlapping areas of multiple projectors, so that the entire image transitions smoothly in the fusion area, and the brightness of the fusion area is kept consistent with that of the non-fusion area.

[0047] The screen scene is drawn according to the above results, and the output screen of each projector is determined so that the projected image is consistent with the curved surface, thus achieving the fusion and splicing effect of the projection.

[0048] The present invention provides an immersive cockpit hybrid display system for super panoramic presentation, the whole Figure 8As shown, the overall solution of combined screen and hatch projection is mainly composed of display control processor, multi-projector system, transparent hatch and combined screen. The multi-projector system includes top projector 1, left projector 2 and right projector 3, which receive the video images after fusion and distribution by the display control processor for optical projection and project them to the transparent hatch area for display. The transparent hatch is perfectly integrated with the main body of the cockpit structure and arranged in the main visual area of ​​the cockpit, including organic glass hatch, dimming film and front projection film: the organic glass hatch plays the role of physical support and structural connection as the structural foundation, and meets the basic transparent display requirements at the same time. The dimming film is attached to the outer layer of the organic glass hatch, and the requirements of transparent / closed hybrid display of the hatch are realized by turning on and off the internal current, and the full range of projection display is realized in the closed state; the front projection film is attached to the inner layer of the organic glass hatch, and is used in conjunction with the dimming film to realize the projection display with local information enhancement in the transparent state. The combined screen includes a front integrated display and control screen, left and right auxiliary display and control screens, left and right exterior display screens, and a bottom exterior display screen, which receive the video images processed and distributed by the display and control processor for display: the front integrated display and control screen and the left and right auxiliary display and control screens are used to display the cockpit function interface information and support touch operation; the left and right exterior display screens and the bottom exterior display screen are used to display the exterior image and enhanced information from the corresponding perspective of the driver, display the real exterior scene collected by the image sensor outside the cabin in a transparent state, and display the virtual exterior scene constructed by the virtual vision generation technology in a closed state. The display and control processor resides in the full-view image generation and distribution software, the projection surface fusion software, and the combined screen picture matching software: the full-view image generation and distribution software is used to construct a full-view image based on display data, exterior image and mode switching control signal, including cockpit function interface, enhanced information mark and real exterior scene or virtual exterior scene image; the projection surface fusion software is used to fuse and process the projection picture displayed in the transparent cabin cover area; the combined screen picture matching software is used to match and process the screen picture displayed in the combined screen area.

[0049] In summary, the present invention provides an immersive cockpit hybrid display system for super panoramic presentation. Through the system composition architecture, design principles, assembly relationship and application examples shown in the abstract drawings, the combined screen and the canopy projection are effectively integrated to form a cockpit super panoramic mixed reality display system for high-speed flight environments, solving the display requirements of sight distance matching, clear images and compatibility with the cockpit structure.

[0050] So far, the purpose of the present invention has been achieved.

[0051] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An immersive cockpit hybrid display system for super panoramic presentation, characterized in that: include: The display control processor hosts relevant display processing software and supports the closed-loop implementation of display system functions. The core includes panoramic image generation and projection surface fusion. The multi-projector system receives the video images after being fused and distributed by the display control processor for optical projection, and projects them onto the transparent hatch area for display; The transparent canopy is arranged in the main visual area of ​​the cockpit, and receives the optical projection of the multi-projector system for projection fusion image display; The combined screen receives the video image processed by the display control processor for screen display; The transparent canopy projection display and the combined screen display are integrated to reduce the limitation of the pilot's field of vision due to the physical form of the cockpit, and to achieve a mixed display of the visual scene outside the full field of vision and the augmented reality information within the effective field of vision.

2. The immersive cockpit hybrid display system for super panoramic presentation according to claim 1, characterized in that: The display control processor is resident with full-view image generation and distribution software, projection curved surface fusion software, and combined screen picture adjustment software; The full-view image generation and distribution software is used to construct and distribute the full-view image, the projection surface fusion software is used to fuse and process the projection image displayed in the transparent cabin cover area, and the combined screen image matching software is used to match and process the screen image displayed in the combined screen area.

3. An immersive cockpit hybrid display system for super panoramic presentation according to claim 1 or 2, characterized in that: The multi-projector system includes a top projector, a left projector and a right projector. The top projector is located at the top and rearward position of the seat in the cabin, and the left projector and the right projector are symmetrically distributed on the left and right sides of the seat in the cabin. The projection surface of the multi-projector system covers the entire middle area of ​​the glass cabin cover, meeting the driver's vision requirements for the upper half of the front.

4. The immersive cockpit hybrid display system for super panoramic presentation according to claim 3, characterized in that: The transparent hatch cover includes an organic glass hatch cover, a dimming film and a front projection film. The organic glass hatch cover is used for structural support and transparent display, the dimming film is used to realize transparent / closed switching display of the hatch cover, and the front projection film is used to realize projection display with local information enhancement.

5. The immersive cockpit hybrid display system for super panoramic presentation according to claim 4, characterized in that: The cockpit canopy is based on transparent organic glass and presents a special-shaped curved surface structure, which is divided into three areas from top to bottom: the upper area and the lower area are outside the level field of vision and are not processed; the middle area is within the level field of vision, and a flexible dimming film and a front projection film are attached to form an electro-liquid crystal film dimming glass, which is a projection display carrier for the transparent canopy; at the same time, the three sections of the area are closely spliced ​​through the edging structure, the transparent organic glass of the three sections is fixed, and the mechanical connection with the cabin body is undertaken.

6. The immersive cockpit hybrid display system for super panoramic presentation according to claim 5, characterized in that: In the middle area of ​​the cockpit canopy, the dimming film is attached to the outer surface of the transparent organic glass, and the front projection film is attached to the inner surface of the transparent organic glass. When used together with the dimming film, both the dimming film and the front projection film are flexible structures.

7. The immersive cockpit hybrid display system for super panoramic presentation according to claim 6, characterized in that: The combined screen includes a front integrated display and control screen, left and right auxiliary display and control screens, left and right external view display screens and a bottom external view display screen, which mainly covers the visual field space in the area below the pilot's front hemisphere.

8. The immersive cockpit hybrid display system for super panoramic presentation according to claim 7, characterized in that: The bottom exterior display screen is composed of three display screens; the front integrated display control screen is used to display the main function interface information, the left auxiliary display control screen and the right auxiliary display control screen are used to display shortcut command controls and key backup information, and both support touch operation; the left exterior display screen, the right exterior display screen and the bottom exterior display screen are used to display the exterior image and enhanced information from the corresponding perspective of the driver. In a closed state, the virtual exterior scene constructed by the virtual vision generation technology is used as the base for enhanced information display, and in a transparent state, the real exterior scene collected by the extravehicular image sensor is used as the base for enhanced information display.

9. The immersive cockpit hybrid display system for super panoramic presentation according to claim 8, characterized in that: The multi-projector system, transparent cabin cover and combined screen work together to meet the driver's display needs for the entire forward field of view. The horizontal field of view of the overall display area of ​​the cockpit is 210±10°, and the vertical field of view is 175±10°.

Images