Rapid prototype construction system and method for aviation human-computer interaction interface

Abstract

The invention relates to the technical field of aviation human-computer interaction interface design, in particular to a rapid prototype construction system and method for an aviation human-computer interaction interface. The system comprises an aviation man-machine interaction prototype node framework and an aviation man-machine interaction prototype rapid construction environment. The aviation man-machine interaction prototype node framework is used for describing hierarchical structures of nodes of a 2D prototype and a 3D prototype and attributes of all the nodes, the aviation man-machine interaction prototype node framework comprises a 2D prototype node framework and a 3D prototype node framework, and equipment display unit nodes in the 2D prototype node framework can be imported into the 3D prototype node framework; the aviation man-machine interaction prototype rapid construction environment comprises a project management tool, a component library editing tool, a 2D prototype editing tool, a 3D prototype editing tool and a console tool, and a component library, a 2D prototype and a 3D prototype can be rapidly constructed according to an aviation man-machine interaction prototype node framework. And the structural description requirement of the aviation man-machine interaction prototype can be met.

Description

Technical Field

[0001] This invention relates to the field of aviation human-computer interaction interface design technology, and specifically to a rapid prototyping system and method for aviation human-computer interaction interfaces. Background Technology

[0002] Currently, 2D prototypes are mainly built or custom-developed using 2D prototype editing tools such as Sketch, Axure, and Qt Design Studio, while 3D prototypes are mainly custom-developed using 3D engines such as Unity3D and UE.

[0003] Current rapid prototyping tools and methods are insufficient to support the concept verification needs of future aviation human-machine interfaces. This is because: 1) the 2D and 3D prototyping processes are relatively separate, resulting in high time, manpower, and learning costs; 2) they are not suitable for the professional and standardized requirements of human-machine interfaces in the aviation field; 3) they cannot meet the construction needs of future rapid prototyping 3D HMI design, dynamic mission simulation-driven, and multi-channel interaction; and 4) they cannot support collaborative design by multiple people in complex projects, making it difficult to reuse design results between the same project and different projects.

[0004] To address these issues, this invention proposes a rapid prototyping system and method for aviation human-computer interaction interfaces. Summary of the Invention

[0005] The purpose of this invention is to address the design and proof-of-concept requirements of aviation human-computer interaction interfaces by proposing a rapid prototyping system and method for aviation human-computer interaction interfaces. It constructs a building environment based on component library editing tools, 2D prototype editing tools, 3D prototype editing tools, and console tools, which can quickly build aviation human-computer interaction interface prototypes with multi-channel interaction, dynamic task simulation, and 2D / 3D hybrid rendering capabilities.

[0006] According to a first aspect of the present invention, a rapid prototyping system for aviation human-computer interaction interfaces is proposed, comprising: an aviation human-computer interaction prototype node framework and an aviation human-computer interaction prototype rapid construction environment; The aviation human-computer interaction prototype node framework is used to describe the hierarchical structure and attributes of nodes in 2D and 3D prototypes, including a 2D prototype node framework and a 3D prototype node framework. Device display unit nodes in the 2D prototype node framework can be imported into the 3D prototype node framework. The rapid prototyping environment for aviation human-computer interaction includes project management tools, component library editing tools, 2D prototype editing tools, 3D prototype editing tools, and console tools, which can quickly build component libraries, 2D prototypes, and 3D prototypes based on the aviation human-computer interaction prototype node framework.

[0007] The 2D prototype is used to simulate the human-machine interface software in the cockpit display and control equipment. It is built in the 2D prototype project by the 2D prototype editing tool and includes display resources at each level of the human-machine interface and their change logic.

[0008] The 3D prototype is used to simulate the three-dimensional cockpit environment and mission scenarios, and is built in the 3D prototype project using 3D prototype editing tools.

[0009] The component library is a node resource library composed of component nodes. It is built in the component library project by the component library editing tool and output as component prefabs, which are stored in the component library folder. The component nodes are nodes at the widget or window level, have style attribute referencing capabilities, and have restricted style attribute editing permissions.

[0010] The style attribute editing permission means that after a component is added to a 2D prototype project, only the text content and icon path of the component can be modified, and the style attributes can only be modified in the component library editing tool; the style attribute referencing function means that after the style attributes of a component are modified, the component nodes added to the 2D prototype project can also be modified synchronously; the style attributes include size attributes, position attributes, border attributes, fill attributes, image attributes, text style attributes, etc.

[0011] In one possible embodiment, the 2D prototype node framework includes device display unit nodes, page frame nodes, page nodes, window nodes, component (group) nodes, graphic element (group) nodes, and 3D model nodes; the 2D prototype node framework includes one or more device display unit nodes. The device display unit node refers to a display screen in the cockpit display and control equipment, such as the MFD device display unit, HUD device display unit, or HMD device display unit.

[0012] The device display unit node has a name attribute, an effective display area attribute, and a size attribute. The name attribute refers to the name of the node, the effective display area attribute refers to the width and height of the device display unit in millimeters, and the size attribute refers to the resolution in pixels.

[0013] A single device display unit node includes one or more page frame nodes. A page frame node refers to the page hierarchy and layout partitions within a display page, defined based on interactive behavior and functional characteristics; such as background layer, content layer, global control layer, and temporary layer. (See details) Figure 3 )Page frame nodes have name, size, page hierarchy, and grid attributes. The size attribute of a page frame node follows the size attribute of the device's display unit nodes. The page hierarchy attribute includes the current page hierarchy number, name, and layout partitions; a single page frame node can have multiple page hierarchy attributes. The layout partition attribute describes the name, position, and size of each partition within the current page hierarchy. The grid attributes include grid color, opacity, and grid size.

[0014] A single page frame node comprises one or more page nodes. A page node refers to the display page of a specific screen in a cockpit display and control device during a particular mode, such as the HUD navigation mode page, the HUD empty mode page, etc. Each page node has a name, size attribute, and current display page level attribute. The size attribute of a page node follows the size attribute of the device's display unit nodes. The current display page level attribute is used to control the visibility of each page level.

[0015] A single page node can contain zero, one, or more window nodes. A window node refers to a window within a page frame, such as the navigation window, fuel system window, or electronic flight instrument window in the MFD device navigation mode page's functional window hierarchy. Window nodes possess name, position, area size, scaling, and layout partition attributes. The area size attribute consists of content area size, placeholder area size, and operation area size. ， Center alignment of the three regions (See details) Figure 3 ) The content area size attribute describes the actual display size of the node. The operation area size describes the size of the node's operation response area, such as the operation range of a button on a touchscreen. The placeholder area size attribute describes the size of the node's footprint; overlapping placeholder areas of windows within the same page hierarchy is not recommended. The position attribute is the position of the top-left corner of the node's placeholder area within its parent node. The scaling attribute describes the scaling factor of the node along the X and Y axes. The layout partition attribute describes the page hierarchy where the window node resides and the layout partition within that page hierarchy; the top-left corner of the window node's placeholder area is aligned with the top-left corner of its corresponding layout partition.

[0016] The component (group) node refers to a small component in the window that has a certain function or meaning, such as the data block component on the left or the situation target component. A window node can have 0, 1, or more component nodes, and a component node can contain 1 or more nested component nodes. Multiple component nodes can be combined into a component group node. Component nodes have name, position attributes, area size attributes, and scaling attributes.

[0017] The primitive (group) nodes refer to the most basic 2D components in a 2D prototype, categorized into text primitives, vector primitives (including points, lines, polygons, circles, etc.), and image primitives. A window or component node can have one or more primitive nodes, and a component node can also have one or more primitive nodes. Multiple primitive nodes can form a primitive group node, and primitive (group) nodes can be upgraded to component nodes. Primitive nodes possess name, position, rotation, area size, and scaling attributes. Furthermore, text primitive nodes have text attributes, vector primitive nodes have border and fill attributes, and image primitive nodes have image attributes. Text attributes include text content and text style attributes such as color, alignment, font, size, and line spacing. Border attributes include line type, color, and width. Fill attributes include color and opacity. Image attributes include image path, width, height, and opacity.

[0018] The three-dimensional model node refers to the most basic 3D component element in the 2D prototype; one component node includes one or more three-dimensional model nodes; the three-dimensional model nodes in the 2D prototype have attributes such as mesh, material texture, spatial pose, lighting, and particle effects.

[0019] In one possible embodiment, the 3D prototype node framework includes one external view node and one aircraft module node; The external view node is used to describe the external scene of the aircraft and includes one landscape weather module node, one airport module node, one equipment node and multiple camera nodes. The landscape weather module node encapsulates 3D landscape, skybox, light source, and weather particle effects, possessing both landscape and weather attributes. The landscape attributes include scene type and external landscape data path attributes. Scene types include plains, mountains, islands, cities, and external landscapes. The external landscape refers to the 3D landscape automatically generated by reading landscape elevation data from external sources based on the external landscape data path attributes. The weather attributes include weather type, time, light intensity, shadows, and other attributes. The airport module node encapsulates a 3D airport model and has spatial attributes and airport type attributes. The spatial attributes include position, rotation, and scaling attributes in the X, Y, and Z axes. The airport type attributes include options such as plains, mountains, islands, and cities, allowing users to switch between different types of airport models. The equipment node is used to manage the 3D model nodes of ground or air platforms in the external view, such as radar vehicles and drones. Each equipment node contains 0, 1, or more 3D model nodes and 3D model group nodes. Each 3D model node is composed of 3D models and includes name attributes, spatial attributes, and 3D model attributes. The 3D model attributes include material, texture mapping, normal mapping, primary color, and collider attributes. Each 3D model group node is formed by grouping 3D model nodes and has name attributes and spatial attributes. The camera node is used to capture images from various angles in the 3D prototype, and includes name attributes, spatial attributes, and camera attributes. The camera attributes include field of view, projection method, camera label, and whether it is the main camera. The main camera is the viewpoint currently observed by the user. There can only be one main camera in the 3D prototype, and the main camera can be switched among multiple camera nodes through the camera label. The aircraft module node is used to describe the aircraft and its internal scene. It consists of one internal environment structure node and one cockpit module node, and has name and spatial attributes. The internal environment structure node is used to manage the aircraft's structure and lighting environment, and includes 0, 1 or more 3D model (group) nodes and light source nodes; the light source node includes attributes such as name, space, light source type, and light intensity; The cockpit module node is used to describe the cockpit and the cockpit interior scene. It consists of one interior environment structure node, one camera node, and zero, one, or more display and control device module nodes, and has name and spatial attributes. The display and control device module node is used to describe the display and control device, such as a multi-functional display, control panel, HOTAS stick, etc. It consists of 0, 1 or more 3D model nodes, device display unit nodes, and audio nodes, and has name and spatial attributes. The device display unit node in the 3D prototype is the same node as the device display unit node in the 2D prototype. The audio node includes attributes such as name, space, audio path, volume, and playback count, and has 3D sound effects.

[0020] In one possible embodiment, the project management tool includes a module for viewing and opening projects and a module for creating new projects, used for creating, viewing, and opening projects; The "View and Open Projects" module can display basic information about recently opened projects and case template projects in two ways: thumbnails and lists. It can also open the corresponding editing tools based on the project type. The basic information includes project name, thumbnail (thumbnail mode), type, size, creation time, etc. The new project function module can create 2D prototype projects, 3D prototype projects, and component library projects respectively; when creating a new project, the interface configuration file path, component library path, operation method, and deployment platform information can be configured; The interface configuration file path refers to the path of the internal interface configuration file output by the console tool, and the component library path refers to the path of the component library output by the component library editing tool, which can copy the selected internal interface configuration script and component library to the newly created project folder; the operation mode includes desktop mode and head-mounted display mode; the deployment platform refers to the platform deployed after the 2D prototype or 3D prototype is compiled, including platforms such as Win7, Win10, WingHui, and Kylin.

[0021] In one possible embodiment, the 2D prototype editing tool consists of a 2D structure tree module, a file management module, an attribute editing module, a template library module, a 2D component library module, a design draft module, a log checking module, a 2D canvas editing module, a logic control module, and a preview and compilation module, and is used to edit and output 2D prototypes. The 2D structure tree module manages the hierarchical structure of the 2D prototype node framework. It allows for the creation of various node types, merging of component or element nodes, upgrading element (group) nodes to component nodes, and performing operations such as copying, pasting, cutting, moving (drag and drop), and deleting nodes at each level. Different node types are distinguished by icons: component nodes (nodes added to the project from the component library window) are marked in red, and hidden nodes are grayed out. Nodes at each level can be searched by name. The visibility of each level and its sub-level nodes can be controlled via shortcut buttons on the structure tree. Prefabs for each level of nodes can be imported and exported.

[0022] The prefab encapsulates the attributes, 2D / 3D resources, logic, and other content of a node object in a 2D or 3D prototype, which can be reused between projects; once the prefab is added to a project, it becomes a node in the project's framework structure. The file management module can view the project's directory structure, open the file's path, and drag and drop files to the 2D attribute editing module and the 2D canvas editing module. The attribute editing function module is used to manage the attributes of nodes at each level in the 2D or 3D prototype node framework. It can add custom scripts, logic templates and logic blueprints to nodes at each level in the attribute window, and can view and edit public variables in custom scripts and logic templates in the attribute window. The template library module has functions for generating template prefabs and adding templates. The template prefabs are generated by the template library module. The function to generate template prefabs encapsulates a node object from a 2D or 3D prototype into a prefab and saves it in the template library folder under the tool's installation directory. Each project's template library module can read this template library folder. The function to add templates can display template thumbnails and names in categories and can add template prefabs as node objects in 2D or 3D prototypes. The 2D component library module can display thumbnails and names of component prefabs in categories, and add component prefabs to the 2D prototype project as component nodes. These component nodes maintain a reference relationship with the component prefabs. Added component nodes only allow modification of node attributes and control attributes such as name, position, text content, and icon path; style attributes such as text style, fill style, and size can only be viewed, not modified. The style attributes of component nodes need to be modified in the component library editing tool. After modification, re-importing the component prefab into the 2D prototype project will replace prefabs with the same name, and the style attributes of component nodes that have established a reference relationship with the prefab will be updated synchronously. The component prefabs are generated by the component library editing tool. The design draft function module can generate design draft prefabs based on the design draft; the design draft is generated by external design tools, such as Lunacy, Sketch, etc., and the nodes of each level of the 2D prototype and their style attributes are defined in the design draft according to the design draft naming convention; the design draft prefab is a prefab automatically generated after the 2D prototyping tool imports the design draft, and can be added to the 2D prototype or component library project.

[0023] The log inspection module includes log output and management functions, design draft parsing result output functions, hierarchical structure inspection functions, internal interface inspection functions, mesh alignment inspection functions, and placeholder area overlap inspection functions. The log output and management function can output log information during the tool's opening, running, and compilation processes, categorizing it into normal, warning, and error types, allowing selection of log output sources, and providing log classification display, stacking, searching, and saving functions. The design draft parsing result output function outputs the results of design draft import and parsing; if no mapping attribute value is found in the design draft, a warning log is output. The hierarchical structure inspection function checks for the existence of isolated elements and element groups (not under component nodes) under window nodes; if found, a warning log is output. The internal interface inspection function checks the consistency between the simulation control interface involved in the external prefab and the simulation control interface used in this project during prefab import; if inconsistent, an error log is output. The mesh alignment inspection function checks the alignment of reference lines, windows, components, element groups, elements, and meshes; if misaligned, a warning log is output. The overlap detection function can check the overlap of the placeholder areas between windows and windows, and between components at the same level. If there is an overlap, a warning log will be output.

[0024] The 2D canvas editing module includes canvas auxiliary editing and node editing functions. The canvas auxiliary editing functions include a ruler tool, a mesh tool, and a canvas right-click menu. The ruler tool displays X-axis and Y-axis rulers, with the origin aligned with the top-left corner of the canvas. As nodes move within the canvas, their horizontal and vertical positions are displayed in real-time on the ruler, with the top-left corner of the designated area as the origin. The mesh tool fills the canvas window with a mesh, displayed on top. The mesh display style is adjusted based on the mesh attributes of the page frame nodes, allowing control over the mesh's visibility. The top-left corner of any 2D node object in the canvas aligns with the top-left corner of the mesh. The canvas right-click menu displays the designated area of ​​the currently selected node object within a blue bounding box, and includes options such as copy, cut, delete, hide, group, move to top, move to bottom, group again, and ungroup. The node editing canvas includes a device display unit canvas, a page frame canvas, a page canvas, a window canvas, and a component (group) canvas. The device display unit canvas is opened by double-clicking the device display unit node in the structure tree. Its size matches the size of the device display unit, displaying multiple page layers overlaid. The page frame canvas is opened by double-clicking the page frame node in the structure tree. Its size matches the size of the device display unit, displaying the currently selected page layer and layout divisions. It provides canvas tools such as moving (reference lines), adding horizontal reference lines, and adding vertical reference lines. Page layers can be divided into layout sections by adding horizontal and vertical reference lines in the page frame canvas. The page canvas is opened by double-clicking the page frame node in the structure tree. Its size matches the size of the device display unit, displaying the layout divisions and content of each page layer in layers, and providing canvas tools for moving (windows, components). The window canvas is opened by double-clicking a window node in the structure tree. Its size is consistent with the window's placeholder area, displaying the content under that window node. It provides canvas tools such as moving (components / component groups, elements / element groups), adding lines, adding rectangles, adding triangles, adding circles, adding text, and adding images. The component (group) canvas is opened by double-clicking a component / component group node in the structure tree. Its size is consistent with the component / component group's placeholder area, displaying the content under that component / component group node. It also provides canvas tools such as moving (components (groups), elements (groups)), adding lines, adding rectangles, adding triangles, adding circles, adding text, adding images, and adding masks.

[0025] The logic control module includes custom script control and control functionality. The custom script control function allows for the editing of the internal and external logic of the rapid prototyping process through code programming. It can bind custom scripts edited in an external IDE to node objects, control changes in node attributes at various levels, and pre-configure commonly used custom scripts within the tool. The control functionality can encapsulate node objects into controls, such as buttons, progress bars, scale strips, and 2D maps.

[0026] The preview and compilation module includes prototype preview, executable file output, and multi-channel interaction. The prototype preview supports desktop mode preview, allowing each display unit node (one or more) in the display state to be output as a preview window. Users can select the scaling factor of the preview window (0.5, 0.72, 1.0, 1.5, and 2.0) and whether the preview window has a title bar. The executable file output function can output the corresponding version of the prototype executable file to a specified path based on the selected deployment platform (Win7, Win10, Wingtech, Kylin). The multi-channel interaction function allows interaction with the prototype executable file through desktop mode or head-mounted display mode. The desktop mode includes display and control channels such as a desktop monitor, HOTAS, and keyboard / mouse. It can display the prototype preview window on the screen, control the main camera's viewpoint via mouse and keyboard, and interact with the prototype through channels such as mouse, keyboard, HOTAS, and console. The head-mounted display mode features multiple display and control channels, including a helmet-mounted display, a desktop display, gesture control, eye tracking, Hotas, and keyboard / mouse. It allows viewing of 3D prototypes via the head-mounted display and supports interaction channels such as keyboard / mouse, gesture control, eye tracking, and Hotas. It can automatically calibrate its initial position and adjust the viewing angle through head tracking. 2D prototypes support desktop mode interaction, while 3D prototypes support both desktop and head-mounted display modes.

[0027] In one possible embodiment, the component library editing tool consists of a component library structure tree module, a file management module, an attribute editing module, a template library module, a component library export module, a design draft module, a log checking module, a 2D canvas editing module, a logic control module, and a preview and compilation module, and is used to edit and output the component library.

[0028] The component library structure tree function module has component library node management and automatic structure tree creation functions from design drafts. The component library node management function includes category label nodes and window, component (group), and element (group) nodes in the 2D prototype node framework. The automatic structure tree creation function from design drafts, after importing a design draft, can automatically add corresponding category labels, window, and component (group) nodes to the component library structure tree window according to the multi-level classification of components in the design draft.

[0029] The component library export function module can create a corresponding directory structure in the component library folder according to the hierarchical structure of the category tag nodes, and can encapsulate the selected window and component (group) nodes into component prefabs and output them to the corresponding component library folder directory.

[0030] In one possible embodiment, the 3D prototype editing tool consists of a 3D structure tree module, a file management module, an attribute editing module, a template library module, a 3D model editing module, a log checking module, a 3D scene editing module, a logic control module, and a preview and compilation module, and is used to edit and output 3D prototypes.

[0031] The 3D structure tree module is used to manage the hierarchical structure of the 3D prototype node framework and can import device display unit prefabs exported from 2D prototype editing tools.

[0032] The 3D model editing module includes functions such as model import, model lightweighting, adding colliders, automatic fitting, and interference checking. The model import function imports 3D models in STP and FBX (embedded texture mapping) formats. The model lightweighting function automatically removes redundant triangles while maintaining the model's appearance. The adding collider function automatically adds colliders to the 3D model to fit its geometry. The automatic fitting function ensures that selected faces of two 3D models automatically fit together, preventing clipping. The interference checking function checks for interference between 3D models, highlighting interfering areas.

[0033] The 3D scene editing module includes functions such as adjusting the viewpoint, adjusting the model, adjusting the light source, adjusting the camera, adjusting the collider, isolation mode, and main camera preview. Adjusting the viewpoint allows for free adjustment of the viewing angle, setting the viewpoint movement speed, and quick switching between front, top, and side views. Adjusting the model displays the coordinate axes and bounding box of the selected model node, allowing for quick adjustment of the selected node's position, rotation angle, size, and scaling factor. Adjusting the light source displays the light sources in the scene, allowing for quick adjustment of the selected light source's position and rotation angle. Adjusting the camera displays the scene's cameras, allowing for quick adjustment of the selected camera's position and rotation angle. Adjusting the collider displays the collider of the selected node, allowing for quick adjustment of the collider's position, rotation angle, size, and scaling factor. The isolation mode allows double-clicking a node in the 3D structure tree to enter that node's isolation mode, displaying only the content within that node and suppressing content outside the node. The main camera preview displays the scene within the current main camera's field of view.

[0034] In one possible embodiment, the console tool includes interface management and data communication functions. The interface management function manages the interface relationships between the 2D prototypes, 3D prototypes, and external simulation environments and interactive devices, including HOTAS joysticks, control buttons, VR / AR headsets, etc. The data communication function handles data forwarding and parsing between the 2D prototypes, 3D prototypes, and external simulation environments and interactive devices.

[0035] According to a second aspect of the present invention, a rapid prototyping method for aviation human-computer interaction interfaces is proposed, employing the aforementioned rapid prototyping system for aviation human-computer interaction interfaces, comprising the following steps: Edit and output the component library using the component library editing tool; Import the component library using a 2D prototyping tool; Import 2D prototype prefabs using 3D prototype editing tools, then edit and output 3D prototypes. Use console tools to forward data, preview changes between 2D and 3D prototypes, view 2D and 3D prototypes, and simulate aviation human-machine interface and mission simulation scenarios.

[0036] Advantages and beneficial effects of the present invention: 1) It can meet the requirements for structured description of aviation human-computer interaction prototypes. Based on the domain characteristics of aviation human-computer interaction interfaces and prototypes, a 2D prototype node framework composed of device display unit nodes, page frame nodes, page nodes, window nodes, component / component group nodes, graphic element / graphic element group nodes, and 3D model nodes, and a 3D prototype node framework composed of external view nodes and aircraft module nodes were established. It can describe aviation human-computer interaction prototypes in a structured way through hierarchical node structure and the attributes of each node. 2) It can meet the needs of rapid construction of aviation human-computer interaction prototypes. First, it provides a structure tree function module and an attribute editing function module, which can manage and edit the node hierarchy structure and attributes in a visual way; second, it provides a template library function module, which can improve the utilization efficiency of reusable nodes; third, the design draft function module can directly convert design drafts into node objects in the prototype project, reducing the workload of designers and the threshold for editing control logic; fourth, the 2D canvas editing function module and the 3D scene editing function module can preview the display of each node in the 2D prototype and 3D prototype in real time, and provide auxiliary editing tools such as rulers, grid alignment, view adjustment, and model adjustment, which improves the efficiency of prototype editing. 3) It can meet the 2D and 3D compatibility requirements of aviation human-computer interaction prototypes. First, both the 2D prototype node framework and the 3D prototype node framework have device display unit nodes. The device display unit nodes of the 2D prototype project can be imported into the 3D prototype project through prefab, thereby achieving 3D prototype compatibility with 2D prototypes. Second, a component node can have one or more 3D model nodes, thereby achieving 2D prototype compatibility with 3D display elements and supporting 3D DMI style aviation human-computer interaction prototype design. 4) It can meet the standardization requirements of aviation human-computer interaction prototype design. First, the hierarchical structure of 2D and 3D prototypes is described in a standardized node framework, ensuring structural standardization; second, a 2D component library functional module and component library editing tool are built. Added component nodes can only modify node properties and control properties, and style properties cannot be modified, ensuring style standardization. 5) It can meet the needs of multi-person collaborative construction of aviation human-computer interaction prototypes. First, the 2D and 3D structure tree functional modules provide prefab import and export functions, ensuring that the design results are reusable and transferable; second, the standardization of multi-person collaboration is ensured through structural standardization and style standardization. 7) It can meet the immersive interaction requirements of aviation human-computer interaction prototypes. First, in response to the future development trend of aviation human-computer interaction interfaces, the preview and compilation function module provides two preview modes: desktop mode and head-mounted display mode, and supports interaction channels such as mouse, keyboard, HOTAS, console, and helmet display; second, the console function can configure the interface relationship between 2D prototypes, 3D prototypes and external interaction devices, and forward multi-channel interface data.

[0037] 8) It can meet the driving requirements of the simulation environment for aviation human-computer interaction prototype tasks. The console tool provides interface management and data communication functions, which can configure the interface relationship between 2D prototypes, 3D prototypes and task-based external simulation environments, drive the display changes of 2D prototypes and 3D prototypes, and 2D prototypes and 3D prototypes can also control the external simulation environment. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of a rapid prototyping method for aviation human-computer interaction interfaces. Figure 2 This is a schematic diagram of a 2D prototype node framework; Figure 3This is a schematic diagram of the page frame; Figure 4 This is a schematic diagram of the 3D prototype node framework; Figure 5 This is a diagram illustrating the rapid prototyping process. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] The features and illustrative embodiments of various aspects of the present invention will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific setups and methods set forth below, but covers any improvements, substitutions, and modifications to structures, methods, and devices without departing from the spirit of the invention. Well-known structures and techniques are not shown in the drawings and the following description to avoid unnecessarily obscuring the invention.

[0042] In the description of this invention, it should be noted that the directions or positional relationships indicated by terms such as "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer" are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience of describing and simplifying the invention, and should not be construed as limiting the invention. Furthermore, the use of ordinal numbers (e.g., "first and second," etc.) is for distinguishing objects and is not limited to this order, and should not be construed as indicating or implying relative importance.

[0043] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly, encompassing both direct connection and indirect connection via an intermediate medium. Those skilled in the art can understand the specific meaning of these terms in this invention based on the specific circumstances.

[0044] It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other, and the various embodiments can be referenced and cited in each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0046] This invention implements a comprehensive cockpit ergonomics evaluation tool based on an indicator system. It consists of six parts: a data synchronization module, a comprehensive processing module, a test task module, an analysis results module, a comprehensive evaluation module, and a data management module. Residing at the test workstation, it collects test data and clock source signals through a time synchronization controller, completing the test task planning and organization, process data processing and analysis, and subjective and objective comprehensive evaluation of cockpit ergonomics. The following describes the composition of each module in this evaluation tool and its usage process.

[0047] The present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the described embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0048] Figure 1 This diagram illustrates the components of a rapid prototyping node framework and construction method for aviation human-computer interaction interfaces, consisting of the following parts: 101: Aviation Human-Computer Interaction Prototype Node Framework, used to describe the hierarchical structure and attributes of nodes in 2D and 3D prototypes, including 2D prototype node framework and 3D prototype node framework. 102: Rapid construction environment for aviation human-computer interaction prototypes, consisting of project management tools, component library editing tools, 2D prototype editing tools, 3D prototype editing tools and console tools, which can quickly build component libraries, 2D prototypes and 3D prototypes based on the node framework; 103: Project management tools, including modules for viewing and opening projects, and creating new projects, used to create, view, and open projects; 104: A 2D prototype editing tool, consisting of a 2D structure tree module, a file management module, an attribute editing module, a template library module, a 2D component library module, a design draft module, a log checking module, a 2D canvas editing module, a logic control module, and a preview and compilation module, used for editing and outputting 2D prototypes; 105: Component Library Editing Tool, consisting of a component library structure tree module, file management module, attribute editing module, template library module, component library export module, design draft module, log inspection module, 2D canvas editing module, logic control module, and preview and compilation module, is used to edit and output the component library; 106: A 3D prototype editing tool, consisting of a 3D structure tree module, a file management module, an attribute editing module, a template library module, a 3D model editing module, a log checking module, a 3D scene editing module, a logic control module, and a preview and compilation module, used for editing and outputting 3D prototypes; 107: Console tools, including interface management and data communication functions, are used to manage the interface relationships between 2D prototypes, 3D prototypes, and external simulation environments and external interactive devices, forward interface data, and drive changes in the display of 2D and 3D prototypes. External simulation environments include DCS, Xplane, AFSIM, etc., and external interactive devices include mice, keyboards, HOTAS, consoles, head-mounted displays, etc. 108: 2D prototype, used to simulate the human-machine interface software in cockpit display and control equipment. It is built in the 2D prototype project by the 2D prototype editing tool and includes display resources and their change logic at each level of the human-machine interface, such as the Canon G1000 main flight display 2D prototype and the Boeing 737 multi-function display 2D prototype. 109: The component library is a node resource library composed of component nodes. It is built in the component library project by the component library editing tool and output as component prefabs, which are stored in the component library folder. It can be added to 2D prototype projects, has the function of referencing style attributes, and restricts the editing permissions of style attributes, such as self-repeating button components, TAB components, drop-down list components, label components, etc. 110: 3D Prototype, used to simulate three-dimensional cockpit environments and mission scenarios, such as Boeing 737 3D Prototype, F35 3D Prototype, etc. It is built in the 3D Prototype project by the 3D Prototype editing tool, and 2D prototypes can be imported into 3D Prototype.

[0049] Figure 2 This is a schematic diagram of a 2D prototype node framework, consisting of the following parts: 201: 2D prototype node framework, from top to bottom: device display unit node, page frame node, page node, window node, component (group) node, graphic element (group) node, and 3D model node; 202: Equipment display unit node, which is a display screen in the cockpit display and control equipment, such as MFD equipment display unit, HUD equipment display unit, HMD equipment display unit; 203: Page frame node, which refers to the page hierarchy and layout partitions within a display page based on interactive behavior and functional characteristics, such as... Figure 3 As shown; 204: Page node, which is the display page of a certain display screen in a certain mode in the cockpit display and control equipment, such as the HUD navigation mode page, the HUD air mode page, etc.; 205: Window node, which is a window within a certain page frame, such as the navigation window, fuel system window, electronic flight instrument window, etc. in the MFD device navigation mode page function window hierarchy; 206: Component (group) nodes refer to small components in the window that have a certain function or meaning, such as the data block component on the left, the situation target component, etc. 207: Primitive (group) nodes are the most basic 2D components in a 2D prototype, and are divided into text primitives, vector primitives (including vector graphics such as points, lines, polygons, and circles), and image primitives; 208: 3D model node, the most basic 3D component in a 2D prototype; Figure 3 This is a schematic diagram of the page framework, divided into four page layers: background layer, content layer, global control layer, and temporary layer. The background layer is divided into area R1, the content layer into areas R1 and R2, the global control layer into areas R1 and R2, and the temporary layer into areas R1, R2, R3, R4, R5, R6, R7, R8, and R9. Window nodes possess the attributes of their respective layout partitions. By selecting the page layer and region of the window, the display layer and position of the window are controlled. The top-left corner of the window node's placeholder area is aligned with the top-left corner of its respective layout partition; for example, the top-left corner of the window node's placeholder area is aligned with the top-left corner of the temporary layer's R5 area.

[0050] Figure 4 This is a schematic diagram of a 3D prototype node framework, consisting of the following parts: 401: 3D prototype node, consisting of one external view node and one aircraft module node; 402: External view node, used to describe the external scene of the aircraft, including 1 landscape weather module node, 1 airport module node, 1 equipment node and multiple camera nodes; 403: Aircraft module node, used to describe the aircraft and its internal scene, consisting of 1 internal environment structure node and 1 cockpit module node; 404: Landscape Weather Module Node, which encapsulates 3D landscape, skybox, light source and weather particle effects, and has landscape attributes and weather attributes; 405: Airport module node, which encapsulates the 3D model of the airport and has spatial attributes and airport type attributes; 406: Equipment nodes are used to manage 3D model nodes of ground or air platforms in the external view, such as radar vehicles and drones. An equipment node contains 0, 1, or more 3D model nodes and 3D model group nodes. 407: A 3D model (group) node is composed of 3D models and includes name attributes, spatial attributes, and 3D model attributes. A 3D model group node is created by grouping 3D model nodes. 408: Camera node, used to capture images from various angles in the 3D prototype, including name attributes, spatial attributes, and camera attributes; 409: Cockpit Module Node, used to describe the cockpit and its interior scene. It consists of 1 interior environment structure node, 1 camera node, and 0, 1, or more display and control device module nodes, and has name and spatial attributes. 410: Display and control device module node, used to describe display and control devices, such as multi-function displays, control panels, HOTAS sticks, etc. It consists of 0, 1 or more 3D model nodes, device display unit nodes, and audio nodes, and has name and spatial attributes; 411: Device display unit node, which is the same node as the device display unit node in the 2D prototype, achieving 2D and 3D compatibility; 412: Audio node, containing attributes such as name, space, audio path, volume, number of plays, etc., and has 3D sound effects; 413: Light source node, containing attributes such as name, space, light source type, and light intensity; 414: Internal environment structure node, used to manage the aircraft's structure and lighting environment, containing 0, 1 or more 3D model (group) nodes and light source nodes; Figure 5 This is a diagram illustrating the rapid prototyping process, with the following specific steps: 501: Editing and exporting the component library using the component library editing tool. Create a component library project using the project management tool, import the Luancy design draft file containing control styles such as buttons, tabs, and dropdown lists using the design draft function module, and automatically restore each control and its structure in the design draft file in the component library structure tree function module after parsing. Edit the content and properties of the components in the 2D canvas editing function module, attribute editing function module, template library function module, etc. After editing, export the component library using the component library export function module; 502: Import component library using 2D prototype editing tool. Create a 2D prototype project using project management tool, import the component library output by component library editing tool, build the node framework of 2D prototype based on 2D structure tree function module, 2D component library function module, design draft function module, and template library function module, edit the attributes of each node using 2D canvas editing function module and attribute editing function module, edit the interaction logic of 2D prototype according to the internal interface defined by console tool using logic control function module, after editing, output 2D prototype executable file using preview and compile function module, and output device display unit prefab using 2D structure tree function module; 503: Import 2D prototype prefabs using a 3D prototype editing tool, and edit and output the 3D prototype. Create a 3D prototype project using a project management tool, import the device display unit prefab exported from the 2D prototype editing tool, build the node framework of the 3D prototype based on the 3D structure tree function module and template library function module, edit the attributes of each node using the 3D scene editing function module, 3D model editing function module, and attribute editing function module, and edit the interaction logic of the 3D prototype according to the internal interface defined by the console tool using the logic control function module. After editing, output the 3D prototype executable file using the preview and compile function module, and output the device display unit prefab using the 3D structure tree function module. 504: Use the console to forward data and preview changes to the 2D and 3D prototypes. Open the external simulation environment for the edited task, connect external interactive devices such as mouse, keyboard, HOTAS, console, and head-mounted display, run the 2D and 3D prototype executable files, and view the 2D and 3D prototypes in desktop mode or head-mounted display mode to simulate the aviation human-machine interface and mission simulation scenario.

[0051] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.

Claims

1. A rapid prototyping system for aviation human-computer interaction interfaces, characterized in that, include: Aviation human-computer interaction prototype node framework, aviation human-computer interaction prototype rapid construction environment; The aviation human-computer interaction prototype node framework is used to describe the hierarchical structure and attributes of nodes in 2D and 3D prototypes, including a 2D prototype node framework and a 3D prototype node framework. Device display unit nodes in the 2D prototype node framework can be imported into the 3D prototype node framework. The rapid prototyping environment for aviation human-computer interaction includes project management tools, component library editing tools, 2D prototype editing tools, 3D prototype editing tools, and console tools, which can quickly build component libraries, 2D prototypes, and 3D prototypes based on the aviation human-computer interaction prototype node framework.

2. The rapid prototyping system for aviation human-computer interaction interfaces according to claim 1, characterized in that, The 2D prototype node framework includes device display unit nodes, page frame nodes, page nodes, window nodes, component nodes, primitive nodes, and 3D model nodes. A device display unit node refers to a display screen in the cockpit display and control equipment, and it possesses name, effective display area, and size attributes. A page node refers to the display page of a display screen in the cockpit display and control equipment in a specific mode. A window node refers to a window within a page frame. A component node refers to a small component within a window that has a specific function or meaning. A primitive node refers to the most basic 2D component in the 2D prototype. A 3D model node refers to the most basic 3D component in the 2D prototype.

3. The rapid prototyping system for aviation human-computer interaction interfaces according to claim 1, characterized in that, The 3D prototype node framework includes one external view node and one aircraft module node. The external view node describes the aircraft's external scene and includes one terrain / weather module node, one airport module node, one equipment node, and multiple camera nodes. The terrain / weather module node encapsulates 3D terrain, skybox, light sources, and weather particle effects, possessing terrain and weather attributes. The airport module node encapsulates a 3D airport model, possessing spatial and airport type attributes. The equipment node manages the 3D model nodes of ground or air platforms in the external view. The camera nodes capture images from various angles in the 3D prototype and include name, spatial, and camera attributes. The aircraft module node describes the aircraft and its internal scene, consisting of one internal environment structure node and one cockpit module node, possessing name and spatial attributes. The internal environment structure node manages the aircraft's structure and lighting environment. The cockpit module node describes the cockpit and its internal scene. The display and control equipment module node describes the display and control equipment.

4. The rapid prototyping system for aviation human-computer interaction interfaces according to claim 1, characterized in that, The project management tool includes modules for viewing and opening projects, and modules for creating new projects, which are used to create, view, and open projects.

5. A rapid prototyping system for aviation human-computer interaction interfaces according to claim 1, characterized in that, The 2D prototype editing tool consists of a 2D structure tree module, a file management module, an attribute editing module, a template library module, a 2D component library module, a design draft module, a log checking module, a 2D canvas editing module, a logic control module, and a preview and compilation module, and is used to edit and output 2D prototypes.

6. A rapid prototyping system for aviation human-computer interaction interfaces according to claim 1, characterized in that, The component library editing tool consists of a component library structure tree module, a file management module, an attribute editing module, a template library module, a component library export module, a design draft module, a log checking module, a 2D canvas editing module, a logic control module, and a preview and compilation module, and is used to edit and output the component library.

7. A rapid prototyping system for aviation human-computer interaction interfaces according to claim 1, characterized in that, The 3D prototype editing tool consists of a 3D structure tree module, a file management module, an attribute editing module, a template library module, a 3D model editing module, a log checking module, a 3D scene editing module, a logic control module, and a preview and compilation module, and is used to edit and output 3D prototypes.

8. A rapid prototyping system for aviation human-computer interaction interfaces according to claim 1, characterized in that, The console tool includes interface management and data communication functions; the interface management function is used to manage the interface relationships between 2D prototypes, 3D prototypes, and external simulation environments and external interactive devices. The data communication function is used for data forwarding and parsing between 2D prototypes, 3D prototypes and external simulation environments and external interactive devices.

9. A rapid prototyping method for aviation human-computer interaction interfaces, characterized in that, A rapid prototyping system for aviation human-computer interaction interfaces, as described in any one of claims 1-8, includes the following steps: Edit and output the component library using the component library editing tool; Import the component library using a 2D prototyping tool; Import 2D prototype prefabs using 3D prototype editing tools, then edit and output 3D prototypes. Use console tools to forward data, preview changes between 2D and 3D prototypes, view 2D and 3D prototypes, and simulate aviation human-machine interface and mission simulation scenarios.

Images