Automatic piloting aircraft cockpit interactive interface optimization method based on VR technology

Abstract

The invention discloses an automatic piloting aircraft cockpit interactive interface optimization method based on a VR technology, and relates to the field of man-machine interaction, the method comprises the following steps: designing a self-response man-machine interaction interface based on a comprehensive audio-visual technology, and integrating the self-response man-machine interaction interface to an aircraft cockpit main screen; collecting aircraft motion parameters by utilizing a human-computer interaction interface, acquiring a flight state change result according to the aircraft motion parameters, generating a state interaction instruction based on the flight state change result, and loading the state interaction instruction to a main screen of an aircraft cockpit; creating an aircraft simulator to obtain a simulated flight video, inputting the simulated flight video to the human-computer interaction interface, and combining the simulated flight video with the flight state change result to generate a flight interaction instruction; and building a man-machine interaction test platform to carry out flight simulation experience, and determining the optimization recognition degree of a man-machine interaction interface according to the flight simulation experience. According to the method, a pilot can conveniently and visually know the change direction, strength and duration of flight state parameters such as overload and angular acceleration.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction, and in particular to a method for optimizing an interactive interface in a cockpit of an autonomous aircraft based on VR technology. Background Art

[0002] The cockpit interaction of an autonomous aircraft refers to the communication and operation method between the pilot and the autonomous driving system, which is usually combined with modern electronic display and control systems to enhance the convenience and safety of operation. The cockpit interaction system of an autonomous aircraft not only includes traditional physical control devices (such as joysticks, throttle levers, etc.), but also integrates advanced human-machine interface technologies (such as touch screens, voice control, head-up displays, etc.), allowing pilots to monitor the flight status and make adjustments without directly controlling the aircraft.

[0003] Information related to autopilot and flight status are usually presented in real time through a multi-function display (MFD) or head-up display (HUD), including current heading, speed, attitude, planned route, weather conditions, etc. The pilot can monitor the flight progress intuitively. During the operation of the autopilot system, the interactive system will issue alarms or suggestions based on the flight status and environmental conditions, reminding the pilot that intervention may be required to ensure flight safety.

[0004] However, the existing human-computer interaction interface is unable to collect aircraft motion parameters and predict changes in flight status parameters in the future. It is also unable to hide the human-computer interaction interface when the overload or angular acceleration is within the normal range, which reduces the pilot's normal perception of the aircraft motion parameters.

[0005] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0006] In response to the problems in the related technology, the present invention proposes an optimization method for the cockpit interactive interface of an autonomous aircraft based on VR technology to overcome the above-mentioned technical problems existing in the existing related technology.

[0007] To this end, the specific technical solution adopted by the present invention is as follows:

[0008] The optimization method of the cockpit interactive interface of an autonomous aircraft based on VR technology includes:

[0009] Design a self-responsive human-computer interaction interface based on integrated audio-visual technology and VR technology, and integrate the self-responsive human-computer interaction interface into the main screen of the aircraft cockpit;

[0010] Using a human-computer interaction interface to collect aircraft motion parameters, obtaining flight state change results based on the aircraft motion parameters, generating state interaction instructions based on the flight state change results, and loading the state interaction instructions onto the main screen of the aircraft cockpit;

[0011] Create an aircraft simulator based on VR technology to obtain simulated flight video, input the simulated flight video into the human-computer interaction interface and combine it with the flight state change results to generate flight interaction instructions, and load the flight interaction instructions into the main screen of the aircraft cockpit;

[0012] Build a human-computer interaction test platform for flight simulation experience, analyze the evaluation of the human-computer interaction interface based on the flight simulation experience, and determine the optimization recognition of the human-computer interaction interface based on the evaluation.

[0013] Preferably, collecting aircraft motion parameters by using a human-computer interaction interface, obtaining flight state change results according to the aircraft motion parameters, generating state interaction instructions based on the flight state change results, and loading the state interaction instructions to the main screen of the aircraft cockpit includes:

[0014] Use the human-computer interaction interface to obtain the vertical acceleration of the autopilot acting on the fuselage and the pilot during the historical maneuvers, and predict the overload change trend based on the vertical acceleration;

[0015] The human-computer interaction interface is used to obtain the angular accelerations of the autopilot when there are lateral roll and longitudinal roll trends during the historical maneuvers.

[0016] Based on the overload change trend and angular acceleration, overload state interaction instructions and speed state interaction instructions are generated accordingly according to instruction generation rules;

[0017] The visual reaction time and auditory reaction time are obtained according to the historical human-computer interaction interface, the prompt time of the status interaction command is determined, and the status interaction command is loaded to the main screen of the aircraft cockpit according to the prompt time.

[0018] Preferably, based on the overload change trend and the angular acceleration, generating the overload state interaction instruction and the speed state interaction instruction correspondingly according to the instruction generation rule includes:

[0019] The overload change trend is divided into several overload intervals according to the overload value range that can be indicated, and the positive load visual prompt instruction and the positive load auditory prompt instruction are determined according to the overload interval division result;

[0020] Determine a display frame corresponding to the overload change trend based on the overload interval division result, and determine a frame prompt instruction according to the display frame;

[0021] Dividing the angular acceleration into a number of speed intervals according to the angular acceleration value range, determining the speed visual prompt instruction according to the speed interval division result, and defining the speed shape prompt instruction based on the speed interval;

[0022] The positive load visual prompt instruction, the positive load auditory prompt instruction and the frame prompt instruction are used as overload interaction instructions, and the speed visual prompt instruction and the speed shape prompt instruction are used as speed state interaction instructions.

[0023] Preferably, the overload value range may be -5.0G to 9.9G, and the overload interval includes a normal positive overload interval, a positive overload transition interval, a dangerous positive overload interval, a normal negative overload interval, and a dangerous negative overload interval.

[0024] Preferably, the speed interval includes a normal angular acceleration, an angular acceleration transition interval, a limit angular acceleration interval and a dangerous angular acceleration interval.

[0025] Preferably, creating an aircraft simulator based on VR technology to obtain a simulated flight video, inputting the simulated flight video into a human-computer interaction interface and combining it with a flight state change result to generate a flight interaction instruction, and loading the flight interaction instruction into the main screen of the aircraft cockpit includes:

[0026] Based on VR technology and modeling technology, VR flight scenes and aircraft models are constructed as aircraft simulators, and flight display animations are produced in conjunction with interactive platforms;

[0027] Write a flight route program according to the preset flight attitude, obtain simulated flight video by combining the aircraft simulator and flight display animation, and input the simulated flight video into the human-computer interaction interface;

[0028] The flight status change results are combined with the simulated flight video to configure the flight voice prompt, and the flight voice prompt is loaded as a flight interaction instruction to the main screen of the aircraft cockpit.

[0029] Preferably, constructing a VR flight scene and an aircraft model as an aircraft simulator based on VR technology and modeling technology, and producing a flight display animation in combination with an interactive platform includes:

[0030] Use modeling technology to draw a pre-defined two-dimensional map of the flight scene, set the corresponding texture information according to the flight scene, and render the flight scene based on the interactive platform and texture setting results;

[0031] Use the interactive platform to perform terrain mapping operations on the rendered flight scene, and perform secondary rendering and repair operations on the terrain based on the mapping results, and obtain the VR flight scene based on the repair results;

[0032] Use modeling technology to draw the weather scene corresponding to the VR flight scene, and import the weather scene into the interactive platform to generate the external flight scene of the aircraft;

[0033] Modeling technology is used to draw a two-dimensional diagram corresponding to the aircraft and the cockpit, and the aircraft cockpit model and flight display animation are generated in combination with the interactive platform.

[0034] Preferably, using modeling technology to draw a two-dimensional picture corresponding to the aircraft and the cockpit, and combining the interactive platform to generate the aircraft cockpit model and flight display animation includes:

[0035] Using modeling technology to draw two-dimensional images corresponding to the aircraft and the cockpit, respectively, set the textures of the aircraft and the cockpit, and after the setting is completed, import the two-dimensional images of the aircraft and the cockpit into the interactive platform;

[0036] According to the pre-configured texture materials, the aircraft and the cockpit are respectively textured in the interactive platform, and the position parameters of the cockpit are set according to the texture results to obtain the aircraft cockpit model;

[0037] A new animation clip is created based on the interactive platform, and the aircraft cockpit model is used as an animation object to be produced in the animation clip to obtain a flight display animation.

[0038] Preferably, building a human-computer interaction test platform for flight simulation experience, analyzing the evaluation of the human-computer interaction interface according to the flight simulation experience, and determining the optimization recognition of the human-computer interaction interface based on the evaluation includes:

[0039] Build the human-computer interaction test environment and flight scenarios corresponding to the self-responsive human-computer interaction interface and the historical human-computer interaction interface respectively, and generate simulated flight videos for flight simulation experience;

[0040] The flight simulation experience results are combined with the subjective evaluation table of the human-computer interaction interface pre-built using cognitive load technology to obtain the corresponding evaluation values ​​of the self-responsive human-computer interaction interface and the historical human-computer interaction interface;

[0041] The reliability of the evaluation value is verified by using Cronbach's coefficient and validity testing technology, and the optimization satisfaction of the self-responsive human-computer interaction interface is determined based on the reliability verification results and the evaluation value.

[0042] Preferably, the reliability of the evaluation value is verified by using Cronbach's coefficient and validity testing technology, and the optimization satisfaction of the self-responsive human-computer interaction interface is determined based on the reliability verification result and the evaluation value, including:

[0043] The subjective evaluation form of human-computer interaction interface was standardized by using data analysis technology, and the reliability value was obtained by performing reliability analysis on the standardized subjective evaluation form of human-computer interaction interface using Cronbach's coefficient.

[0044] The difference between the reliability value and the reliability threshold is used to verify the reliability coefficient of the evaluation value, and the validity of the subjective evaluation form of the human-computer interaction interface is analyzed based on the Bartlett test technique and factor analysis technique;

[0045] The difference between the judgment validity and the validity threshold is used to verify the validity coefficient of the subjective evaluation form of the human-computer interaction interface, and the reliability of the evaluation value is determined based on the reliability coefficient and the validity coefficient;

[0046] The reliability and evaluation value are combined to measure the satisfaction of the self-responsive human-computer interaction interface and the historical human-computer interaction interface, and the optimization satisfaction of the self-responsive human-computer interaction interface is judged based on the satisfaction.

[0047] The beneficial effects of the present invention are:

[0048] 1. The present invention sets a self-responsive human-computer interaction interface design by integrating visual technology and audio prompt technology, so that pilots can have an intuitive understanding of the direction, intensity and duration of changes in flight state parameters such as overload and angular acceleration, and build flight scenes and aircraft models based on VR technology to realize the human-computer interaction interface of autonomous air combat aircraft. At the same time, a subjective evaluation questionnaire for the interaction interface is proposed to obtain the subjective evaluation of the interaction interface information, and the effect of the human-computer interaction interface of the autonomous air combat aircraft is verified according to the subjective evaluation results.

[0049] 2. The human-machine interaction interface proposed in the present invention continuously collects aircraft motion parameters and predicts changes in flight status parameters in the future. When the overload or angular acceleration is within the normal range and has no tendency to exceed the tolerance limit, the human-machine interaction interface is hidden. At this time, the main screen is no different from normal flight, and the human-machine interaction interface will not affect the pilot's normal perception of the aircraft motion parameters. When it is predicted that the overload or angular acceleration will exceed the overload limit, the human-machine interaction interface will automatically pop up and appear on the main screen to prompt the pilot of the aircraft motion status, so that the pilot can make corresponding operations or take countermeasures. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 It is a flowchart of a method for optimizing an interactive interface in an autonomous aircraft cockpit based on VR technology according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments. They can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. By referring to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention.

[0053] According to an embodiment of the present invention, a method for optimizing the interactive interface of an autonomous aircraft cockpit based on VR technology is provided.

[0054] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to an optimization method for an autonomous aircraft cockpit interactive interface based on VR technology according to an embodiment of the present invention, the optimization method includes:

[0055] Step S1: design a self-responsive human-computer interaction interface based on comprehensive audio-visual technology and VR technology, and integrate the self-responsive human-computer interaction interface into the main screen of the aircraft cockpit.

[0056] It should be explained that in order to prevent the human-machine interface from interfering with the pilot's normal perception of the aircraft's motion parameters, this embodiment adopts a pop-up prompt interface design (self-responsive human-machine interface), that is, the human-machine interface continuously monitors the aircraft's motion state. When the aircraft's motion state is normal, the main display interface hides the human-machine interface. When the aircraft's motion state tends to exceed the tolerance limit, the human-machine interface will automatically pop up to the main interface to remind the pilot to pay attention to the aircraft's motion trend. This can minimize the interference of the human-machine interface on the pilot, while providing prompts for aircraft instructions at critical moments.

[0057] The specific human-computer interaction interface continuously collects aircraft motion parameters and predicts changes in flight status parameters in the future. When the overload or angular acceleration is within the normal range and there is no trend of exceeding the tolerance limit, the human-computer interaction interface is hidden. At this time, the main screen is no different from normal flight, and the human-computer interaction interface will not affect the pilot's normal perception of the aircraft motion parameters; when it is predicted that the overload or angular acceleration will exceed the overload limit, the human-computer interaction interface will automatically pop up and appear on the main screen to prompt the pilot of the aircraft's motion status, so that the pilot can make corresponding operations or take countermeasures.

[0058] Step S2: collect aircraft motion parameters using a human-computer interaction interface, obtain flight state change results based on the aircraft motion parameters, generate state interaction instructions based on the flight state change results, and load the state interaction instructions to the aircraft cockpit main screen.

[0059] In this embodiment, when using the human-computer interaction interface to collect aircraft motion parameters, obtaining flight state change results based on the aircraft motion parameters, generating state interaction instructions based on the flight state change results, and loading the state interaction instructions to the aircraft cockpit main screen, the human-computer interaction interface can be used to obtain the vertical acceleration of the autopilot acting on the fuselage and the pilot during historical maneuvers, and predict the overload change trend based on the vertical acceleration; the human-computer interaction interface is used to obtain the angular acceleration of the autopilot when the lateral axis rolling trend and the longitudinal axis rolling trend respectively appear during historical maneuvers; based on the overload change trend and the angular acceleration, overload state interaction instructions and speed state interaction instructions are correspondingly generated according to the instruction generation rules; the visual reaction time and the auditory reaction time are obtained according to the historical human-computer interaction interface, the prompt time of the state interaction instruction is determined, and the state interaction instruction is loaded to the aircraft cockpit main screen according to the prompt time.

[0060] In this embodiment, when the overload state interaction instruction and the speed state interaction instruction are generated correspondingly according to the instruction generation rules based on the overload change trend and the angular acceleration, the overload change trend can be divided into several overload intervals according to the range of overload values ​​that can be prompted, and the positive load visual prompt instruction and the positive load auditory prompt instruction are determined according to the overload interval division result; the corresponding display frame of the overload change trend is determined based on the overload interval division result, and the frame prompt instruction is determined according to the display frame; the angular acceleration is divided into several speed intervals according to the angular acceleration value range, the speed visual prompt instruction is determined according to the speed interval division result, and the speed shape prompt instruction is defined based on the speed interval; the positive load visual prompt instruction, the positive load auditory prompt instruction and the frame prompt instruction are used as overload interaction instructions, and the speed visual prompt instruction and the speed shape prompt instruction are used as speed state interaction instructions.

[0061] It needs to be explained that the aircraft overload indication interface is an optimized design based on the HUD (Head-Up Display) interface. Taking into account the complexity of the HUD interface itself and the tolerance characteristics of longitudinal overload and lateral overload, the longitudinal overload and lateral overload peaks during maneuvering are very small compared to the pilot's tolerance and there is no need for special prompts. Therefore, the overload indication interface does not give special prompts for longitudinal overload and lateral overload. However, the positive overload peak during maneuvering can exceed the pilot's tolerance limit. In this case, the pilot needs to be informed in advance of the overload changes of the autonomous air combat aircraft.

[0062] Use numbers for positive overload prompts. The range of positive overload values ​​that can be prompted is between -5.0G and 9.9G, with 1 decimal place, in G. The unit is not displayed. The positive overload number is preceded by "+" to represent positive overload and "-" to represent negative overload. For example, "+4.5" represents the predicted overload value of 4.5G. The positive and negative overload prompts are located in the upper left position of the HUD.

[0063] Specifically, different colors can be used to distinguish different positive overload levels. For positive overload, in order to minimize interference with the pilot's mission, no prompt is given when the future positive overload value is less than 4.0G; when the future positive overload value is greater than or equal to 4.0G and less than 7.0G, the digital symbol indicating the positive overload is yellow; when the future positive overload value is greater than or equal to 7.0G and less than 10.0G, that is, the positive overload value is close to the limit overload value, the digital symbol indicating the positive overload is red, accompanied by a voice prompt "High overload! High overload! High overload!"; when the negative overload is between -2.0G and -0.1G, the digital symbol indicating the positive overload is yellow, and when the negative overload is between -4.9 and -2.1G, the digital symbol indicating the positive overload is red, accompanied by a voice prompt "Negative overload! Negative overload! Negative overload!". In summary, the pre-prompt scheme for using different colors to divide different positive overload levels is shown in Table 1.

[0064] Table 1 Advance warning schemes for different overload ranges

[0065]

[0066]

[0067] At the same time, different shapes of boxes can be used to distinguish different positive overload levels. The positive overload levels can be divided based on boxes of different shapes as a backup plan for the plan of dividing the positive overload levels with different colors. In the backup plan, in order to minimize interference with the pilot's mission, when the future positive overload value is less than 4.0G, no prompt or special emphasis will be given; when the future positive overload value is greater than or equal to 4.0G and less than 7.0G, the digital symbol indicating the positive overload is highlighted with an arrow box, and the outside of the digital symbol indicating the positive overload is highlighted with an arrow box, which conveys to the pilot that the overload is approaching the overload limit. Compared with the vertical and horizontal line elements that are more common in the HUD interface, the diagonal line in the arrow box makes it easier for the pilot to Note that when the future positive overload value is greater than or equal to 7.0G and less than 10.0G, that is, when the positive overload value is close to the limit, the digital symbol indicating the positive overload is highlighted with an elliptical frame. Compared with the common straight line elements in the HUD, the curved elements are more likely to attract the pilot's attention. Therefore, an elliptical frame is used to emphasize the situation where the positive overload exceeds the limit, accompanied by a voice prompt of "High overload! High overload! High overload!"; when the negative overload is between -2.0G and -0.1G, the digital symbol indicating the positive overload is highlighted with an arrow frame outside. When the negative overload is between -4.9 and -2.1G, the digital symbol indicating the positive overload is highlighted with an elliptical frame, accompanied by a voice prompt of "Negative overload! Negative overload! Negative overload!".

[0068] Specifically, the human body is not sensitive to angular velocity. When the human body enters a constant angular velocity from zero angular velocity, it will only feel the body turning when the angular velocity changes. Eventually, the human body will adapt to the angular velocity and lose the sense of rotation. At the same time, the human body will produce a vestibular reaction to a large angular acceleration, affecting flight safety. In the prior art, almost no aircraft provides a prompt interface for pilots about maneuvering angular acceleration information. According to existing standards, the angular acceleration of the aircraft around each axis of the body axis should not exceed 2rad / s 2 , that is 114.6deg / s 2 .

[0069] Specifically, intuitive arrows can be used to prompt angular acceleration. The roll angular acceleration is along the x-axis direction of the aircraft body, which makes the aircraft tend to roll. A curved arrow is used for prompting. The curved arrow for right roll is at the lower left of the HUD interface, and the curved arrow for left roll is at the lower right of the HUD interface. The pitch angular acceleration is along the y-axis direction of the aircraft body, which makes the aircraft tend to look up and down, and causes altitude changes. Therefore, upward and downward straight arrows are used to prompt the upward pitch acceleration and downward pitch acceleration respectively. The upward straight arrow is above the HUD interface, and the downward straight arrow is below the HUD interface; the yaw angular acceleration is along the z-axis direction of the aircraft body, which makes the aircraft tend to change heading. A curved arrow is used for prompting. The curved arrow for right yaw is at the middle-right position of the HUD interface, and the curved arrow for left yaw is at the middle-left position of the HUD interface.

[0070] At the same time, different colors can be used to distinguish different angular acceleration levels. In order to minimize interference with the pilot's mission, when the absolute value of the angular acceleration is less than 0.5 rad / s 2 When the absolute value of the angular acceleration is greater than or equal to 0.5rad / s 2 And less than 1rad / s 2 When the arrow is green, the absolute value of the angular acceleration is greater than or equal to 1 rad / s 2 And less than 2rad / s 2 The arrow turns yellow when the angular acceleration is greater than or equal to 2 rad / s 2 When , the arrow turns red, as shown in Table 2.

[0071] Table 2 Advance prompt schemes for different angular acceleration ranges

[0072] Angular acceleration range (unit: rad / s) Visual cues Normal angular acceleration range (-0.5, 0.5) No prompt Angular acceleration transition interval (-1, -0.5] and [0.5, 1) Green arrow tips Limiting angular acceleration range (-2, -1] and [1, 2) Yellow arrow tips Dangerous angular acceleration range (-∞, -2] and [2, ∞) Red arrow tips

[0073] At the same time, this embodiment additionally designs the use of shape differences to divide different angular acceleration levels as a backup solution for the solution of using different colors to divide angular acceleration levels. In the backup solution, in order to minimize the interference with the pilot's mission, when the angular acceleration value is less than 1 rad / s 2When the angular acceleration value is greater than or equal to 1rad / s 2 And less than 2rad / s 2 When the angular acceleration value is greater than or equal to 2rad / s 2 When the arrow turns solid green, that is, when there is a tendency to exceed the limit, the angular acceleration in the backup plan is less than 1 rad / s 2 No special prompt is given anymore, and the fallback option can be used when the color setting of the symbol cannot be changed in the HUD interface.

[0074] It should be explained that it takes a certain amount of time from the start of information presentation to the pilot's understanding of the information content. If the information prompt is too short, it will not be conducive to the pilot's correct understanding of the maneuver instruction. The interaction interface of g-load and angular acceleration involves visual information and auditory information, so it is necessary to determine the appropriate advance prompt time based on the visual reaction time data and the auditory reaction time data.

[0075] (1) Visual reaction time refers to the time from the presentation of visual information to the pilot's response, among which the visual reaction time of numbers>the visual reaction time of colors>the visual reaction time of graphics. Therefore, when a visual information stimulus is presented to the subjects, the subjects first complete the recognition of graphics, then the recognition of colors, and finally the recognition of numbers. For overload indication, it involves the recognition of numbers and colors. To ensure that the pilot can accurately recognize all overload prompt information, the overload prompt information should be continuously displayed for at least 3.2 seconds, and the prompt should be continuously displayed until the overload enters the safe range that the pilot can tolerate within the predicted time. For angular acceleration prompt information, it involves the recognition of graphics and colors. To ensure that the pilot can accurately recognize all angular acceleration prompt information, the angular acceleration prompt information should be continuously displayed for at least 0.9 seconds, and the prompt should be continuously displayed until the angular acceleration enters the safe range that the pilot can tolerate within the predicted time.

[0076] (2) Auditory reaction time refers to the time from the audio information being played to the pilot making a corresponding response. In the overload indication method, when the maneuver is about to exceed the Gz overload tolerance boundary limit, the aircraft will play the corresponding voice prompt in advance to leave the pilot with sufficient reaction time. The auditory reaction time is generally shorter than the visual reaction time, indicating that the subjects recognize auditory information faster than visual information. When visual information and auditory information are transmitted to the subjects at the same time, the subjects first complete the recognition of auditory information and then complete the recognition of visual information. Therefore, in order to minimize the pilot’s misunderstanding of the interface information when providing interface prompts, it is necessary to ensure the consistency of visual and auditory information. The audio prompt of the overload information should be transmitted to the pilot at the same time as the visual prompt.

[0077] Step S3, creating an aircraft simulator based on VR technology to obtain a simulated flight video, inputting the simulated flight video into a human-computer interaction interface and combining it with the flight state change result to generate a flight interaction instruction, and loading the flight interaction instruction into the main screen of the aircraft cockpit.

[0078] In this embodiment, when creating an aircraft simulator based on VR technology to obtain a simulated flight video, inputting the simulated flight video into a human-computer interaction interface and combining it with the flight state change results to generate flight interaction instructions, and loading the flight interaction instructions into the main screen of the aircraft cockpit, a VR flight scene and an aircraft model can be constructed as an aircraft simulator based on VR technology and modeling technology, and a flight display animation can be produced in combination with an interactive platform; a flight route program can be written according to a preset flight attitude, and a simulated flight video can be obtained by combining the aircraft simulator and the flight display animation, and the simulated flight video can be input into the human-computer interaction interface; the flight state change results can be combined with the simulated flight video to configure a flight voice prompt, and the flight voice prompt can be loaded as a flight interaction instruction into the main screen of the aircraft cockpit.

[0079] In this embodiment, when constructing a VR flight scene and an aircraft model as an aircraft simulator based on VR technology and modeling technology, and producing a flight display animation in combination with an interactive platform, the modeling technology can be used to draw a pre-defined two-dimensional map of the flight scene, set the corresponding texture information according to the flight scene, and render the flight scene based on the interactive platform and the texture setting results; the interactive platform is used to perform terrain mapping operations on the rendered flight scene, and the terrain is secondary rendered and repaired according to the mapping results, and a VR flight scene is obtained based on the repair results; the modeling technology is used to draw the weather scene corresponding to the VR flight scene, and the weather scene is imported into the interactive platform to generate an external flight scene of the aircraft; the modeling technology is used to draw a two-dimensional map corresponding to the aircraft and the cockpit, and the aircraft cockpit model and flight display animation are generated in combination with the interactive platform.

[0080] In this embodiment, when using modeling technology to draw two-dimensional images corresponding to the aircraft and the cockpit, and combining with the interactive platform to generate the aircraft cockpit model and the flight display animation, the modeling technology can be used to draw the two-dimensional images corresponding to the aircraft and the cockpit respectively, and the textures of the aircraft and the cockpit can be set. After the setting is completed, the two-dimensional images of the aircraft and the cockpit are imported into the interactive platform; according to the pre-configured mapping materials, the mapping operation is performed on the aircraft and the cockpit respectively in the interactive platform, and the position parameters of the cockpit are set according to the mapping results to obtain the aircraft cockpit model; based on the interactive platform, a new animation clip is created, and the aircraft cockpit model is used as the animation object to be produced in the animation clip to obtain the flight display animation.

[0081] It needs to be explained that the construction of a VR flight scene mainly consists of two parts, namely the flight scene and the weather scene. The flight scene needs to first be drawn in a corresponding two-dimensional map in a three-dimensional software such as 3DMax, and then the textures of the ground, mountains, rocks and surface forests are set, and finally imported into the Unity (interactive platform) engine for scene rendering to complete the configuration of the flight scene. Select the desired terrain map style in the window to implement the mapping operation. The terrain with configured mapping may have holes and punch-through, which requires re-rendering and repair operations.

[0082] The setting of flight weather scenes mainly includes the import and lighting of weather scenes. This part requires first using 3D software such as 3DMax to model the clouds and sun in the sky, and then importing them into Unity to generate the sky scene, generate the corresponding VR video, and test the interactive interface.

[0083] Specifically, the principle of building an aircraft model is similar to that of building a flight scene. You need to first model the aircraft in the three-dimensional software 3DMax, and set the corresponding textures of the aircraft parts, such as metal texture, leather texture, etc. Since the VR video simulates the pilot's driving perspective, the dashboard inside the cockpit needs to be drawn with emphasis. The steps are similar to the previous ones. You need to first perform two-dimensional modeling on the HUD, all dashboards, driver's seat, cockpit frame, etc. inside the cockpit, prepare their texture materials, and then import them into Unity to configure their position parameters and assemble them into a three-dimensional cockpit space. The configured cockpit environment is convenient for the subsequent design and addition of HUD animations.

[0084] Among them, there are three main ways to create flight display animations: making animations in the Unity engine, importing animations from external tools, and using code to make animations. At the same time, in the process of writing the flight program, the subjects are required to perform the process without any psychological expectations, and they need to fully experience and evaluate the interfaces under the three different flight postures of pitch, roll, and yaw. Therefore, the method adopted in this embodiment is: the three flight postures are randomly shuffled in the video, and corresponding icon indications appear on the interface. After each action is completed, there is an interval of 5-10 seconds before executing the next action. In order to fully reduce the psychological expectations of the subjects and avoid the subjects mastering the time pattern of the flight posture intervals after experiencing part of the video, the interval time of each posture is different to complete the setting of the flight route.

[0085] Depending on the flight status change, the specific voice content is "High overload! High overload! High overload!" or "Negative overload! Negative overload! Negative overload!". The voice configuration needs to synthesize an MP3 audio file in advance, import the audio file into the Unity project, and then use the Audio Source component driver to complete the voice prompt at the overload alarm moment.

[0086] Step S4: construct a human-computer interaction test platform to conduct a flight simulation experience, analyze the evaluation of the human-computer interaction interface according to the flight simulation experience, and determine the optimization recognition of the human-computer interaction interface based on the evaluation.

[0087] In this embodiment, when building a human-computer interaction test platform for flight simulation experience, analyzing the evaluation degree of the human-computer interaction interface according to the flight simulation experience, and determining the optimization recognition degree of the human-computer interaction interface based on the evaluation degree, the human-computer interaction test environment and flight scene corresponding to the self-responsive human-computer interaction interface and the historical human-computer interaction interface can be respectively built, and a simulated flight video can be generated for flight simulation experience; the flight simulation experience results are combined with the subjective evaluation form of the human-computer interaction interface pre-constructed using cognitive load technology to obtain the corresponding evaluation values ​​of the self-responsive human-computer interaction interface and the historical human-computer interaction interface; the reliability of the evaluation value is verified using Cronbach's coefficient and validity detection technology, and the optimization satisfaction of the self-responsive human-computer interaction interface is determined based on the reliability verification results and the evaluation value.

[0088] In this embodiment, when the reliability of the evaluation value is verified by using the Cronbach coefficient and the validity detection technology, and the optimization satisfaction of the self-responsive human-computer interaction interface is determined based on the reliability verification result and the evaluation value, the subjective evaluation form of the human-computer interaction interface can be standardized by using the data analysis technology, and the standardized subjective evaluation form of the human-computer interaction interface can be reliability analyzed by using the Cronbach coefficient to obtain the reliability value; the reliability coefficient of the evaluation value is verified by judging the difference between the reliability value and the reliability threshold, and the validity of the subjective evaluation form of the human-computer interaction interface is analyzed according to the Bartlett test technology and the factor analysis technology; the validity coefficient of the subjective evaluation form of the human-computer interaction interface is verified by judging the difference between the validity and the validity threshold, and the reliability of the evaluation value is determined according to the reliability coefficient and the validity coefficient; the reliability and the evaluation value are combined as a measure of the satisfaction of the self-responsive human-computer interaction interface and the historical human-computer interaction interface, and the optimization satisfaction of the self-responsive human-computer interaction interface is judged based on the satisfaction.

[0089] It should be explained that since the final user of the interactive interface is the pilot, after the interface is designed and implemented, it is necessary for personnel with certain flight experience to experience it in order to verify the effect of the interface design. The workload of the personnel participating in the experience during interface evaluation is one of the important dimensions for evaluating the quality of interface design, because there is an upper limit to the workload of pilots. When the workload of pilots is close to the upper limit, it is easy to make operational errors, which may lead to the inability to complete the task or flight safety problems. The evaluation methods based on the subjective evaluation index of cognitive load mainly include the Cooper-Harper evaluation method, the subjective load evaluation method SWAT scale, the NASA-TLX scale, etc. This embodiment uses the NASA-TLX scale as the subjective evaluation method of the interface to generate a questionnaire dimension table containing evaluation dimensions and evaluation contents, as shown in Table 3.

[0090] Table 3 Dimensions of subjective evaluation questionnaire

[0091]

[0092]

[0093] The purpose of the human-computer interaction interface design in this embodiment is to enable the pilot to have an intuitive and clear understanding of the maneuver to be performed and to form a certain psychological expectation. Therefore, the designed human-computer interaction interface of the autonomous air combat aircraft should be comprehensively evaluated by the pilot. Specifically, the test environment can be built in advance, and the experimental tasks can be explained to the test subjects after the test subjects are selected, the flight scene can be determined, and the VR equipment can be debugged. The helmet is worn and the experimental video is played to simulate the flight experience. There are two groups of experimental videos, namely, interactive interface videos with icon display as the experimental group, and ordinary HUD interface videos as the control group. After the experiment, the subjects fill in the interface evaluation subjective scale.

[0094] It should also be noted that reliability analysis is used to determine whether the survey questions in the questionnaire can reflect the purpose and intention of the survey, and whether the various questions in the questionnaire measure the same content and information. It is necessary to do a reliability analysis before formally using the questionnaire. At the same time, the correctness of the questionnaire can be measured by "validity" analysis, which mainly refers to the degree of conformity between the data measured by the questionnaire and the actual situation itself. In the field of prediction, validity measurement is a relatively scientific measurement standard. The validity measurement value is proportional to the authenticity of the content measured by the questionnaire. Its basic measurement method is mainly factor analysis.

[0095] (1) In reliability analysis, the size of Cronbach α coefficient is often used to measure the reliability of the questionnaire. Generally speaking, if the reliability coefficient of the subjective evaluation form of the human-computer interaction interface reaches 0.9 or above, the reliability of the subjective evaluation form of the human-computer interaction interface is good; if the reliability coefficient is above 0.8, it is good; it is generally believed that the reliability of the subjective evaluation form of the human-computer interaction interface is reasonable within 0.5 to 0.9. If the reliability coefficient is lower than 0.5, the reliability of the evaluation result of the subjective evaluation form of the human-computer interaction interface is poor.

[0096] (2) Validity analysis can be judged based on the size of KMO (Measure of Sampling Adequacy) to determine whether factor analysis can be used for analysis. When the KMO value is greater than 0.9, it means that the subjective evaluation form of the human-computer interaction interface can be analyzed using factor analysis. When the KMO value is above 0.8, it means that the subjective evaluation form of the human-computer interaction interface is more suitable for explanation and corresponding analysis using factor analysis. When the KMO value is above 0.7, it means that the subjective evaluation form of the human-computer interaction interface can be analyzed using factor analysis. If the KMO value of the subjective evaluation form of the human-computer interaction interface is above 0.6, it means that factor analysis can only be performed reluctantly. Only when the KMO value is lower than 0.6, it indicates that the subjective evaluation form of the human-computer interaction interface is indeed not suitable for relevant research using factor analysis.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An optimization method for the cockpit interactive interface of an autonomous aircraft based on VR technology, characterized in that: The optimization method includes: Design a self-responsive human-computer interaction interface based on integrated audio-visual technology and VR technology, and integrate the self-responsive human-computer interaction interface into the main screen of the aircraft cockpit; Using a human-computer interaction interface to collect aircraft motion parameters, obtaining flight state change results based on the aircraft motion parameters, generating state interaction instructions based on the flight state change results, and loading the state interaction instructions onto the main screen of the aircraft cockpit; Create an aircraft simulator based on VR technology to obtain simulated flight video, input the simulated flight video into the human-computer interaction interface and combine it with the flight state change results to generate flight interaction instructions, and load the flight interaction instructions into the main screen of the aircraft cockpit; Build a human-computer interaction test platform for flight simulation experience, analyze the evaluation of the human-computer interaction interface based on the flight simulation experience, and determine the optimization recognition of the human-computer interaction interface based on the evaluation.

2. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 1, characterized in that: The method of collecting aircraft motion parameters by using a human-computer interaction interface, obtaining flight state change results according to the aircraft motion parameters, generating state interaction instructions based on the flight state change results, and loading the state interaction instructions to the main screen of the aircraft cockpit includes: The human-computer interaction interface is used to obtain the vertical acceleration of the autopilot acting on the fuselage and the pilot during the historical maneuvers, and the overload change trend is predicted based on the vertical acceleration. The human-computer interaction interface is used to obtain the angular accelerations of the autopilot when there are lateral roll and longitudinal roll trends during the historical maneuvers. Based on the overload change trend and angular acceleration, overload state interaction instructions and speed state interaction instructions are generated accordingly according to instruction generation rules; The visual reaction time and auditory reaction time are obtained according to the historical human-computer interaction interface, the prompt time of the status interaction command is determined, and the status interaction command is loaded to the main screen of the aircraft cockpit according to the prompt time.

3. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 2, characterized in that: The generating of the overload state interaction instruction and the speed state interaction instruction according to the instruction generation rule based on the overload change trend and the angular acceleration includes: The overload change trend is divided into several overload intervals according to the overload value range that can be indicated, and the positive load visual prompt instruction and the positive load auditory prompt instruction are determined according to the overload interval division result; Determine a display frame corresponding to the overload change trend based on the overload interval division result, and determine a frame prompt instruction according to the display frame; Dividing the angular acceleration into a number of speed intervals according to the angular acceleration value range, determining the speed visual prompt instruction according to the speed interval division result, and defining the speed shape prompt instruction based on the speed interval; The positive load visual prompt instruction, the positive load auditory prompt instruction and the frame prompt instruction are used as overload interaction instructions, and the speed visual prompt instruction and the speed shape prompt instruction are used as speed state interaction instructions.

4. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 3, characterized in that: The overload value range that can be indicated is -5.0G to 9.9G, and the overload interval includes a normal positive overload interval, a positive overload transition interval, a dangerous positive overload interval, a normal negative overload interval and a dangerous negative overload interval.

5. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 4, characterized in that: The speed interval includes a normal angular acceleration, an angular acceleration transition interval, a limit angular acceleration interval and a dangerous angular acceleration interval.

6. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 1, characterized in that: The method of creating an aircraft simulator based on VR technology to obtain a simulated flight video, inputting the simulated flight video into a human-computer interaction interface and combining it with a flight state change result to generate a flight interaction instruction, and loading the flight interaction instruction into the main screen of the aircraft cockpit includes: Based on VR technology and modeling technology, VR flight scenes and aircraft models are constructed as aircraft simulators, and flight display animations are produced in conjunction with interactive platforms; Write a flight route program according to the preset flight attitude, obtain simulated flight video by combining the aircraft simulator and flight display animation, and input the simulated flight video into the human-computer interaction interface; The flight status change results are combined with the simulated flight video to configure the flight voice prompt, and the flight voice prompt is loaded as a flight interaction instruction to the main screen of the aircraft cockpit.

7. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 6, characterized in that: The VR flight scene and the aircraft model are constructed based on the VR technology and the modeling technology as an aircraft simulator, and the flight display animation is produced in combination with the interactive platform, including: Use modeling technology to draw a pre-defined two-dimensional map of the flight scene, set the corresponding texture information according to the flight scene, and render the flight scene based on the interactive platform and texture setting results; Use the interactive platform to perform terrain mapping operations on the rendered flight scene, and perform secondary rendering and repair operations on the terrain based on the mapping results, and obtain the VR flight scene based on the repair results; Use modeling technology to draw the weather scene corresponding to the VR flight scene, and import the weather scene into the interactive platform to generate the external flight scene of the aircraft; Modeling technology is used to draw a two-dimensional diagram corresponding to the aircraft and the cockpit, and the aircraft cockpit model and flight display animation are generated in combination with the interactive platform.

8. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 7, characterized in that: The method of using modeling technology to draw a two-dimensional picture corresponding to the aircraft and the cockpit, and combining the interactive platform to generate the aircraft cockpit model and flight display animation includes: Using modeling technology to draw two-dimensional images corresponding to the aircraft and the cockpit, respectively, set the textures of the aircraft and the cockpit, and after the setting is completed, import the two-dimensional images of the aircraft and the cockpit into the interactive platform; According to the pre-configured texture materials, the aircraft and the cockpit are respectively textured in the interactive platform, and the position parameters of the cockpit are set according to the texture results to obtain the aircraft cockpit model; A new animation clip is created based on the interactive platform, and the aircraft cockpit model is used as an animation object to be produced in the animation clip to obtain a flight display animation.

9. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 1, characterized in that: The step of building a human-computer interaction test platform for flight simulation experience, analyzing the evaluation of the human-computer interaction interface according to the flight simulation experience, and determining the optimization recognition of the human-computer interaction interface based on the evaluation includes: Build the human-computer interaction test environment and flight scenarios corresponding to the self-responsive human-computer interaction interface and the historical human-computer interaction interface respectively, and generate simulated flight videos for flight simulation experience; The flight simulation experience results are combined with the subjective evaluation table of the human-computer interaction interface pre-built using cognitive load technology to obtain the corresponding evaluation values ​​of the self-responsive human-computer interaction interface and the historical human-computer interaction interface; The reliability of the evaluation value is verified by using Cronbach's coefficient and validity testing technology, and the optimization satisfaction of the self-responsive human-computer interaction interface is determined based on the reliability verification results and the evaluation value.

10. The method for optimizing the cockpit interactive interface of an autonomous aircraft based on VR technology according to claim 9, characterized in that: The method of verifying the reliability of the evaluation value by using the Cronbach coefficient and validity detection technology, and determining the optimization satisfaction of the self-responsive human-computer interaction interface based on the reliability verification result and the evaluation value includes: The subjective evaluation form of human-computer interaction interface was standardized by using data analysis technology, and the reliability value was obtained by performing reliability analysis on the standardized subjective evaluation form of human-computer interaction interface using Cronbach's coefficient. The difference between the reliability value and the reliability threshold is used to verify the reliability coefficient of the evaluation value, and the validity of the subjective evaluation form of the human-computer interaction interface is analyzed based on the Bartlett test technique and factor analysis technique; The difference between the judgment validity and the validity threshold is used to verify the validity coefficient of the subjective evaluation form of the human-computer interaction interface, and the reliability of the evaluation value is determined based on the reliability coefficient and the validity coefficient; The reliability and evaluation value are combined as a measure of the satisfaction between the self-responsive human-computer interaction interface and the historical human-computer interaction interface, and the optimization satisfaction of the self-responsive human-computer interaction interface is judged based on the satisfaction.

Images