Task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method

Abstract

The embodiment of the invention relates to the technical field of display interaction control, in particular to a task-oriented multi-dimensional information intelligent presentation and multi-mode interaction method, which comprises the following steps of: dividing flight stages and task modes of an aircraft, and defining types and importance degrees of information in different flight stages and different task modes; according to the display characteristics of the display device, the playing characteristics of the spatial audio device, and the types and importance degrees of information in different flight stages and different task modes, an information presentation channel intelligent distribution rule is set, and a presentation mode of the display device is adaptively generated; information to be displayed is obtained, display pictures of different display devices are generated to perform presentation of a visual channel, and meanwhile presentation of an auditory channel is performed through a spatial audio device; and obtaining a multi-mode fuzzy interaction instruction input by a pilot, and performing interaction intention response. The method can effectively reduce the task load of the pilot.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of display interaction control technology, and in particular to a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method. Background Art

[0002] In a highly dynamic low-altitude environment, aircraft often face various unexpected threats. Pilots are under great pressure and heavy load in operational control and mission decision-making, making it difficult to quickly and accurately complete situation analysis, threat assessment, mission decision-making, sensor control, launch control and other operations. The current onboard cockpit display control system has weak auxiliary capabilities for tasks. During the entire mission, pilots rely almost entirely on their own experience to manually complete decision-making and operational control level tasks, and the overall load is too heavy.

[0003] As the mission radius continues to expand, the mission objectives become more and more numerous, and the low-altitude environment becomes more and more complex in the future, the amount of situational information pilots receive will increase dramatically, the difficulty of mission decision-making will increase further, and operations will become more complicated. If pilots still rely solely on their own abilities to process mission information, it will become very difficult to make correct judgments and operations immediately. Summary of the invention

[0004] In order to solve the above technical problems, the embodiments of the present application propose a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method, which can intelligently present key information of the task to the pilot through multiple terminals, and at the same time support the pilot to perform command control through multi-modal interaction, thereby assisting the pilot to complete multi-dimensional information perception and interaction, which can effectively improve the pilot's command efficiency and effectively reduce the pilot's task load.

[0005] To achieve the above-mentioned purpose, the embodiment of the present application proposes a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method, comprising the following steps: dividing the flight phases and mission modes of the aircraft, defining the types of information and their importance in different flight phases and different mission modes; setting information presentation channel intelligent allocation rules according to the display characteristics of different display devices, the playback characteristics of spatial audio devices, and the types of information and their importance in different flight phases and different mission modes, and adaptively generating presentation modes of different display devices; wherein the different display devices include integrated displays, flat panel displays, helmet displays, and heads-up displays; obtaining information to be displayed, and based on the types of information to be displayed and their importance, information presentation channel intelligent allocation rules, and presentation modes of different display devices, generating display images of different display devices for visual channel presentation, and performing auditory channel presentation through spatial audio devices; obtaining multi-modal interaction instructions input by the pilot, and responding to the multi-modal interaction instructions with interactive intent based on the current flight phase and current mission of the aircraft; wherein the multi-modal interaction instructions include key control instructions, tablet touch instructions, and fuzzy voice instructions.

[0006] To achieve the above-mentioned purpose, the embodiment of the present application also proposes a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction system, the system comprising: a division module, a rule-making module, a presentation module, a multi-modal interaction response module, different display devices and spatial audio devices, the different display devices including integrated displays, flat panel displays, helmet displays and heads-up displays; a division module for dividing the flight phases and mission modes of the aircraft, defining the types of information and their importance in different flight phases and different mission modes; a rule-making module for defining the types of information and their importance in different flight phases and different mission modes based on the display characteristics of different display devices, the playback characteristics of spatial audio devices, and the types of information in different flight phases and different mission modes. and its importance, set the intelligent allocation rules of information presentation channels, and adaptively generate presentation modes for different display devices; a presentation module, used to obtain the information to be displayed, and based on the type of information to be displayed and its importance, the intelligent allocation rules of information presentation channels, and the presentation modes of different display devices, generate display pictures of different display devices for visual channel presentation, and at the same time present the auditory channel through spatial audio equipment; a multimodal interaction response module, used to obtain the multimodal interaction instructions input by the pilot, and respond to the multimodal interaction instructions based on the current flight phase and current task of the aircraft; wherein the multimodal interaction instructions include button control instructions, tablet touch instructions and fuzzy voice instructions.

[0007] To achieve the above-mentioned purpose, an embodiment of the present application also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method as described above.

[0008] To achieve the above objectives, an embodiment of the present application also proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method as described above.

[0009] The present application proposes a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method, which is based on the existing equipment, operation method and display on the aircraft. By upgrading the software, the intelligent presentation of multi-dimensional information and the optimization of multi-modal interaction can be realized. It is highly practical and does not require major adjustments to the existing equipment. On the one hand, the present application divides the flight phase and mission mode of the aircraft, defines the type of information and its importance, sets the intelligent allocation rules of the information presentation channel, and adaptively generates the presentation mode of different display devices, and adaptively allocates the information to be displayed to the total display, flat panel display, helmet display, head-up display and spatial audio equipment, so as to perform comprehensive multi-dimensional information intelligent presentation. On the other hand, the present application receives the multi-modal interaction instructions input by the pilot in real time, including key control instructions, flat panel touch instructions and fuzzy voice instructions, and responds to the control intention in combination with the task information, thereby assisting the pilot to complete the multi-dimensional information perception and interaction, which can effectively improve the pilot's command efficiency and effectively reduce the pilot's task load.

[0010] In some optional embodiments, the flight phases of the aircraft are specifically divided into take-off phase, ultra-low altitude cruising phase, medium and high altitude cruising phase, approach phase and landing phase; the mission modes of the aircraft are specifically divided into reconnaissance missions, strike missions and collaborative missions; the types of information defined in different flight phases and different mission modes specifically include situational awareness and assessment information, target search and identification information, mission management and decision-making information, precise guidance information and health information; the importance of the information is defined specifically includes general information, important information and emergency information.

[0011] In some optional embodiments, the set intelligent allocation rules for information presentation channels are specifically: general information is presented through an integrated display, important information is presented through multiple channels of an integrated display, a flat-panel display and a spatial audio device, and emergency information is presented through multiple channels and in all directions through a helmet display, a head-up display, an integrated display, a flat-panel display and a spatial audio device; wherein the intelligent allocation rules for information presentation channels support pilots to make customized corrections.

[0012] In some optional embodiments, the presentation methods of different display devices specifically include: the integrated display displays the text and images of general information, important information and emergency information; the head-up display and helmet display present emergency information in three dimensions; the flat-panel display reminds and synchronously commands important information and emergency information; wherein, the presentation methods of different display devices support the pilot to make customized corrections, and when generating display images of different display devices for presenting visual channels, it is necessary to perform adaptive adjustment processing on the overall layout of the image.

[0013] In some optional embodiments, the spatial audio device reminds of important information and alarms for emergency information; the auditory channel is presented through the spatial audio device, including: obtaining the azimuth information of the target corresponding to the information to be displayed and the distance information between the target and the aircraft, and synthesizing a sound prompt with spatial characteristics based on the azimuth information and the distance information, so as to indicate the azimuth information and distance information of the target through sound.

[0014] In some optional embodiments, the fuzzy voice instructions include display screen adjustment instructions, operation control instructions and tactical mission execution instructions. For tactical mission execution instructions, an interactive intention response is made to the multimodal interactive instructions based on the current flight phase and current mission of the aircraft, including: intention recognition of the tactical mission execution instructions, and multi-channel and all-round presentation of the recognition results of the tactical mission execution instructions through a helmet display, a head-up display, an integrated display, a flat-panel display and spatial audio equipment for the pilot to make a second confirmation. After the second confirmation is passed, the tactical mission execution instructions are executed.

[0015] In some optional embodiments, after responding to the multimodal interactive instruction with interactive intent based on the current flight phase and current mission of the aircraft, the method further includes: prompting the pilot with instruction execution completion information by means of speech synthesis by a spatial audio device. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flowchart of a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method provided in one embodiment of the present application;

[0018] Figure 2 It is a schematic diagram of the principle of intelligent presentation of multi-dimensional information provided in one embodiment of the present application;

[0019] Figure 3 is a schematic diagram of the principle of multimodal interaction provided in one embodiment of the present application;

[0020] Figure 4 is a structural schematic diagram of a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction system provided in another embodiment of the present application;

[0021] Figure 5 It is a structural schematic diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. In the various embodiments of the present application, many technical details are proposed in order to make the reader better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is only for the convenience of description, and the specific implementation mode of the present application should not constitute any limitation. The various embodiments can be combined with each other and referenced to each other under the premise of no contradiction.

[0023] An embodiment of the present application proposes a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction, which is applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The following is a specific description of the implementation details of a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction proposed in this embodiment. The following content is only provided for the convenience of understanding the implementation details and is not necessary for the implementation of this solution. The specific process of a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction proposed in this embodiment can be as follows: Figure 1 As shown, including:

[0024] Step 101 , classify the flight phases and mission modes of the aircraft, and define the types and importance of information in different flight phases and different mission modes.

[0025] In the specific implementation, the server first needs to divide the flight phases and mission modes of the aircraft into different flight phases and different mission modes, and then define the type of information and its importance in different flight phases and different mission modes.

[0026] In one example, the flight phase of an aircraft can be specifically divided into the take-off phase, the ultra-low altitude cruising phase, the medium and high altitude cruising phase, the approach phase and the landing phase, and the mission mode of the aircraft can be specifically divided into reconnaissance missions, strike missions and collaborative missions. The server defines the types of information in different flight phases and different mission modes, including situational awareness and assessment information, target search and identification information, mission management and decision-making information, precise guidance information and health information, and defines the importance of information, including general information, important information and emergency information.

[0027] It is worth noting that the importance of information can also be called priority.

[0028] Step 102, according to the display characteristics of different display devices, the playback characteristics of spatial audio devices, and the type and importance of information in different flight phases and different mission modes, set the intelligent allocation rules of information presentation channels, and adaptively generate presentation methods for different display devices, where the different display devices include integrated displays, flat panel displays, helmet displays, and head-up displays.

[0029] In a specific implementation, after defining the types and importance of information in different flight phases and mission modes, the server can set intelligent allocation rules for information presentation channels based on the display characteristics of different display devices, the playback characteristics of spatial audio devices, and the types and importance of information in different flight phases and mission modes, and adaptively generate presentation modes for different display devices. Among them, different display devices include integrated displays, flat panel displays, helmet displays, and head-up displays.

[0030] In one example, the set intelligent allocation rules for information presentation channels are as follows: general information is presented through an integrated display, important information is presented through multiple channels of an integrated display, a flat-panel display and a spatial audio device, and emergency information is presented through multiple channels and in all directions through a helmet display, a head-up display, an integrated display, a flat-panel display and a spatial audio device.

[0031] In one example, the intelligent allocation rules for information presentation channels support pilots to make customized corrections to meet the actual needs of pilots during flight.

[0032] In one example, the integrated display is large and comprehensive, and its corresponding presentation method is to display the text and images of general information, important information and emergency information. The head-up display and helmet display have the characteristics of rapid feedback, high attention, and virtual-real fusion, but the display field of view and color capabilities are limited, so their corresponding presentation method is to present emergency information in three dimensions. Flat panel displays have the characteristics of information feedback and command integration, so their presentation method can be to remind and synchronously command important information and emergency information.

[0033] In one example, different display device presentation methods support pilots to make customized corrections to meet the pilots' actual needs during flight.

[0034] In one example, when the server generates display images for different display devices to present visual channels, it is necessary to perform adaptive adjustment processing on the overall layout of the image to ensure a complete and beautiful display.

[0035] Step 103, obtain the information to be displayed, and based on the type of information to be displayed and its importance, the intelligent allocation rules of the information presentation channel, and the presentation methods of different display devices, generate display images of different display devices to present the visual channel, and at the same time present the auditory channel through the spatial audio device.

[0036] In a specific implementation, after completing the setting of the intelligent allocation rules for information presentation channels and the setting of the presentation methods of different display devices, the server can obtain the information to be displayed in real time, and based on the type of information to be displayed and its importance, the intelligent allocation rules for information presentation channels, and the presentation methods of different display devices, generate display images of different display devices to present the visual channel, and at the same time present the auditory channel through spatial audio equipment.

[0037] In one example, the principle of intelligent presentation of multi-dimensional information can be as follows Figure 2 As shown, target rapid search and intelligent identification information, comprehensive situational awareness and assessment information, mission management and intelligent decision-making information, and precise guidance and position information waiting for display information generate a multi-dimensional information display demand list based on multi-dimensional information needs. At the same time, information presentation channels are allocated according to information content, type, and importance, and the information presentation method of each channel is determined to generate spatial audio, comprehensive display screen, HUD screen, head display screen, and tablet screen, thereby realizing the presentation of visual channels and auditory channels. When adjustments are needed, pilots can adjust the display presentation channel and presentation method.

[0038] In one example, spatial audio has audio perception and positioning capabilities, which is suitable for reminding important information and alarming emergency information. It can express the target's position, distance and other information through sound to improve the pilot's reaction time and effectively improve the pilot's situational awareness. When the server presents the auditory channel through the spatial audio device, it needs to obtain the target's position information and distance information from the aircraft corresponding to the information to be displayed, and synthesize a sound prompt with spatial characteristics based on the position information and distance information to indicate the target's position information and distance information through sound.

[0039] Step 104, obtaining the multimodal interaction instructions input by the pilot, and responding to the multimodal interaction instructions based on the current flight phase and current mission of the aircraft; wherein the multimodal interaction instructions include key control instructions, tablet touch instructions and fuzzy voice instructions.

[0040] In the specific implementation, the server can obtain the multi-modal interactive instructions input by the pilot in real time, including key control instructions, tablet touch instructions and fuzzy voice instructions, and respond to the multi-modal interactive instructions based on the current flight phase and current mission of the aircraft. Among them, the key control instructions are obtained by the pilot triggering the control buttons of the aircraft, the tablet touch instructions are obtained by the pilot operating on the tablet display (tablet controller), and the fuzzy voice instructions are obtained through the spatial audio device.

[0041] In one example, the principle of multimodal interaction can be described as follows Figure 3 As shown in the figure, the key information input by the pilot is converted into key commands, the voice information is converted into voice commands after noise removal and voice recognition, and the touch screen control information is converted into touch commands. Key commands, voice commands and touch commands can be combined to obtain combined commands. The server responds to the intention based on these commands to achieve task switching, screen switching, parameter adjustment, target selection and detail display, etc.

[0042] In one example, when the navigation guidance route deviates, the pilot can issue instructions to correct the route. After voice reply confirmation, the route correction can be adaptively completed in combination with the screen information.

[0043] In one example, fuzzy voice commands include display screen adjustment commands, operation control commands, and tactical mission execution commands. For tactical mission execution commands, the server responds to the multimodal interactive commands based on the current flight phase and current mission of the aircraft. It is necessary to identify the intent of the tactical mission execution commands, and present the recognition results of the tactical mission execution commands through helmet displays, head-up displays, integrated displays, flat-panel displays, and spatial audio devices in a multi-channel and all-round manner for pilots to confirm twice. After the second confirmation is passed, the tactical mission execution commands are executed. The second confirmation will be displayed on the screen and voice broadcast, and the pilot will confirm or abandon the operation by pressing buttons or voice.

[0044] In one example, after the server completes the interactive intention response to the multimodal interactive instruction based on the current flight phase and current mission of the aircraft, the server can also prompt the pilot with instruction execution completion information through speech synthesis by means of a spatial audio device.

[0045] The present embodiment proposes a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method, which is based on the existing equipment, operation method and display on the aircraft. By upgrading the software, the optimization of the intelligent presentation of multi-dimensional information and multi-modal interaction can be realized, which is highly practical and does not require major adjustments to the existing equipment. On the one hand, the present application divides the flight phase and mission mode of the aircraft, defines the type of information and its importance, sets the intelligent allocation rules of the information presentation channel, and adaptively generates the presentation mode of different display devices, and adaptively allocates the information to be displayed to the total display, flat panel display, helmet display, head-up display and spatial audio equipment, so as to perform comprehensive multi-dimensional information intelligent presentation. On the other hand, the present application receives the multi-modal interaction instructions input by the pilot in real time, including key control instructions, flat panel touch instructions and fuzzy voice instructions, and responds to the control intention in combination with the task information, so as to assist the pilot to complete the multi-dimensional information perception and interaction, which can effectively improve the pilot's command efficiency and effectively reduce the pilot's task load.

[0046] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0047] Another embodiment of the present application proposes a task-oriented multi-dimensional information intelligent presentation and interaction system. The implementation details of the task-oriented multi-dimensional information intelligent presentation and interaction system proposed in this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for the implementation of this example. Figure 4 It is a structural diagram of a task-oriented multi-dimensional information intelligent presentation and interaction system proposed in this embodiment, including: a division module 200, a rule-making module 300, a presentation module 400, a multimodal interactive response module 500, different display devices (including an integrated display 601, a flat panel display 602, a helmet display 603 and a head-up display 604) and a spatial audio device 605.

[0048] The classification module 200 is used to classify the flight phases and mission modes of the aircraft, and define the types and importance of information in different flight phases and different mission modes.

[0049] The rule-making module 300 is used to set intelligent allocation rules for information presentation channels according to the display characteristics of different display devices, the playback characteristics of the spatial audio device 605, and the type and importance of information in different flight phases and different mission modes, and to adaptively generate presentation methods for different display devices.

[0050] The presentation module 400 is used to obtain information to be displayed, and based on the type of information to be displayed and its importance, the intelligent allocation rules of the information presentation channel, and the presentation methods of different display devices, generate display images of different display devices to present the visual channel, and at the same time present the auditory channel through the spatial audio device 605.

[0051] The multimodal interaction response module 500 is used to obtain the multimodal interaction instructions input by the pilot, and respond to the multimodal interaction instructions based on the current flight phase and current mission of the aircraft; wherein the multimodal interaction instructions include key control instructions, tablet touch instructions and fuzzy voice instructions.

[0052] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0053] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above method embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above method embodiment.

[0054] Another embodiment of the present application provides an electronic device, whose specific structure is as follows: Figure 5 As shown, it includes: at least one processor 701; and a memory 702 that is communicatively connected to the at least one processor 701; wherein the memory 702 stores instructions that can be executed by the at least one processor 701, and the instructions are executed by the at least one processor 701 so that the at least one processor 701 can execute a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method as described in the above-mentioned method embodiments.

[0055] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be further described in this article. The bus interface is responsible for providing an interface between the bus and the transceiver. The transceiver can be one component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0056] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0057] Another embodiment of the present application proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement a task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method as described in the above method embodiments.

[0058] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk and other media that can store program codes.

[0059] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method, characterized in that: include: Divide the flight phases and mission modes of the aircraft, and define the types and importance of information in different flight phases and mission modes; According to the display characteristics of different display devices, the playback characteristics of spatial audio devices, and the types and importance of information in different flight phases and different mission modes, the intelligent allocation rules of information presentation channels are set, and presentation modes of different display devices are adaptively generated; among which, different display devices include integrated displays, flat panel displays, helmet displays and head-up displays; Acquire information to be displayed, and based on the type of information to be displayed and its importance, the intelligent allocation rule of information presentation channels, and the presentation modes of different display devices, generate display images of different display devices to present visual channels, and present auditory channels through spatial audio devices; The multimodal interaction instructions input by the pilot are obtained, and the interaction intention response to the multimodal interaction instructions is performed based on the current flight phase and current mission of the aircraft; wherein the multimodal interaction instructions include button control instructions, tablet touch instructions and fuzzy voice instructions.

2. The task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method according to claim 1, characterized in that: The flight phases of an aircraft are specifically divided into take-off phase, ultra-low altitude cruising phase, medium and high altitude cruising phase, approach phase and landing phase. The mission modes of an aircraft are specifically divided into reconnaissance missions, strike missions and collaborative missions. The types of information defined in different flight phases and different mission modes include situational awareness and assessment information, target search and identification information, mission management and decision-making information, precise guidance information and health information. The importance of information is defined as general information, important information and emergency information.

3. The task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method according to claim 2, characterized in that: The specific rules for intelligent allocation of information presentation channels are as follows: general information is presented through integrated displays, important information is presented through integrated displays, flat panel displays and spatial audio equipment through multiple channels, and emergency information is presented through helmet displays, head-up displays, integrated displays, flat panel displays and spatial audio equipment through multiple channels and in all directions; Among them, the intelligent allocation rules of information presentation channels support pilots to make customized corrections.

4. The task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method according to claim 3, characterized in that: The presentation modes of different display devices include: The integrated display shows the text and images of general information, important information and emergency information; Head-up displays and helmet displays provide three-dimensional presentation of emergency information; Flat panel display provides reminders and simultaneous charges for important and urgent information; Among them, the presentation methods of different display devices support pilots to make customized corrections. When generating display images of different display devices for presenting visual channels, the overall layout of the image needs to be adaptively adjusted.

5. The task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method according to claim 4, characterized in that: Spatial audio equipment reminds you of important information and issues alarms for emergency information; Presentation of auditory channels through spatial audio equipment, including: The target's position information and distance information from the aircraft corresponding to the information to be displayed are obtained, and a sound prompt with spatial characteristics is synthesized based on the position information and the distance information to indicate the target's position information and distance information through sound.

6. A task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method according to any one of claims 1 to 5, characterized in that: Fuzzy voice commands include display adjustment commands, operation control commands, and tactical mission execution commands. For tactical mission execution commands, the interactive intention response to the multimodal interactive commands is based on the current flight phase and current mission of the aircraft, including: The intention of tactical mission execution instructions is recognized, and the recognition results of tactical mission execution instructions are presented in a multi-channel and all-round manner through helmet displays, head-up displays, integrated displays, flat-panel displays and spatial audio equipment for the pilot to make a second confirmation. After the second confirmation is passed, the tactical mission execution instructions are executed.

7. A task-oriented multi-dimensional information intelligent presentation and multi-modal interaction method according to any one of claims 1 to 5, characterized in that: After responding to the multimodal interaction instruction with an interaction intention based on the current flight phase and the current mission of the aircraft, the method further includes: The spatial audio device prompts the pilot with the completion information of the command execution through speech synthesis.

8. A task-oriented multi-dimensional information intelligent presentation and interaction system, characterized in that: include: A division module, a rule-making module, a presentation module, a multimodal interactive response module, different display devices and spatial audio devices. The different display devices include integrated displays, flat panel displays, helmet displays and head-up displays; A division module is used to divide the flight phases and mission modes of the aircraft, and define the type and importance of information in different flight phases and different mission modes; A rule-making module is used to set intelligent allocation rules for information presentation channels according to the display characteristics of different display devices, the playback characteristics of spatial audio devices, and the types and importance of information in different flight phases and mission modes, and to adaptively generate presentation modes for different display devices; A presentation module, used to obtain information to be displayed, and based on the type of information to be displayed and its importance, the intelligent allocation rules of information presentation channels, and the presentation modes of different display devices, generate display images of different display devices for presentation of visual channels, and present auditory channels through spatial audio devices; The multimodal interaction response module is used to obtain the multimodal interaction instructions input by the pilot, and respond to the multimodal interaction instructions based on the current flight phase and current mission of the aircraft; wherein the multimodal interaction instructions include button control instructions, tablet touch instructions and fuzzy voice instructions.

9. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; In which, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a task-oriented multi-dimensional information intelligent presentation and interaction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it can implement a task-oriented multi-dimensional information intelligent presentation and interaction method as described in any one of claims 1 to 7.

Images