Airborne intelligent auxiliary control equipment and method based on communication information

Abstract

The invention relates to an intelligent auxiliary control method. The method comprises the following steps: acquiring communication information from communication equipment; extracting a control instruction from the communication information; acquiring aircraft information from other external equipment; determining the current working state of the aircraft based on the aircraft information; determining an auxiliary level of the control instruction based on the control instruction and the current working state of the aircraft; and providing intelligent assistance about the control instruction based on the assistance level. Other aspects of the disclosure also relate to a corresponding intelligent auxiliary control device.

Description

Technical Field

[0001] The present application generally relates to cockpit human-machine interaction interface design, and more particularly to an airborne intelligent auxiliary control device and method based on communication information. Background Art

[0002] In the cockpit human-machine interaction interface design of certain aircraft models, existing solutions generally use a tuning control panel and an audio control panel to control communication and backup navigation tuning. The crew has a heavy workload when performing control tasks of communication, navigation, and other aircraft systems / equipment. The human-machine interaction interface is crowded and complex, and crew operation is inconvenient.

[0003] For example, the various technical solutions currently proposed fail to consider factors such as the flight phase, system status, crew composition, and the impact of control commands on safety. Consequently, they fail to provide precise assistance to the crew in controlling communications, navigation, and other equipment, resulting in a low level of intelligence. Furthermore, the human-machine interface (HMI) is traditional and lacks auxiliary functions, resulting in a high workload for the crew.

[0004] Therefore, there is a need in this field for an improved solution that can intelligently provide the crew with precise assistance in performing communication, navigation and other equipment control, thereby reducing the crew's workload, by considering factors such as the flight phase, system status, crew members, and / or the safety impact of control instructions. Summary of the Invention

[0005] One aspect of the present disclosure relates to an intelligent assisted control method, comprising acquiring communication information from a communication device; extracting a control instruction from the communication information; acquiring aircraft information from other external devices; determining a current operating status of the aircraft based on the aircraft information; determining an assistance level for the control instruction based on the control instruction and the current operating status of the aircraft; and providing intelligent assistance with respect to the control instruction based on the assistance level.

[0006] According to some exemplary embodiments, determining the assistance level of the control instruction based on the control instruction and the current operating state of the aircraft includes determining an operation factor of the control instruction based on one or more attributes of the control instruction; determining a state factor of the aircraft based on one or more attributes of the current operating state of the aircraft; and determining the assistance level of the control instruction based on the operation factor of the control instruction and the state factor of the aircraft.

[0007] According to some exemplary embodiments, the assistance level includes at least a regular level and a managed level, wherein providing intelligent assistance for the control instruction includes, if the assistance level of the control instruction determined is a regular level: displaying the control instruction; providing an operation button; and if the operation button is operated, outputting the corresponding control instruction, or if the assistance level of the control instruction determined is a managed level: directly outputting the control instruction; and displaying the execution status of the control instruction.

[0008] According to some exemplary embodiments, the intelligent assistance for providing the control instructions includes a display control interaction interface, wherein the display control interaction interface includes a main control area for providing a manual control interface; and an intelligent assistance area for providing an intelligent assistance interface.

[0009] According to some exemplary embodiments, when the aircraft is currently in a two-crew mode, and the determined control instruction is a control instruction for at least one of the communication device or the navigation device and the assistance level is a normal level, the main control area of ​​the control interaction interface of the operation executing crew side in the two-crew mode is similarly displayed in the main control area of ​​the control interaction interface of the other side crew, and the intelligent assistance area of ​​the control interaction interface of the operation executing crew side is not displayed in the intelligent assistance control area of ​​the control interaction interface of the other side crew.

[0010] According to some exemplary embodiments, when the assistance level is a managed level, the control instructions and the managed execution status are displayed in the intelligent assistance area; or when the assistance level is a regular level, the control instructions are displayed in the intelligent assistance area and the operation buttons are provided.

[0011] According to some exemplary embodiments, the intelligent auxiliary control method further includes synchronously updating the control interaction interface based on the execution of the control instruction by the target device of the control instruction, wherein the target device includes one or more of the following or any combination thereof: the communication device, the navigation device, and the other external devices.

[0012] Another aspect of the present disclosure relates to an intelligent auxiliary control device, comprising an input / output unit for obtaining communication information from a communication device and aircraft information from other external devices; an intelligent auxiliary unit for extracting control instructions from the communication information, determining the current operating status of the aircraft, and determining an assistance level of the control instructions based on the control instructions and the current operating status of the aircraft; and a display unit for displaying a control interaction interface regarding the control instructions based on the assistance level.

[0013] According to some example embodiments, the assistance level comprises at least a regular level and a managed level, wherein the display unit is further configured to, if the determined assistance level of the control instruction is the regular level: display the control instruction; and provide an operation button; and wherein the intelligent assistance control device further comprises a control management unit configured to, if the operation button is operated, output a corresponding control instruction to the target device via the input / output unit.

[0014] According to some example embodiments, the control management unit is further configured to, if the determined assistance level of the control instruction is the managed level, output the control instruction directly to the target device via the input / output unit; and wherein the display unit is further configured to display an execution status of the control instruction.

[0015] Other aspects of the present disclosure also include corresponding devices, apparatuses, computer readable medium, and the like. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 An architecture diagram of an onboard intelligent assistance control device according to an aspect of the present disclosure is shown.

[0017] Figure 2 A flowchart of an intelligent assistance control method based on communication information according to an example aspect of the present disclosure is shown.

[0018] Figure 3 A flowchart of an intelligent assistance control interaction interface display and control instruction output method based on communication information according to an example aspect of the present disclosure is shown.

[0019] Figure 4 A schematic diagram of an intelligent assistance control interaction interface according to an example aspect of the present disclosure is shown.

[0020] Figure 5 A schematic diagram of an intelligent assistance control interaction interface displayed according to an example embodiment of the present disclosure is shown.

[0021] Figure 6 A schematic diagram of an intelligent assistance control interaction interface displayed according to an example embodiment of the present disclosure is shown.

[0022] Figure 7 A schematic diagram of a height indication of an auto flight control panel updated synchronously according to an example embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] The present disclosure relates to an onboard intelligent auxiliary control device and method based on communication information. According to some exemplary embodiments, the onboard intelligent auxiliary control device based on communication information according to the present disclosure may include, for example, an electronic device having one or more of a processor, a memory, a human-machine interface, and / or a communication bus, wherein the human-machine interface includes, but is not limited to, a control device, a touch screen, etc., or any combination thereof.

[0024] According to exemplary embodiments, the device disclosed herein can employ intelligent auxiliary control methods to intelligently analyze aircraft and communication information, providing the crew with hierarchical auxiliary control of communication and navigation equipment and other onboard equipment / systems. Communication and navigation equipment may refer to all communication and navigation equipment used to receive and transmit audio / data information between aircraft and between aircraft and the ground, including but not limited to radio communication / navigation equipment, satellite communication equipment, aircraft intercom communication equipment, data link communication equipment, CPDLC, etc. Other onboard equipment / systems may include but are not limited to flight management systems (FMS), navigation systems, and autopilot systems.

[0025] Figure 1 FIG. 1 shows an architecture diagram of an onboard intelligent auxiliary control device 100 according to one aspect of the present disclosure. Figure 1 As shown, according to an exemplary embodiment, the onboard intelligent auxiliary control device 100 can communicate with communication equipment, navigation equipment, and / or other onboard external devices / systems such as flight management systems and automatic flight systems. The communication can include various communication methods and technologies such as wired and wireless communication.

[0026] According to an exemplary embodiment, the onboard intelligent auxiliary control device 100 may receive input information from a communication and navigation device and / or other external devices / systems, and / or send control information output by the onboard intelligent auxiliary control device 100 to the communication and navigation device and / or other external devices / systems.

[0027] Specifically, according to an exemplary embodiment, the onboard intelligent auxiliary control device 100 may include a control management unit 102 for controlling and managing aircraft communication and navigation equipment, and generating control instructions for other identified external devices / systems.

[0028] According to an exemplary embodiment, the onboard intelligent auxiliary control device 100 may further include a display unit 104 : a control interaction interface for providing hierarchical assistance to the crew.

[0029] According to an exemplary embodiment, the onboard intelligent auxiliary control device 100 may further include an intelligent auxiliary unit 106 for intelligently identifying aircraft information and communication information (e.g., including but not limited to voice and / or data), and generating control instructions and the assistance level of the control instructions.

[0030] According to an exemplary embodiment, the onboard intelligent auxiliary control device 100 may also include an input and output unit 108: used to receive input information from the communication and navigation device and / or other external devices / systems, and send control information output by the onboard intelligent auxiliary control device 100 to the communication and navigation device and / or other external devices / systems.

[0031] Another aspect of the present disclosure relates to an intelligent auxiliary control method based on communication information. The intelligent auxiliary control method can be, for example, combined with the above Figure 1 The onboard intelligent auxiliary control device 100 shown and described is implemented.

[0032] According to the intelligent auxiliary control method disclosed in the present invention, for example, the aircraft state factor can be obtained by obtaining the aircraft position, altitude, speed, attitude, the status of each onboard external device / system, determining the aircraft's current flight phase, system failure level, environmental safety level and crew working status.

[0033] According to the intelligent auxiliary control method disclosed in the present invention, for example, control instructions can be extracted from communication information through natural language analysis, text recognition and other technologies, and the safety impact, workload and control frequency of the control instructions can be analyzed to determine the operation factors of the control instructions; the auxiliary level of the control instructions can be determined by combining the aircraft status factors and the operation factors, and a corresponding human-computer interaction interface can be provided.

[0034] Figure 2 A flow chart of an intelligent auxiliary control method 200 based on communication information according to an exemplary aspect of the present disclosure is shown.

[0035] According to an exemplary embodiment, the intelligent assistance control method 200 may include, at block 202, obtaining communication information from a communication device. According to an exemplary embodiment, the communication information obtained from the communication device may include, but is not limited to, voice information, data information, and the like. According to some exemplary embodiments, the communication information obtained from the communication device may include, but is not limited to, communication information such as radio voice, CPDLC, and / or DATALINK data, or any combination thereof.

[0036] According to some exemplary embodiments, the communication information may be obtained by combining the above Figure 1 According to some exemplary embodiments, the above combined with the input and output unit 108 of the airborne intelligent auxiliary control device 100 is obtained from the communication device. Figure 1 The input and output unit 108 of the onboard intelligent auxiliary control device 100 described above can send the above information to the above combined Figure 1 The display unit 104 and / or the intelligent assistance unit 106 of the onboard intelligent auxiliary control device 100 are described.

[0037] According to an exemplary embodiment, the intelligent assistance control method 200 may include extracting control instructions from the acquired communication information at block 204. According to an exemplary embodiment, extracting control instructions from the acquired communication information may include extracting control instructions by performing language / text recognition on the acquired communication information.

[0038] According to some exemplary embodiments, the above combination Figure 1 The intelligent assistance unit 106 of the described onboard intelligent auxiliary control device 100 inputs the acquired voice / data communication information into a natural language recognition / text recognition model to understand the crew's intention and extract control instructions.

[0039] According to some exemplary embodiments, the control instructions may include, but are not limited to, control instructions such as radio communication tuning, adjusting flight altitude, changing heading, and the like.

[0040] According to some exemplary embodiments, the intelligent auxiliary control method 200 may include determining an operation factor for the extracted control instruction at block 206. According to some exemplary embodiments, determining the operation factor for the extracted control instruction may include assigning weights to one or more attributes of the extracted control instruction (e.g., safety impact, workload, control frequency, etc.), and performing weighted summation to obtain the operation factor of the control instruction. According to some exemplary embodiments, the above combination may be used to determine the operation factor for the extracted control instruction. Figure 1 The intelligent assistance unit 106 of the onboard intelligent assistance control device 100 is described to determine the operation factor of the control instruction.

[0041] On the other hand, according to exemplary embodiments, the intelligent auxiliary control method 200 may include obtaining aircraft information from an external device / system at block 212. According to some exemplary embodiments, the aircraft information may include, but is not limited to, aircraft position, altitude, speed, attitude, and / or status information of various onboard devices / systems, etc. or any combination thereof. According to some exemplary embodiments, the above may be combined to obtain aircraft information. Figure 1 The input and output unit 108 of the described onboard intelligent auxiliary control device 100 obtains aircraft information from external devices / systems.

[0042] According to an exemplary embodiment, the intelligent auxiliary control method 200 may include, at block 214, determining the current operating state of the aircraft based on aircraft information obtained from an external device / system. According to some exemplary embodiments, determining the current operating state of the aircraft may include determining the current flight phase of the aircraft, the degree of system failure, the degree of environmental safety, and / or the operating state of the crew, or any combination thereof. According to some exemplary embodiments, the above may be combined to determine the current operating state of the aircraft. Figure 1 The intelligent assistance unit 106 of the onboard intelligent assistance control device 100 is described to determine the current working status of the aircraft.

[0043] According to an exemplary embodiment, the intelligent auxiliary control method 200 may include, at block 216, determining an aircraft state factor based on the determined current operating state of the aircraft. According to some exemplary embodiments, determining the aircraft state factor based on the determined current operating state of the aircraft may include assigning weights to one or more attributes of the current operating state of the aircraft (e.g., flight phase, system failure level, environmental safety level, and / or crew operating state (e.g., two-person / single-person crew mode), etc.), and performing weighted summation to obtain the current aircraft state factor. According to some exemplary embodiments, the above combination may be used to determine the aircraft state factor. Figure 1 The intelligent assistance unit 106 of the onboard intelligent assistance control device 100 is described to determine the aircraft state factor.

[0044] According to an exemplary embodiment, the intelligent auxiliary control method 200 may include, at block 222, assigning weights to the determined aircraft state factors and operation factors, and performing weighted summation to obtain an assistance level for the control instruction. According to some exemplary embodiments, the assistance level may be divided into two levels, for example, conventional and managed. According to other exemplary embodiments, the assistance level may be divided into more or fewer levels. According to some exemplary embodiments, the above may be combined to obtain the assistance level. Figure 1 The intelligent assistance unit 106 of the onboard intelligent assistance control device 100 described above may be used to obtain the assistance level of the control command. For example, according to some exemplary embodiments, as an illustrative but non-limiting example of an aircraft state factor, a two-crew mode may be assigned a weight that is more inclined toward the conventional level than the managed level compared to a single-crew mode.

[0045] According to an exemplary embodiment, the intelligent auxiliary control method 200 may include displaying the auxiliary control instruction at block 224. According to some exemplary embodiments, displaying the auxiliary control instruction may include providing a control interaction interface based on the identified control instruction and its auxiliary level. According to some exemplary embodiments, the above may be combined with Figure 1 The display unit 104 of the onboard intelligent auxiliary control device 100 is described to provide a control interaction interface.

[0046] Figure 3 A flowchart of a method 300 for displaying an intelligent auxiliary control interaction interface and outputting control instructions based on communication information according to an exemplary aspect of the present disclosure is shown.

[0047] According to some exemplary embodiments, the intelligent auxiliary control interactive interface display method 300 can be, for example, combined with the above Figure 1 The display unit 104 of the onboard intelligent auxiliary control device 100 is described to provide a hierarchical auxiliary control interaction interface for the crew.

[0048] According to some exemplary embodiments, the intelligent auxiliary control interactive interface display method 300 may be a combination of the above Figure 2 The intelligent auxiliary control method 200 described herein may be extended, supplemented, or further included in combination with Figure 2 For example, according to some exemplary embodiments, the intelligent auxiliary control interactive interface display method 300 may be implemented or included in the above combination. Figure 2 The assist control command is displayed in block 224 of the depicted intelligent assist control method 200 .

[0049] According to some exemplary embodiments, the intelligent auxiliary control interactive interface display method 300 may include, at block 326, determining the assistance level of the control instruction. According to exemplary embodiments, the assistance level of the control instruction may be divided into two levels, for example, regular and managed. According to other exemplary embodiments, the assistance level may be divided into more or fewer levels. Figure 3 In the exemplary embodiment shown in FIG, block 326 may include determining whether the assistance level of the control instruction is a managed level. If so, the process 300 goes to block 328. If not, the process 300 goes to block 332.

[0050] According to an exemplary embodiment, the intelligent auxiliary control interactive interface display method 300 may include directly outputting the control instruction in box 328 when it is determined in box 326 that the assistance level of the control instruction is the managed level, and displaying the execution status of the control instruction in box 330. The output of box 328 may be, for example, combined with the above. Figure 1 The input and output unit 108 of the depicted onboard intelligent auxiliary control device 100 outputs to a target device (eg, a communication device, a navigation device, or other external device / system).

[0051] The display of frame 330 can be obtained by combining the above Figure 1 The description is performed by the display unit 104 of the onboard intelligent auxiliary control device 100.

[0052] According to some exemplary embodiments, when the assistance level of the control instruction is the trusteeship level, the display unit may only display the identified control instruction and the trusteeship execution status without providing an operation button for the crew to select.

[0053] On the other hand, according to an exemplary embodiment, the intelligent auxiliary control interactive interface display method 300 may include displaying the identified control instruction in box 332 and providing operation buttons for the crew to select when it is determined in box 326 that the assistance level of the control instruction is not the managed level (for example, when the assistance level of the control instruction is normal).

[0054] According to an exemplary embodiment, the intelligent auxiliary control interactive interface display method 300 may include determining whether an execution operation is selected at block 334. If so, the process 300 goes to block 336. If not, the process 300 returns to block 334.

[0055] According to an exemplary embodiment, the intelligent auxiliary control interactive interface display method 300 may include outputting a control instruction in box 336 when an execution operation is selected in box 334. According to an exemplary embodiment, the selection of the execution operation may be achieved by operating an operation button. According to an exemplary embodiment, the output of box 336 may be a combination of the above Figure 1 The input and output unit 108 of the depicted onboard intelligent auxiliary control device 100 outputs to a target device (eg, a communication device, a navigation device, or other external device / system).

[0056] According to an exemplary embodiment, the intelligent auxiliary control interaction interface display method 300 may include updating the display at block 338. In at least some embodiments of such exemplary embodiments, updating the display at block 338 may include, for example, updating the main control area of ​​the control interaction interface (hereinafter in conjunction with Figure 4 According to at least some embodiments, updating the display at block 338 may include, for example, synchronously updating the control interaction interface (eg, its main control area) based on a corresponding control indication after the target device performs an operation.

[0057] Figure 4 A schematic diagram of an intelligent auxiliary control interaction interface 400 according to an exemplary aspect of the present disclosure is shown.

[0058] According to some exemplary embodiments, the intelligent auxiliary control interactive interface 400 may include a main control area 402 and / or an intelligent auxiliary area 404. For example, the main control area may provide a manual control interface for the crew and may display the results of operations and / or management, while the intelligent auxiliary area may provide an intelligent auxiliary interface for the crew.

[0059] According to some exemplary embodiments, if the control instruction is for a communication and navigation device and the assistance level is normal, and the current mode is two-person crew mode, the main control area 402 of the control interaction interface 400 on the operating unit side can display the corresponding communication and navigation control page, and the intelligent assistance area 404 can display the recognized communication and navigation control instruction and provide operation buttons for the crew to select. In two-person crew mode, the interface display designed for the operating unit may differ from that on the other side. For example, according to an exemplary embodiment, if the crew selects a setting operation, the display unit can send the communication and navigation control instruction to the control management unit. At the same time, the communication and navigation control results are displayed in the main control area 402. The main control area 402 of the control interaction interface 400 on the other side of the unit can display the same communication and navigation control page, but the intelligent assistance area 404 may not display the communication and navigation control instruction and status.

[0060] According to some exemplary embodiments, if the control command is for an external system / device and the assistance level is normal, the control interaction interface 400 may maintain the current main control area 402 while displaying the identified external system / device control command in the intelligent assistance area 404 and providing an operation button for the crew to select. If the crew selects a setup operation, the display unit may send the external system / device control command to the control management unit.

[0061] According to some exemplary embodiments, if the control instruction is for a communication and navigation device and the assistance level is managed, the display unit sends the communication and navigation control instruction to the control management unit. Simultaneously, the main control area 402 of the control interaction interface 400 displays the corresponding communication and navigation control page, and the intelligent assistance area 404 displays the recognized communication and navigation control instruction and the managed execution status.

[0062] According to some exemplary embodiments, if the control instruction is for an external system / device and the assistance level is managed, the display unit sends the control instruction for the external system / device to the control management unit. Simultaneously, the control interaction interface 400 maintains the current main control area 402, while the intelligent assistance area 404 displays the identified control instruction for the external system / device and the managed execution status.

[0063] According to some exemplary embodiments, the control management unit transmits valid control instructions to the corresponding communication and navigation device and / or external control device / system via the input and output unit. Once the communication and navigation device and / or external control device / system receives and executes the control instruction, the intelligent assistance area 404 displays the status of the control instruction. Simultaneously, if the external control device / system has a corresponding control instruction, the control instruction for that device / system is updated synchronously.

[0064] The following provides some specific implementation scenarios and embodiments of the technology of the present disclosure to facilitate those skilled in the art to more easily understand the present disclosure. It should be understood by those skilled in the art that the following specific embodiments are merely specific implementations of the technical solutions of the present disclosure described above in some specific implementation scenarios, and the specific details thereof do not constitute a limitation on the broadest scope protected by the present disclosure.

[0065] Example 1

[0066] Assume that the current flight phase of the aircraft is before takeoff, all systems are fault-free, the aircraft environment is safe, and the crew is in two-person working mode.

[0067] The input and output unit obtains aircraft information from external devices / systems, including aircraft position, altitude, speed, attitude, status information of various onboard devices / systems, etc.

[0068] The intelligent assistance unit determines, based on the information provided by the input and output units, that the aircraft's current flight phase is before takeoff, all systems are fault-free, the aircraft environment is safe, and it is in a two-crew working mode.

[0069] The intelligent auxiliary unit assigns weights to the flight phase, system failure level, environmental safety level, and crew working status, and takes the weighted sum to obtain the aircraft status factor.

[0070] The intelligent auxiliary unit queries the corresponding aeronautical chart data based on the aircraft's position, obtains the communication and navigation equipment number and frequency information broadcast by the current departure airport, and generates radio communication tuning control instructions.

[0071] The intelligent auxiliary unit performs weighted summation on the safety impact, workload, and control frequency allocation weight of the radio communication tuning control instruction to obtain an operation factor of the radio communication tuning control instruction.

[0072] The intelligent assistance unit assigns weights to the above aircraft status factors and operation factors, performs weighted summation, and obtains a normal assistance level for the radio communication tuning control instruction.

[0073] The display unit displays the radio tuning page in the main control area of ​​the control interaction interface on the operating unit side, displays the identified radio communication tuning control instructions in the intelligent auxiliary area, and provides operation buttons for the unit to select.

[0074] Figure 5 A schematic diagram of an intelligent auxiliary control interaction interface 500 displayed according to an exemplary embodiment of the present disclosure is shown. The intelligent auxiliary control interaction interface 500 displayed according to the exemplary embodiment may be generated in the above-described embodiment 1. According to the exemplary embodiment, the displayed intelligent auxiliary control interaction interface 500 may include a main controller 502 and an auxiliary display area 504.

[0075] The intelligent auxiliary control interaction interface 500 displayed according to an exemplary embodiment can be displayed by a display unit. The main control area 502 of the control interaction interface 500 on the operating unit side can display a radio tuning page. The intelligent auxiliary area 504 can display the recognized radio communication tuning control instructions (for example, Figure 5 SET VHF1 118.300) is displayed in the display and operation buttons are provided for the crew to select.

[0076] According to an exemplary embodiment, Figure 5 As shown in the embodiment of FIG, the operation buttons provided may include active, standby, and ignore. As can be seen, this is just an example. In other embodiments, more / fewer operation buttons may be provided, and the names and functions of the buttons may be different.

[0077] The crew can make selections using touch gestures, voice control, and / or control devices. For example, if the crew selects the "active" button, the tuning information is updated to the active radio communication device and frequency on the radio tuning page. If the crew selects the "standby" button, the tuning information is entered into the standby radio communication device and frequency on the radio tuning page. The display unit sends the selected radio tuning control instructions to the control management unit. If the crew selects the "ignore" button, the radio communication tuning control instructions are cleared from the intelligent auxiliary display area, and the radio tuning page is not updated.

[0078] According to an exemplary embodiment, the control management unit may send the radio tuning control instruction signal selected by the unit to the corresponding target device through the input and output unit. The target device operates according to the radio tuning control instruction signal from the control management unit and feeds back the operation result to the input and output unit. The input and output unit receives the operation result of the target device and sends the operation result to the display unit. The display unit updates the display according to the received operation result of the target device. In such exemplary embodiments, updating the display may include, for example, updating the main control area of ​​the control interaction interface. According to at least some embodiments, updating the display may include, for example, synchronously updating the control interaction interface (for example, its main control area) based on the corresponding control instructions after the target device performs the operation. For example, in Figure 5 In the example shown, the main control area may display that the activated VHF1 frequency is 118.300.

[0079] Example 2

[0080] Assume that the aircraft is currently in the approach phase, the automatic navigation system is faulty, the aircraft environment is safe, and the crew is in single-person working mode.

[0081] The input and output unit obtains the controller's voice call information from the communication and navigation equipment and the aircraft information provided by external equipment / systems, including the aircraft's position, altitude, speed, attitude, and status information of each onboard equipment / system.

[0082] The intelligent auxiliary unit determines, based on the information provided by the input and output units, that the current flight phase of the aircraft is approach, the automatic navigation system is faulty, the aircraft environment is safe, and the aircraft is in single-crew working mode.

[0083] The intelligent auxiliary unit assigns weights to the current flight phase, system failure level, environmental safety level, and crew working status, and takes the weighted sum to obtain the aircraft status factor.

[0084] The intelligent auxiliary unit performs semantic recognition on the controller's voice call information and generates radio navigation tuning control instructions.

[0085] The intelligent assistance unit performs semantic recognition on the controller's voice call information and generates altitude change control instructions.

[0086] The intelligent auxiliary unit performs weighted summation on the safety impact, workload and control frequency allocation weight of the radio navigation tuning control command to obtain the operation factor of the radio navigation tuning control command.

[0087] The intelligent auxiliary unit assigns weights to the safety impact, workload, and control frequency of the altitude change control instruction, and performs weighted summation to obtain the operation factor of the altitude change control instruction.

[0088] The intelligent assistance unit assigns weights to the above aircraft status factors and operation factors, performs weighted summation, and determines that the assistance level of the radio navigation tuning control instruction is managed and the assistance level of the altitude change control instruction is conventional.

[0089] Figure 6 A schematic diagram of an intelligent auxiliary control interaction interface 600 displayed according to an exemplary embodiment of the present disclosure is shown. The intelligent auxiliary control interaction interface 600 displayed according to the exemplary embodiment may be generated in the above-described embodiment 2. According to the exemplary embodiment, the displayed intelligent auxiliary control interaction interface 600 may include a main controller 602 and an auxiliary display area 604.

[0090] exist Figure 6 In the schematic diagram of control interface 600 of Example 2, the display unit displays the radio navigation tuning page in main control area 602 of control interface 600, and the radio navigation equipment and frequency are set according to the instructions. The intelligent assistance area 604 displays the recognized radio navigation tuning control instruction (e.g., SETILS135.100) and the managed execution status. Simultaneously, the display unit displays the recognized altitude change control instruction (e.g., SETALT 600) in the line below the radio navigation tuning control instruction and provides operation buttons for the crew to select. The operation buttons may include: set, hold, and ignore.

[0091] The crew can make a selection using touch gestures, voice commands, or control devices. If the crew selects the "set" button, the display unit sends the altitude control command selected by the crew to the control management unit. The control management unit then sends the altitude command signal to the autoflight system via the input and output unit.

[0092] According to an exemplary embodiment, the automatic flight system receives the instruction and executes it. Simultaneously, the altitude indication of the automatic flight control panel is updated synchronously. Figure 7 FIG. 7 is a schematic diagram showing an altitude indication 700 of an automatic flight control panel that is updated synchronously according to an exemplary embodiment of the present disclosure. Figure 7 As shown, the height has been updated to 600.

[0093] Back to Figure 6 If the crew selects the “hold” operation button, the instruction is moved down to the bottom of the intelligent assistance area 604. If the crew selects the “ignore” operation button, the altitude change control instruction in the intelligent assistance display area 604 is cleared.

[0094] The above description is merely an exemplary embodiment of the present invention. However, the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention.

[0095] The technical solution disclosed herein provides an intelligent assisted control method for an aircraft and a corresponding human-machine interface. This solution analyzes factors such as the current flight phase, system status, and the safety impact of control commands to rank the assistance level of identified control commands. Furthermore, this solution provides the crew with a concise human-machine interface based on the control assistance level.

[0096] The technical solution disclosed herein can improve, alleviate, and / or resolve at least one or more of the following technical issues: the heavy workload associated with controlling communications, navigation, and other equipment / systems, and the complex human-machine interface. Thus, the technical solution disclosed herein can minimize crew workload while also reasonably ensuring the necessary control rights exercised by the crew, achieving precise assistance and a high degree of intelligence.

[0097] The technical solutions disclosed herein can optimize the cockpit control interface layout, significantly improving the user experience. They can also reduce aircraft weight and enable cost-effective upgrades of onboard control equipment. As will be understood by those skilled in the art, the aforementioned technical effects of this disclosure may not be achieved solely by a single embodiment.

[0098] The various illustrative logical blocks, modules, units, and circuits described in conjunction with the present disclosure may be implemented or executed with a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0099] The steps of the method or algorithm described in conjunction with the present disclosure can be implemented directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module can reside in any form of storage medium known in the art. Some examples of usable storage media include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, etc. The software module can include a single instruction or many instructions and can be distributed over several different code segments, distributed between different programs and distributed across multiple storage media. A storage medium can be coupled to a processor so that the processor can read and write information from / to the storage medium. Alternatively, a storage medium can be integrated into the processor.

[0100] The methods disclosed herein include one or more steps or actions for achieving the described method. These method steps and / or actions may be interchangeable with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of the specific steps and / or actions may be modified without departing from the scope of the claims.

[0101] The processor can execute software stored on a machine-readable medium. The processor can be implemented with one or more general and / or special-purpose processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuit systems that can execute software. Software should be broadly interpreted to mean instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or other. As an example, the machine-readable medium may include RAM (random access memory), flash memory, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), registers, magnetic disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. The machine-readable medium may be implemented in a computer program product. The computer program product may include packaging materials.

[0102] In a hardware implementation, the machine-readable medium may be a portion of the processing system that is separate from the processor. However, as will be readily appreciated by those skilled in the art, the machine-readable medium or any portion thereof may be external to the processing system. As an example, the machine-readable medium may include a transmission line, a carrier wave modulated by data, and / or a computer product separate from the wireless node, all of which may be accessed by the processor via a bus interface. Alternatively or in addition, the machine-readable medium or any portion thereof may be integrated into the processor, as may be the case with a cache and / or general register file.

[0103] The processing system can be configured as a general-purpose processing system having one or more microprocessors providing processor functionality and external memory providing at least a portion of the machine-readable medium, all linked together with other supporting circuitry via an external bus architecture. Alternatively, the processing system can be implemented as an ASIC (application-specific integrated circuit) with a processor, bus interface, user interface (in the case of an access terminal), supporting circuitry, and at least a portion of the machine-readable medium integrated into a single chip, or as one or more FPGAs (field programmable gate arrays), PLDs (programmable logic devices), controllers, state machines, gating logic, discrete hardware components, or any other suitable circuitry, or any combination of circuits capable of performing the various functionalities described throughout this disclosure. Depending on the specific application and the overall design constraints imposed on the overall system, those skilled in the art will recognize how to best implement the functionality described with respect to the processing system.

[0104] The machine-readable medium may include several software modules. These software modules include instructions that, when executed by a device (such as a processor), cause a processing system to perform various functions. These software modules may include a transmitting module and a receiving module. Each software module may reside in a single storage device or be distributed across multiple storage devices. As an example, when a triggering event occurs, a software module may be loaded from a hard drive into RAM. During execution of the software module, the processor may load some instructions into a cache to increase access speed. One or more cache lines may then be loaded into a general register file for execution by the processor. When describing the functionality of a software module below, it will be understood that such functionality is implemented by the processor when the processor executes instructions from the software module.

[0105] If implemented in software, each function may be stored as one or more instructions or codes on or transmitted by a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (such as infrared (IR), radio, and microwave), then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (such as infrared, radio, and microwave) is included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Disks, where disks often reproduce data magnetically, and discs reproduce data optically with lasers. Thus, in some aspects, computer-readable media may include non-transitory computer-readable media (e.g., tangible media). Additionally, for other aspects, computer-readable media may include transient computer-readable media (e.g., signals). Combinations of the above should also be included within the scope of computer-readable media.

[0106] Thus, some aspects may include a computer program product for performing the operations presented herein. For example, such a computer program product may include a computer-readable medium having stored (and / or encoded) thereon instructions, which are executable by one or more processors to perform the operations described herein. In some aspects, the computer program product may include packaging materials.

[0107] It will be understood that the claims are not limited to the precise configuration and components illustrated above. Various changes, substitutions and variations may be made in the arrangement, operation and details of the methods and apparatus described above without departing from the scope of the claims.

Claims

1. An intelligent auxiliary control method, comprising: obtaining communication information from a communication device; extracting control instructions from the communication information; Obtain aircraft information from other external devices; determining a current operating status of the aircraft based on the aircraft information; determining an assistance level for the control command based on the control command and a current operating state of the aircraft; as well as Based on the assistance level, intelligent assistance with respect to the control command is provided.

2. The intelligent auxiliary control method according to claim 1, wherein: Determining an assistance level of the control instruction based on the control instruction and the current operating state of the aircraft includes: determining an operation factor of the control instruction based on one or more attributes of the control instruction; determining a state factor of the aircraft based on one or more attributes of the current operating state of the aircraft; and The assistance level of the control instruction is determined based on an operational factor of the control instruction and a state factor of the aircraft.

3. The intelligent auxiliary control method according to claim 1, wherein: The assistance levels include at least a regular level and a managed level, wherein the intelligent assistance providing the control instructions includes: If the determined assistance level of the control instruction is a normal level: displaying the control instructions; Provide action buttons; and If the operation button is operated, the corresponding control instruction is output, or If the assistance level of the control instruction is determined to be the trusteeship level: directly outputting the control instruction; and Displays the execution status of the control instruction.

4. The intelligent auxiliary control method according to claim 3, wherein: The intelligent assistance for providing the control instructions includes a display control interaction interface, and the display control interaction interface includes: A main control area, used to provide a manual control interface; and The intelligent assistance area is used to provide an intelligent assistance interface.

5. The intelligent auxiliary control method according to claim 4, wherein: When the aircraft is currently in two-crew mode, and the determined control instruction is a control instruction for at least one of the communication device or the navigation device and the assistance level is a normal level, the main control area of ​​the control interaction interface of the operation executing crew side in the two-crew mode is similarly displayed in the main control area of ​​the control interaction interface of the other side crew, and the intelligent assistance area of ​​the control interaction interface of the operation executing crew side is not displayed in the intelligent assistance control area of ​​the control interaction interface of the other side crew.

6. The intelligent auxiliary control method according to claim 4, wherein: When the assistance level is the trusteeship level, displaying the control instruction and trusteeship execution status in the intelligent assistance area; or When the assistance level is a normal level, the control instruction is displayed in the intelligent assistance area and the operation button is provided.

7. The intelligent auxiliary control method according to claim 4, further comprising: Based on the execution of the control instruction by the target device of the control instruction, the control interaction interface is synchronously updated, wherein the target device includes one or more of the following or any combination thereof: the communication device, the navigation device, and the other external devices.

8. An intelligent auxiliary control device, comprising An input and output unit, used to obtain communication information from the communication device and aircraft information from other external devices; an intelligent assistance unit, configured to extract a control instruction from the communication information, determine a current operating state of the aircraft, and determine an assistance level for the control instruction based on the control instruction and the current operating state of the aircraft; as well as A display unit is configured to display a control interaction interface regarding the control instruction based on the assistance level.

9. The intelligent auxiliary control device according to claim 8, wherein: The assistance level includes at least a regular level and a managed level, wherein the display unit is further configured to: If the determined assistance level of the control instruction is a normal level: displaying the control instructions; and provides an action button; and wherein The intelligent auxiliary control device further includes a control management unit, which is used to output corresponding control instructions to the target device through the input and output unit if the operation button is operated.

10. The intelligent auxiliary control device according to claim 9, wherein: The control management unit is further configured to: if the determined assistance level of the control instruction is the trusteeship level, directly output the control instruction to the target device through the input and output unit; and wherein The display unit is further configured to display the execution status of the control instruction.

Images