Interface system for cockpit communication

Abstract

To provide an interface system and a method for flight deck communications to perform a conversation with a pilot over a chatbot.SOLUTION: An interface system 100 for flight deck communications includes a chatbot 102 configured to perform a conversation with a pilot. The conversation includes speech communications and / or visual communications using a display. The interface system also includes a dynamic conversational graph generator 128. The dynamic conversational graph generator: determines a flight operational procedure from the conversation with the pilot; provides information associated with the flight operational procedure to the chatbot for communication with the pilot; and responds to any request received from the pilot by the chatbot during the conversation with the pilot.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to aircraft and other vehicles, and more particularly to an interface system for flight deck communication.

Background Art

[0002] During the operation of an aircraft, flight crew members interact with various aircraft systems, monitor the equipment installed on the aircraft, access information, communicate with air traffic controllers, and respond to various events during all phases of flight, among other tasks involved in flying the aircraft. Along with actually flying the aircraft, accessing information, monitoring equipment, and responding to specific events can sometimes be very busy for crew members. This is especially true when the workload is high. Therefore, there is a need for a system that reduces the pilot's workload, improves situation awareness, and allows the crew to focus on the actual operation and navigation of the aircraft.

Summary of the Invention

[0003] According to one embodiment, an interface system for flight deck communication includes a chatbot configured to converse with a pilot. The conversation includes voice communication, visual communication using a display, or both. The interface system also includes a dynamic conversation graph generation unit configured to perform a set of functions. The set of functions includes determining a flight operation procedure from a conversation with the pilot, providing information associated with the flight operation procedure to the chatbot for communicating with the pilot, and responding to any requests received from the pilot by the chatbot during the conversation with the pilot.

[0004] In another embodiment, a method for cockpit communication includes a chatbot conversing with a pilot, in which case the conversation includes voice communication, visual communication using a display, or both. The method also includes a dynamic conversation graph generator determining flight procedures from the conversation with the pilot. The method further includes the dynamic conversation graph generator providing the chatbot with information associated with the flight procedures in order to communicate with the pilot. The method further includes the dynamic conversation graph generator responding to any requests received by the pilot via the chatbot during the conversation with the pilot.

[0005] According to one embodiment and any of the embodiments described above, the interface system and method further include a tokenizer. The tokenizer is configured to perform the following: convert text received from the chatbot into tokens to be sent by the tokenizer, and convert the tokens received by the tokenizer into text to be sent to the chatbot to converse with the pilot.

[0006] According to one embodiment and any of the embodiments described above, a set of functions performed by the dynamic conversation graph generation unit includes detecting keywords in a conversation with a pilot using tokens from a tokenizer, the keywords identifying a specific flight operation procedure, searching for multiple nodes in the dynamic conversation graph to identify a node corresponding to the keyword in the conversation with the pilot, each node corresponding to a different flight operation procedure, searching for multiple nodes, handing off communication to a system-oriented conversation module depending on whether the node corresponding to the keyword has an owner, the owner being a specific aircraft system associated with the flight operation procedure, and determining the identity of any adjacent node to the node in the dynamic conversation graph depending on whether the node corresponding to the keyword does not have an owner, the identity of any adjacent node being communicated to the pilot by the chatbot in the conversation.

[0007] According to one embodiment and any of the embodiments described above, the interface system and method further include a system-oriented conversation module configured to provide a tokenizer with specific tokens containing specific information associated with a flight operation procedure, depending on whether the flight operation procedure is associated with a particular aircraft system among a plurality of aircraft systems. A dynamic conversation graph generator is configured to hand off communication to the system-oriented conversation module depending on whether the flight operation procedure is associated with a particular aircraft system.

[0008] According to one embodiment and any of the embodiments described above, the interface system and method further include a command encoder / decoder configured to receive and execute formatted instruction commands relating to flight operation procedures from a system-oriented conversation module for communication with a specific aircraft system among a plurality of aircraft systems, and to decode specific information from the specific aircraft system into a set of responses for communication with the system-oriented conversation module.

[0009] According to one embodiment and any of the embodiments described above, the interface system and method further include a tokenizer or a dynamic widget / form configured to perform a set of functions including receiving a token from a text-receiving Air Traffic Controller Pilot Data Link Communications (CPDLC) / ACARS (ACARS) text analyzer, and generating a specific form, widget, or both in response to the token.

[0010] According to one embodiment and any of the embodiments described above, the interface system and method further include a dynamic widget / form generation unit configured to generate specific forms, widgets, or both associated with flight operation procedures for presentation within a window on a display. The specific forms, widgets, or both are configured to interact with the pilot.

[0011] According to one embodiment and any of the embodiments described above, the interface system and method correlates a specific visual arrangement or alignment of a form, widget, or both associated with a flight operation procedure with the corresponding visual chatbot conversation text.

[0012] According to one embodiment and any of the embodiments described above, the interface system and method include at least one of the following interactions between a pilot and a specific form, widget, or both: an interaction by conversing with a chatbot, or an interaction by the pilot touching a feature of the specific form, widget, or both.

[0013] According to one embodiment and any of the embodiments described above, the interface system and method further includes an automatic speech recognition device configured to receive speech from a pilot and convert the speech into text for transmission to a chatbot. The interface system and method further includes a text-to-speech converter configured to receive text from a chatbot and convert the text into speech for transmission to the pilot via a speaker. In this case, the speaker is located in the cockpit or in the pilot's headset.

[0014] According to one embodiment and any of the embodiments described above, the interface system and method are configured to have a chatbot converse with a pilot using at least one of the following: an acoustic panel in the cockpit, a display in the cockpit, or a portable electronic device.

[0015] The aforementioned features, functions, and advantages may be realized individually in various embodiments or incorporated into yet another embodiment, and further details of such yet another embodiment can be understood by referring to the following description and drawings. [Brief explanation of the drawing]

[0016] [Figure 1] This is a schematic block diagram of one embodiment of an interface system for cockpit communication according to one embodiment of the present disclosure. [Figure 2] This is a schematic block diagram of one embodiment of an interface system for cockpit communication according to one embodiment of the present disclosure. [Figure 3]This is a flowchart of one embodiment of the operation method of the dynamic conversation graph generation unit according to one embodiment of the present disclosure. [Figure 4A] This is a flowchart of one embodiment of how a system-oriented conversation module operates according to one embodiment of the present disclosure. [Figure 4B] This is a diagram of one embodiment of a database used by a system-oriented conversation module according to one embodiment of the present disclosure. [Figure 5A] This is a flowchart of one embodiment of how the interface system shown in Figures 1 and 2 operates, according to one embodiment of the present disclosure. [Figure 5B] This is a flowchart of one embodiment of how the interface system shown in Figures 1 and 2 operates, according to one embodiment of the present disclosure. [Figure 5C] This is a flowchart of one embodiment of how the interface system shown in Figures 1 and 2 operates, according to one embodiment of the present disclosure. [Figure 6A] This table shows one embodiment of performing a conversation with a pilot according to one embodiment of the present disclosure. [Figure 6B] This table shows one embodiment of performing a conversation with a pilot according to one embodiment of the present disclosure. [Figure 7] This is an embodiment of a dynamic conversation graph according to one embodiment of the present disclosure. [Figure 8] This is a flowchart of one embodiment of the operation method of the dynamic widget / form generation unit according to one embodiment of the present disclosure. [Figure 9A] This is an embodiment of an interaction window between a chatbot and a pilot, which is generated during a conversation by a dynamic widget / form generation unit, according to one embodiment of the present disclosure. [Figure 9B] This is an embodiment of an interaction window between a chatbot and a pilot, which is generated during a conversation by a dynamic widget / form generation unit, according to one embodiment of the present disclosure. [Figure 10] This is a flowchart of one embodiment of a method for generating a historical / trained conversation database according to one embodiment of the present disclosure. [Figure 11]A flowchart of an example of a method for communication in a cab according to an example of the present disclosure. [Figure 12] An example of a system for communication in a cab according to an example of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The following detailed description of the examples refers to the accompanying drawings, which show specific examples of the present disclosure. Other examples having various structures and processes are not outside the scope of the present disclosure. Like reference numerals may represent the same elements or components in various drawings.

[0018] The present disclosure may be a system, method, and / or computer program product. The computer program product may include one or more computer-readable storage media storing computer-readable program instructions for causing a processor to execute aspects of the present disclosure.

[0019] A computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. A computer-readable storage medium may be, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any preferred combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, mechanically encoded devices (such as punch cards or grooved raised structures on which instruction commands are recorded), and any preferred combination thereof. When used herein, computer-readable storage media should not be construed as transient signals such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0020] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers these computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0021] Computer-readable program instructions for performing the operations of this disclosure include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or small talk or C. ++The source code or object code may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as C and other similar languages, and conventional procedural programming languages ​​such as C or similar languages. Computer-readable program instructions may be executed exclusively on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or exclusively on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (for example, via the Internet using an Internet service provider). In some examples, an electronic circuit (including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA)) may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions in order to customize the electronic circuit for the purpose of carrying out aspects of the disclosure.

[0022] This specification describes aspects of the disclosure with reference to flowcharts and / or block diagrams of the methods, apparatus (systems), and computer program products illustrated in the disclosure. Each block in a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, will be understood to be implementable by computer-readable program instructions.

[0023] The above computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device for the purpose of manufacturing a machine, thereby creating means for these instructions, executed via the processor of the computer or other programmable data processing device, to implement functions / operations specified in one or more blocks of a flowchart and / or block diagram. The above computer-readable program instructions may further be stored in a computer-readable storage medium capable of issuing instructions to a computer, a programmable data processing device, and / or other device in order to function in a particular manner, thereby including an article of manufacture in which the computer-readable storage medium containing the instructions includes instructions for implementing modes of functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0024] Computer-readable program instructions may further be loaded into a computer, another programmable device, or another device that generates a computer-implemented process, so that the instructions executed by the computer, other programmable device, or other device implement the functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0025] According to some embodiments of this disclosure, an interface system for cockpit communication includes a chatbot configured to converse with a pilot. The conversation includes voice communication or acoustic communication, visual communication using a display or two-way touchscreen, or both. The interface system includes a dynamic conversation graph generator configured to perform a set of functions. This set of functions includes, but is not limited to, determining flight operation procedures from the conversation with the pilot, providing the chatbot with information associated with the flight operation procedures for communication with the pilot, and responding to any requests received by the chatbot from the pilot during the conversation with the pilot. In some embodiments, the interface system also includes a tokenizer. The tokenizer is configured to convert text received from the chatbot into tokens to be sent by the tokenizer, and convert tokens received by the tokenizer into text to be sent to the chatbot to perform a conversation with the pilot.

[0026] In some embodiments, the interface system also includes a system-oriented conversation module configured to direct the conversation with the pilot to a specific aircraft system associated with the flight procedure. A dynamic conversation graph generator is configured to hand off communication to the system-oriented conversation module depending on whether the flight procedure is associated with a specific aircraft system among several aircraft systems.

[0027] In some embodiments, the interface system also further includes a dynamic widget / form generator configured to generate specific forms, widgets, or both associated with flight operation procedures for presentation within a window on the display. These specific forms, widgets, or both are configured to interact with the pilot. The pilot's interaction with the specific forms, widgets, or both includes, but is not limited to, interaction by conversing with a chatbot, or interaction by the pilot touching features of the specific forms, widgets, or both. Embodiments of the display include, but are not limited to, a cockpit touchscreen display or a display for a portable electronic device. Thus, the embodiments of the interface systems for cockpit communication described herein reduce the pilot's workload, improve situational awareness, and allow the crew to focus on the actual operation and navigation of the aircraft.

[0028] Figures 1 and 2 are schematic block diagrams of one embodiment of an interface system 100 for cockpit communication according to one embodiment of the present disclosure. The interface system 100 includes a chatbot 102 configured to converse with a pilot 103. The conversation includes voice or acoustic communication, visual communication using a display, or both. One embodiment of the execution of a conversation 602 between the chatbot 102 and the pilot 103 will be described with reference to Figures 6A to 6B. The chatbot 102 is configured to converse 602 with the pilot 103 using at least one of the following: an acoustic panel 104 in the cockpit 105, a cockpit display 106 in the cockpit 105, or a portable electronic device 108 such as a tablet computer, electronic flight bag (EFB), or similar device. The acoustic panel 104 includes a microphone 110 for receiving voice or acoustic signals from the pilot 103 and a speaker 112 for transmitting voice or acoustic signals to the pilot 103. In some embodiments, the cockpit display 106 includes a touchscreen 114 configured to receive input from a pilot 103 touching the cockpit display 106, as described herein. The portable electronic device 108 also includes a touchscreen 116 configured to receive input by touch.

[0029] In the embodiments shown in Figures 1 and 2, the display computer 118 interconnects the chatbot 102 to the cockpit display 106. The display computer 118 is configured to convert text signals from the chatbot 102 into video signals for display on the cockpit display 106, and to convert input signals from the pilot 103 to the touchscreen 114 into text signals usable by the chatbot 102.

[0030] In the embodiments shown in Figures 1 and 2, the aircraft interface device 120 interconnects the chatbot 102 to the portable electronic device 108. The aircraft interface device 120 is configured to convert text signals from the chatbot 102 into video signals for display on the portable electronic device 108, and to convert input signals from the pilot to the touchscreen 116 into text signals usable by the chatbot 102.

[0031] The interface system 100 also includes an automatic speech recognition (ASR) device 122 configured to receive voice from the pilot 103 via a microphone 110 and convert that voice into text for transmission to the chatbot 102. The interface system 100 further includes a text-to-speech (TTS) converter 124 configured to receive text from the chatbot 102 and convert that text into voice or sound for transmission to the pilot 103 via a speaker 112. The speaker 112 is located in the cockpit 105 or in the pilot 103's headset.

[0032] The interface system 100 further includes a tokenizer 126 configured to convert text received from the chatbot 102 into tokens to be sent by the tokenizer 126 in order to have a conversation 602 with the pilot 103, and to convert the tokens received by the tokenizer 126 into text 126 to be sent to the chatbot 102. The tokenizer 126 breaks down a sentence or sequence of words or phrases spoken by the pilot 103 into elements called tokens, such as words, keywords, phrases, symbols, or other elements. Tokens can be individual words, phrases, or entire sentences.

[0033] The interface system 100 also includes a dynamic conversation graph generator 128 configured to perform a set of functions 130. See also Figure 3, which is a flowchart of one embodiment of a method 300 of operation of the dynamic conversation graph generator 128 according to one embodiment of the present disclosure. The exemplary method 300 includes one embodiment of the set of functions 130. Method 300 or the set of functions 130 includes, but is not limited to, receiving one or more tokens 302 from the tokenizer 126 in block 304, determining a flight operation procedure from the tokens 302 corresponding to the conversation 602 with the pilot 103 in block 306, providing the chatbot 102 with information associated with the flight operation procedure for communication to the pilot 103 in block 308, and responding to any requests received by the chatbot 102 from the pilot 103 during the conversation 602 with the pilot 103 in block 310. Further examples of the functions of the set of functions 130 performed by the dynamic conversation graph generation unit 128 will be described with reference to Figures 5A-5C and 6A-6B.

[0034] The dynamic conversation graph generation unit 128 accesses the historical / trained conversation database 132 to perform a set of functions 130. One embodiment of the method 1000 for generating the historical / trained conversation database 132 will be described with reference to Figure 10. The historical / trained conversation database 132 contains and / or is trained to generate a dynamic conversation graph in order to have a conversation with the pilot 103 (for example, exemplary conversation 602 (Figures 6A-6B)). One embodiment of the dynamic conversation graph 700 is shown in Figure 7 and can be used to have a conversation such as exemplary conversation 602 in Figures 6A-6B.

[0035] The interface system 100 further includes a system-oriented conversation module 134 configured to direct conversation with the pilot 103 to a specific aircraft system 136 associated with a flight operation procedure. When used herein, aircraft system 136 also includes subsystems of aircraft system 136. The system-oriented conversation module 134 is configured to provide a tokenizer 126 with specific tokens containing specific information associated with a flight operation procedure, depending on which aircraft system 136 among a plurality of aircraft systems 136a to 136n the flight operation procedure is associated with. A dynamic conversation graph generator 128 is configured to hand off communication to the system-oriented conversation module 134 depending on which aircraft system 136 the flight operation procedure is associated with. Embodiments of aircraft systems 136a to 136n include, but are not limited to, a flight management system (FMS) 136a, a flight computer (FC) 136b, an electronic checklist 136c performed by a display system, a health management system 136d, and other airborne systems 136n or subsystems.

[0036] The system-oriented conversation module 134 is configured to perform a set of functions 138. Referring also to Figure 4A, Figure 4A is a flowchart of one embodiment of a method 400 of operation of the system-oriented conversation module 134 according to one embodiment of the present disclosure. The exemplary method 400 includes one embodiment of the set of functions 138. Method 400 or the set of functions 138 will be described with reference to, but will not be limited to, blocks 402-416. Block 402 includes method 400 or the set of functions 138 receiving one or more tokens 403 from the tokenizer 126. The tokens 403 contain information associated with a flight operation procedure.

[0037] In block 404, a method or set of functions 138 includes using tokens 403 to query a database 140 related to flight operation procedures via a system-oriented conversation module 134. One embodiment of the database 140 used by the system-oriented conversation module 134 will be described with reference to Figure 4B.

[0038] Block 406 includes a method 400 or a set of functions 138 that determines a specific aircraft system 136 from among a plurality of aircraft systems 136a to 136n for performing a flight operation procedure.

[0039] Block 408 includes a method 400 or a set of functions 138 performing a sequence of steps 410 (Figure 4B) associated with a flight operation procedure and fetching a set of commands 412 that describe the flight operation procedure stored in the database 140.

[0040] In block 414, method 400 or a set of functions 138 includes communicating command 412 to a command encoder / decoder 142 via a system-oriented conversation module 134 for communication with a specific aircraft system 136. The specific aircraft system 136 is configured to execute command 412 and return information related to flight operation procedures similar to those described with reference to the embodiments in Figures 6A-6B and 9A-9B.

[0041] Block 416 includes a method 400 or a set of functions 138 receiving information related to flight operation procedures from a specific aircraft system via a command encoder / decoder 142 and a system-oriented conversation module 134.

[0042] Block 418 includes a method 400 or a set of functions 138 in which a system-oriented conversation module 123 communicates information related to flight operation procedures to a pilot 103 via a tokenizer 126 and a chatbot 102.

[0043] In some embodiments, the database 140 includes one or more subsystem-specific voice / TTS databases 140a to 140n. The database 140 is generated for a desired flight operation procedure and is field-loadable. See also Figure 4B, which is a diagram of one embodiment of the database 140 used by a system-oriented conversation module 134 according to one embodiment of the present disclosure. The exemplary database 140 shown in Figure 4B includes one embodiment of a sequence of steps 410 for presenting an approach checklist 419 by an electronic checklist 136c on a display system 420 (e.g., a cockpit display 106, a portable electronic device 108, or both). The database 140 is used by the system-oriented conversation module 134 in translating tokens 403 into a sequence of steps 410. The sequence of steps 410 includes a mnemonic / opcode and operand / formatted instruction 422, which are configured to be interpreted by a command encoder / decoder 142. The response from the command encoder / decoder 142 is used by the system-oriented conversation module 134 to fetch a token from the database 140 and transfer that token to the tokenizer 126 for communication in the conversation 602 with the pilot 103.

[0044] Referring back to Figures 1 and 2, the interface system 100 further includes a command encoder / decoder 142. The command encoder / decoder 142 is configured to receive and execute mnemonics / opcodes and operands / formatted instruction commands from the system-oriented conversation module 134. Thereafter, the command encoder / decoder 142 encodes avionics data and communicates it with a specific aircraft system 136 among a plurality of systems 136a-136n, as shown in Figures 1 and 2. The command encoder / decoder 142 is also configured to decode specific information from a specific aircraft system 136 among the plurality of systems 136a-136n into a set of responses for communication with the system-oriented conversation module 134 and execution of a set of functions 138.

[0045] Referring to Figure 2, the interface system 100 further includes a dynamic widget / form generator 144 configured to generate specific forms, widgets, or both associated with flight operation procedures for presentation within a display window (e.g., the cockpit display 106, the display of the portable electronic device 108, or both). The dynamic widget / form generator 144 generates specific forms, widgets, or both using a keyword / phrase-to-dynamic widget / form database 146. A method of operation of the dynamic widget / form generator 144 and an embodiment of using the keyword / phrase-to-dynamic widget / form database 146 to generate specific forms, widgets, or both will be described with reference to Figures 8 and 9A and 9B. The specific forms, widgets, or both are configured to interact with the pilot 103. As shown in one embodiment of Figure 2, the dynamic widget / form generator 144 is coupled to the cockpit display 106 by a display computer 118. The cockpit display 106 is also coupled to the chatbot 102, as previously described and shown in Figure 1. The display computer 118 is configured to convert signals from the dynamic widget / form generation unit 144 into video signals for presentation on the cockpit display 106, and to convert input signals entered by the pilot 103 into signals usable by the dynamic widget / form generation unit 144 for communication with a specific aircraft system 136. This communication is carried out using a tokenizer 126, a system-oriented conversation module 134, and a command encoder / decoder 142.

[0046] The dynamic widget / form generation unit 144 is coupled to the portable electronic device 108 by the aircraft interface device 120. The portable electronic device 108 is also coupled to the chatbot 102, as previously described and shown in Figure 1. The aircraft interface device 120 is configured to convert signals from the dynamic widget / form generation unit 144 into video signals for presentation on the portable electronic device 108, and to convert input signals entered by the pilot 103 into signals usable by the dynamic widget / form generation unit 144 for communication with a specific aircraft system 136. This communication is performed using a tokenizer 126, a system-oriented conversation module 134, and a command encoder / decoder 142.

[0047] In some embodiments, the dynamic widget / form generation unit 144 is configured to perform a set of functions 148, which include, but are not limited to, receiving a token 149 from a tokenizer 126, or receiving a token 152 from a data link / controller-pilot data link communication (DL / CPDLC) / air traffic controller-pilot data link communication (CPDLC) / ACARS (ACARS) text analyzer 150 that receives text 156, and generating a specific form, widget, or both in accordance with the token 149 or 152. The DL / CPDLC / ACARS text analyzer 150 receives text 156 from a ground base station or other source.

[0048] An embodiment of the method 800 for the operation of the dynamic widget / form generation unit 144 will be described with reference to Figure 8. Embodiments of forms, widgets, or both presented in windows 904a to 904d of the display 900 are shown in Figures 9A to 9B and will be described with reference to Figures 9A to 9B. The visual arrangement or alignment of a particular form, widget, or both associated with a flight operation procedure correlates with the corresponding visual chatbot conversation text 905, as shown in Figures 9A to 9B. The pilot's interaction with a particular form, widget, or both includes at least one of the following: interaction by conversing with the chatbot 102, or interaction by the pilot 103 touching a particular form, widget, or both on the touchscreen 114 or 116, as described with reference to Figures 8 and 9A to 9B.

[0049] Figures 5A to 5C are flowcharts of one embodiment of a method 500 for the operation of the interface system 100 of Figures 1 and 2, according to one embodiment of the present disclosure. The flowchart is divided into functions or operations that can be performed by the various components of Figures 1 and 2. In some embodiments, a set of functions 130 that can be performed by the dynamic conversation graph generation unit 128 and a set of functions 138 that can be performed by the system-oriented conversation module 134 are embodied in method 500. Block 502 includes method 500 receiving text by a display. According to some embodiments, the text is received by the pilot 103 via the touchscreen 114 of the cockpit display 106 or the touchscreen 116 of the display of a portable electronic device 108, as previously described.

[0050] Furthermore, or instead of receiving text via a display, block 504 includes method 500 receiving audio from pilot 103. In some embodiments, block 506 includes method 500 presenting audio text on a display.

[0051] In block 508, method 500 includes receiving text by chatbot 102. In block 510, method 500 includes converting the text into tokens by tokenizer 126.

[0052] According to several embodiments, a set of functions 130 that can be performed by the dynamic conversation graph generation unit 128 includes blocks 512-526 and blocks 548-560 in Figure 5B. Block 512 includes the method 500 or the set of functions 130 receiving tokens by the dynamic conversation graph generation unit 128.

[0053] Referring also to Figures 6A and 6B, Figures 6A and 6B are Table 600 showing one embodiment of the execution of a conversation 602 with the pilot 103 according to one embodiment of the present disclosure. Block 514 (Figure 5A) includes a method 500 or a set of functions 130 performed by the dynamic conversation graph generation unit 128, which uses tokens from the tokenizer 126 to detect a keyword 604 (Figure 6A) or a set of keywords in the conversation 602 with the pilot 103. The keyword 604 identifies a particular flight operation procedure. In one embodiment of Figure 6A, the keyword 604 or a set of keywords is the “approach phase” that identifies a particular flight operation procedure.

[0054] See also Figure 7, which is an embodiment of a dynamic conversation graph 700 according to one embodiment of the present disclosure. Block 516 further includes method 500 or a set of functions 130 searching several nodes 702-706 (Figure 7) of the dynamic conversation graph 700 to identify node 702 corresponding to keyword 604 (Figure 6A) in conversation 602 with pilot 103. Each node 702-706 of the dynamic conversation graph 700 corresponds to a different flight operation procedure. In the embodiments of Figures 6A-6B and Figure 7, keyword 604 "approach phase" corresponds to flight operation procedure node 702 in Figure 7.

[0055] Block 518 further includes a method 500 or a set of functions 130 performed by the dynamic conversation graph generation unit 128 to determine whether a node 702 corresponding to a flight operation procedure has an owner, i.e., a specific aircraft system 136, for performing the flight operation procedure.

[0056] Block 520 further includes a method 500 or a set of functions 130 performed by the dynamic conversation graph generation unit 128 handing off communication to a system-oriented conversation module 134 (block 522), depending on whether the node 702 corresponding to keyword 604 has an owner. The owner is a specific aircraft system 136 associated with a flight operation procedure, or a specific aircraft system 136 that performs a flight operation procedure. If the node 702 corresponding to keyword 604 does not have an owner, the method 500 or the set of functions 130 proceeds to block 524.

[0057] In block 524, a method 500 or a set of functions 130 performed by the dynamic conversation graph generation unit 128 further includes determining the identity of any adjacent node 704 to node 702 in the dynamic conversation graph 700, in response that node 702 corresponding to keyword 604 in block 520 does not have an owner. In one embodiment shown in Figure 7, adjacent nodes 704 to node 702 corresponding to keyword 604 and flight operation procedure include identity 606, namely, “Approach Chart” 606a (adjacent node 704a), “Approach Clearance” 606b (adjacent node 704b), “Approach Checklist” 606c (adjacent node 704c), and “Approach Briefing” 606d (adjacent node 704d). The identity 606 of adjacent node 704 is communicated to the pilot 103 by the chatbot 102 in conversation 602, as shown in the exemplary conversation in Figure 6A and in block 526 in Figure 5B.

[0058] In block 526, a method 500 or a set of functions 130 performed by the dynamic conversation graph generation unit 128 includes generating a response that includes the identity 606 of adjacent nodes 704, depending on whether node 702, which corresponds to keyword 604 and flight operation procedure, has adjacent nodes 704a to 704d. The response generated by the dynamic conversation graph generation unit 128 includes a plurality of tokens, each token corresponding to the identity 606a to 606d of adjacent nodes 704a to 704d.

[0059] In block 528, the tokenizer 126 converts tokens into text. In block 530, the chatbot 102 constructs one or more sentences using the text from the tokenizer 126, as shown in Figure 6A. In block 532, the text-to-speech (TTS) converter 124 converts the text into speech. As shown in Figure 5B, the speech is transmitted to the pilot 103 by the speaker 112 in block 532, or the text is presented on the display 106 or 108 in block 536, or both. Performing a conversation with the pilot 103 is shown in block 514 to include the dynamic conversation graph generator detecting one or more keywords “approach phase” in the conversation with the pilot using tokens from the tokenizer. From such detection, nodes corresponding to the keywords in the conversation are determined to be related to a specific aircraft system associated with a flight operation procedure. During the execution of the conversation, the dynamic conversation graph generator then generates a response containing tokens corresponding to identity transformed by the tokenizer, which constitute one or more sentences that are output or transmitted to the pilot as voice and / or displayed on the cockpit display. The pilot can then request an approach checklist for the display (which may be further confirmed by input from the pilot using voice or touchscreen).

[0060] In one embodiment shown in Figures 5A-5B and 6A-6B, the conversation 602 with pilot 103 continues as previously described. Block 538 includes method 500 receiving a text response entered by pilot 103 using touchscreen 114 or 116 on a display (e.g., cockpit display 106 or the display of portable electronic device 108). Alternatively or further, block 540 includes method 500 receiving a voice response from pilot 103 by an automatic speech recognition (ASR) device 122. In one embodiment shown in Figure 6A, the response 608 is either a text response, a voice response, or both. Block 542 shows the ASR device 122 converting the voice to text.

[0061] Block 544 includes method 500 receiving a text response 608 from the ASR device 122, the display, or both, via the chatbot 102. Block 546 includes method 500 converting the text response 608 into tokens via the tokenizer 126.

[0062] In block 548, method 500 includes receiving a token by the dynamic conversation graph generator 128. In block 550, method 500 or a set of functions 130 includes using the token to detect another keyword 620 or more keywords corresponding to another flight operation procedure in response 608 from pilot 103. In one embodiment of conversation 602 in Figure 6A, the other keyword 620 or more keywords is “approach checklist” in response 608 from pilot 103.

[0063] In block 552, a method 500 or a set of functions 130 performed by the dynamic conversation graph generation unit 128 further includes searching for adjacent nodes 704a-704d of the dynamic conversation graph 700 to identify a specific adjacent node 704c corresponding to another keyword 620 or more keywords during the conversation 602 with the pilot 103. In one embodiment of Figure 7, the keyword 620 "approach checklist" corresponds to adjacent node 704c.

[0064] Block 554 further includes a method 500 or a set of functions 130 determining whether a particular adjacent node 704c corresponding to a flight operation procedure “Approach Checklist” has an owner, i.e., an aircraft system 136 for performing the flight operation procedure. In one embodiment of Figure 7, the adjacent node 704c has an owner node 706d “Electronic Checklist” corresponding to an electronic checklist 136c performed by the display system of Figures 1 and 2.

[0065] In block 556, method 500 or a set of functions 130 hand off communication to a system-oriented conversation module 134 depending on whether a particular adjacent node 704c has an associated owner node 706d (block 558). Alternatively, method 500 proceeds to block 560 depending on whether a particular adjacent node 704 does not have an owner i.e., an aircraft system 136 for performing flight operation procedures. In block 560, the conversation continues depending on whether it has ended or whether pilot 103 is continuing the conversation 602, with the voice being received by the ASR device 122 and / or the text being received by a display (e.g., a cockpit display 106 or a display on a portable electronic device 108).

[0066] Figure 5C shows an embodiment of a conversation continuing using a particular aircraft system 136 according to one embodiment of the present disclosure. Block 562 includes Method 500 receiving text via a display. According to some embodiments, the text is received via a touchscreen 114 of a cockpit display 106 or a touchscreen 116 of a portable electronic device 108 display. Furthermore, or instead of receiving text via a display, Block 564 includes Method 500 receiving audio from an ASR device 122. Block 566 shows the ASR device 122 converting the audio to text. In some embodiments, Method 500 includes presenting the audio text on a display.

[0067] In block 568, method 500 includes receiving text by chatbot 102. In block 570, method 500 includes converting the text into tokens by tokenizer 126. In block 572, method 500 includes receiving tokens by system-oriented conversation module 134, as shown in one embodiment of Figure 1. The tokens contain information associated with a flight operation procedure from a conversation with pilot 103. The system-oriented conversation module 134 is configured to use the tokens to identify a specific aircraft system 136 for performing the flight operation procedure. In block 572, as shown in one embodiment of Figure 1, mnemonics / opcodes and operands / formatted instructions related to the flight operation procedure are fetched from database 140 by system-oriented conversation module 134. The mnemonics / opcodes and operands / formatted instructions are sent by system-oriented conversation module 134 to command encoder / decode 142.

[0068] In block 574, the command encoder / decoder 142 is configured to receive and execute formatted instruction commands related to flight operation procedures from the system-oriented conversation module for communication with a specific aircraft system 136 among multiple aircraft systems 136a to 136n. The command encoder / decoder 142 interprets the mnemonic / opcode and operand / formatted instruction commands and communicates avionics data with the specific aircraft system 136 among multiple flight systems 136a to 136n in order to execute the flight operation procedures.

[0069] Block 576 includes Method 500 performing an action by a specific aircraft system 136 in accordance with encoded information. An embodiment of an action performed by a specific aircraft system 136 and communication with the pilot 103 is shown in conversation 602 of one embodiment in Figure 6B. In one embodiment in Figure 6B, the specific aircraft system 136 (e.g., a display system) includes presenting a checklist on the display (612), verifying that the landing lights have been switched on by the pilot 103 and presenting a confirmation on the display (614), verifying the altimeter setting set by the pilot 103 and presenting the altimeter setting on the display (616), and reading the navigation frequency of the navigation radio set by the pilot 103 (618).

[0070] In block 578, method 500 includes generating a response containing arbitrary information or data by a specific aircraft system 136 for communication to pilot 103. In block 580, the response is transmitted by the specific aircraft system 136 to command encoder / decoder 142.

[0071] In block 582, the command encoder / decoder 142 is configured to decode specific information from a particular aircraft system 136 into a set of responses for communication with the system-oriented conversation module 134. In block 584, the command encoder / decoder 142 generates one or more responses for the mnemonic / opcode and operand / formatted instruction to be executed by the particular aircraft system 136. One or more responses are sent to the system-oriented conversation module 134. In block 586, the system-oriented conversation module 134 receives one or more responses for the mnemonic / opcode and operand / formatted instruction and fetches one or more corresponding tokens from the database 140. The system-oriented conversation module 134 sends one or more tokens to the tokenizer 126, as also shown in one embodiment in Figures 1 and 2. In block 588, the tokenizer 126 converts the tokens to text. The tokenizer 126 sends the text to the chatbot 102 for communication with the pilot 103.

[0072] In block 590, the chatbot 102 constructs one or more sentences using tokens received from the tokenizer 126. An example of sentences constructed by the chatbot 102 during an exemplary conversation 602 with the pilot 103 is shown in Figure 6B. The chatbot 102 sends one or more sentences to the TTS converter 124.

[0073] In block 591, the conversion of text in a document into speech is performed by the TTS converter 124. In block 592, the transmission of the speech to the pilot 103 is performed by the TTS converter 124 using the speaker 112. In some embodiments, as previously described, the speaker 112 is mounted on the acoustic panel 104 in the cockpit 105, on a headset worn by the pilot 103, or both. In block 593, the conversation 602 with the pilot 103 ends, or method 500 returns to block 564 to continue the conversation 602 and receive further speech from the pilot 103. See also Figure 6B, which shows one embodiment of a continuing conversation 602 between the pilot 103 and a specific aircraft system 136 using the chatbot 102, a system-oriented conversation module 134, and a command encoder / decoder 142.

[0074] In some embodiments, in block 594 of Figure 5C, text or sentences from the chatbot 102 are also presented on a display (e.g., the cockpit display 106, the display of the portable electronic device 108, or both). In block 596, the conversation 602 ends, or method 500 returns to block 562 to receive further text entered by the pilot 103 on the display.

[0075] Figure 8 is a flowchart of one embodiment of Method 800 for the operation of the Dynamic Widget / Form Generation Unit 144 according to one embodiment of the present disclosure. Block 802 includes Method 800 receiving tokens 149, 152 from a tokenizer 126 or an Air Traffic Controller Pilot Data Link Communications (CPDLC) / ACERS (ACARS) text analyzer 150 that receives text 156. Method 800 also includes converting text 156 into one or more forms, widgets, or both. Text 156 is a text token 152, or a communications (COMM) token received from the ACERS (ACARS) and / or Data Link / Controller-Pilot Data Link Communications (DL / CPDLC) text analyzer 150, or a TTS token 149 received from the System-Oriented Conversation Module 134 or Dynamic Conversation Graph Generation Unit 128 via the tokenizer 126. One or more tokens are converted into graphical opcodes / operands / formatted instructions by the dynamic widget / form generation unit 144 using the keyword / phrase-to-dynamic form / widget database 146. See also Figures 9A and 9B, which are embodiments of chatbot windows 902a-902d and pilot interaction windows 904a-904d generated by the dynamic widget / form generation unit 144 during a conversation, according to one embodiment of the present disclosure. Figures 9A and 9B also show tables 906a-906d containing keywords 908a-908d and corresponding widget definition files 910a-910d. Tables 906a-906d are stored in the keyword / phrase-to-dynamic widget / form database 146, which are used by the dynamic widget / form generation unit 144 to generate dynamic forms, widgets, or both, as shown in the embodiments in chatbot windows 902a-902d and pilot interaction windows 904a-904d.

[0076] Block 804 includes Method 800 generating panels 901 for pilot interaction windows 904a-904d and placing widgets 903 within panels 901. Block 806 includes Method 800 placing panels 901 within pilot interaction windows 904a-904d so as to align with chatbot responses within chatbot windows 902a-902d. Block 808 includes Method 800 processing graphics and transmitting those graphics to a display computer 118 for presentation on a cockpit display 106, and / or transmitting those graphics to an aircraft interface device 120 for presentation on a portable electronic device 108.

[0077] In block 810, method 800 includes receiving a response or input from pilot 103. As previously described, the response or input takes the form of voice received by system 100 from pilot 103, and / or pilot 103 interacting with the touchscreen 114 of the cockpit display 106 or the touchscreen 116 of the portable electronic device 108. In block 812, method 800 returns to chatbot 102 and ASR device 122 to receive further responses or input from pilot 103.

[0078] Figure 10 is a flowchart of one embodiment of Method 1000 for generating a historical / trained conversation database 132 according to one embodiment of the present disclosure. Block 1002 includes Method 1000 receiving conversation text. For example, the text is received from a pilot or another user. Block 1004 includes Method 1000 converting the text into tokens using a tokenizer.

[0079] Block 1006 includes method 1000, which involves generating a word bag. Generating a word bag is the process of representing text data when modeling text using a machine learning algorithm. A word bag includes a vocabulary of known words, such as approach, phrase, weather, destination, airport, radio, frequency, tuning, etc.

[0080] Block 1008 includes method 1000, which involves performing characterization using document sources 1010. Each word or phrase is called a gram. Generating a vocabulary of pairs of two words (n=2) is now called a bigram model. Examples include approach briefing, flight phase, approach checklist, destination airport, destination weather, etc. Examples of document sources 1010 for performing characterization include, but are not limited to, aircraft system / subsystem lists 1011, airport directories 1012, aircraft flight manuals (AFM) 1014, quick reference handbooks (QRH) 1016, dispatch files 1018, various checklists 1020, and aeronautical charts 1022. Examples of aircraft system / subsystem lists 1011 include, but are not limited to, displays, flight control systems (FCS), flight management systems (FMS), maintenance systems, etc.

[0081] In block 1024, method 1000 involves performing a word-to-vector (Word2Vec) conversation. Associative words are constructed from a large corpus of words. A Word2Vec conversation represents each distinguishable word with a specific numerical list called a vector. In a Word2Vec conversation, each word or phrase arising from document source 1010, and the relationships between the words or phrases characterized in block 1008, are represented numerically as numerical arrays.

[0082] Block 1026 includes method 1000, which performs topic modeling. Topic modeling facilitates the discovery of semantic structures within text bodies by identifying topics by comparing the distances between vectors in a word vector space. Topic modeling also includes extracting key topics from a dataset. Examples of semantic structures include, but are not limited to, weather, navigation, aircraft status, and standard operating procedures (SOPs).

[0083] Block 1028 includes method 1000, which clusters or groups words or phrases into various clusters 1030-1034. Related words or phrases are grouped into the same cluster. Each cluster is then associated with a topic, and similar measures are used, for example, to cluster related words, phrases, or documents together. Examples of clusters include, but are not limited to, AFM cluster 1030, QRH cluster 1032, and checklist cluster 1034. For example, if the conversation text is "Yes bot, can I have an approach checklist?", there is a high probability, e.g., about an 80% chance, of hitting checklist cluster 1034.

[0084] In block 1036, method 1000 includes mapping one or more keywords in the conversation text to the owner of an avionics application. In an embodiment where the keyword is “approach checklist”, the conversation is mapped to a display system which is the owner of an avionics application for displaying an approach checklist.

[0085] In block 1038, method 1000 includes performing graph generation. It constructs the relationships between data elements within clusters, data elements between clusters, and owners of avionics applications and the required data elements in the form of a graph represented by a mesh including nodes and edges. This graph is stored and represented as a field-loadable historical / trained conversation database 132.

[0086] Figure 11 is a flowchart of one embodiment of Method 1100 for cockpit communication according to one embodiment of the present disclosure. In some embodiments, Method 1100 is embodied in and executed by System 100 in Figures 1 and 2. In some embodiments, Method 1100 also includes a set of functions 130 executed by a dynamic conversation graph generator 128, a set of functions 138 executed by a system-oriented conversation module 134, a set of functions 148 executed by a dynamic widget / form generator 144, and functions executed by other components of System 100, as described herein.

[0087] In block 1102, method 1100 includes conducting a conversation with a pilot by a chatbot. The conversation includes voice communication, visual communication using a display, or both.

[0088] In block 1104, method 1100 includes a dynamic conversation graph generator determining a flight operation procedure from a conversation with a pilot. In block 1106, method 1100 includes a dynamic conversation graph generator providing a chatbot with information associated with the flight operation procedure for communication with the pilot. In block 1108, method 1100 includes a dynamic conversation graph generator responding to any requests received by the chatbot during a conversation with a pilot.

[0089] In block 1110, method 1100 further includes a dynamic conversation graph generator handing off communication to a system-oriented conversation module, depending on whether the flight operation procedure is associated with a specific aircraft system i.e., the owner of the avionics application. In block 1112, method 1100 includes guiding the conversation with the pilot to a specific aircraft system, for example, one of the aircraft systems 136a to 136n in Figures 1 and 2.

[0090] Block 1114 includes method 1100 providing a chatbot with information associated with flight operation procedures via a system-oriented conversation module or a specific aircraft system in order to communicate with a pilot.

[0091] Block 1116 includes Method 1100 generating a specific form, widget, or both associated with a flight operation procedure for presentation within a window on a display. The specific form, widget, or both are configured to interact with a pilot. The pilot's interaction with the specific form, widget, or both includes at least one of the following: interaction by conversing with a chatbot, or interaction by the pilot touching features of the specific form, widget, or both.

[0092] In block 1118, method 1100 includes presenting a form, a widget, or both on a display for interaction with a pilot. According to one embodiment in Figures 1 and 2, the display includes a cockpit display 106, a portable electronic device 108 display, or both. Pilot interaction includes the pilot touching the touchscreen 114 of the cockpit display 106, the touchscreen 116 of the portable electronic device 108, or both. In some embodiments, pilot interaction also includes voice communication with the form, widget, or both using the acoustic panel 104, microphone 110, speaker 112, ASR device 122, TTS converter 124, chatbot 102, and tokenizer 126 in Figures 1 and 2.

[0093] Figure 12 shows one embodiment of a system 1200 for cockpit communication according to one embodiment of the present disclosure. In some embodiments, the system 100 of Figures 1 and 2 is embodied in one or more systems similar to or identical to system 1200. In some embodiments, one or more components of system 100 of Figures 1 and 2, such as the chatbot 102, tokenizer 126, dynamic conversation graph generator 128, system-oriented conversation module 134, command encoder / decoder 142, aircraft system 136, dynamic widget / form generator 144, DL / CPDLC / ACARS text analyzer 150, aircraft interface device 120, and display computer 118, are embodied in one or more systems similar to or identical to system 1200. In some embodiments, at least some of methods 300, 400, 500, 800, 1000, and 1100 are embodied in or performed within one or more systems similar to or identical to system 1200.

[0094] System 1200 includes a processing circuit 1202 and a memory 1204 associated with the processing circuit 1202. Memory 1204 includes a computer-readable program instruction 1206 which, when executed by the processing circuit 1202, causes the processing circuit 1202 to execute a set of functions 1208. In some embodiments, the set of functions 1208 includes a set of functions 130 that can be executed by a dynamic conversation graph generation unit 128, a set of functions 138 that can be executed by a system-oriented conversation module 134, and / or a set of functions 148 that can be executed by a dynamic widget / form generation unit 144. The set of functions 130, 138, and 148 is embodied in one or more systems similar to or the same as system 1200. In some embodiments, the set of functions 1208 includes at least some of methods 300, 400, 500, 800, 1000, and 1100. These methods are embodied and executed by one or more systems similar to or identical to system 1200.

[0095] In some embodiments, the system 1200 also includes one or more input / output devices 1210. The input / output devices 1210 include individual input devices, output devices, or combinations of input and output devices. Examples of input / output devices 1210 include, but are not limited to, one or more display devices, touchscreens, microphones, speakers, keyboards or keypads, pointing devices, and devices configured to read or access computer-readable program instructions of a computer program product 1212 similar to those described herein. Any of methods 300, 400, 500, 800, 1000, and / or 1100 may be embodied in a computer program product 1212 that is read by the input / output device 1210 and stored in memory 1204.

[0096] Furthermore, this disclosure includes embodiments as provided for in the following clauses. Article 1. An interface system for communication in the cockpit, A chatbot configured to converse with a pilot, wherein the conversation includes voice communication, visual communication using a display, or both; It comprises a dynamic conversation graph generation unit configured to perform a set of functions, and the set of functions is Determining the flight procedure from the aforementioned conversation with the pilot, To communicate with the pilot, provide the chatbot with information associated with the flight procedure, and An interface system that includes responding to any requests received by the pilot by the chatbot during the conversation with the pilot. Article 2. The system further includes a tokenizer, the tokenizer being: Converting the text received from the chatbot into a token to be sent by the tokenizer, and The interface system according to Clause 1, configured to perform the task of converting tokens received by the tokenizer into text to be sent to the chatbot for the purpose of having the conversation with the pilot. Article 3. The set of functions performed by the dynamic conversation graph generation unit is: Using the tokens from the tokenizer, detect keywords in the conversation with the pilot, wherein the keywords identify a specific flight procedure. To identify a node corresponding to the keyword in the conversation with the pilot, the search involves searching for multiple nodes in a dynamic conversation graph, where each node corresponds to a different flight operation procedure. Depending on whether the node corresponding to the keyword has an owner, the handoff of communication to a system-oriented conversation module, wherein the owner is a specific aircraft system associated with the flight operation procedure, and The interface system according to Clause 1 or 2, further comprising determining the identity of any adjacent node to the node in the dynamic conversation graph, in accordance with whether the node corresponding to the keyword does not have an owner, wherein the identity of any adjacent node is communicated to the pilot by the chatbot in the conversation. Article 4. The set of functions performed by the dynamic conversation graph generation unit is: Searching for adjacent nodes in the dynamic conversation graph to identify a specific adjacent node corresponding to another keyword during the conversation with the pilot, and The interface system according to any one of clauses 1 to 3, further comprising handing off communications to the system-oriented conversation module depending on whether the particular adjacent node has an associated owner. Article 5. The interface system according to any one of Clauses 1 to 4, further comprising a system-oriented conversation module configured to provide the tokenizer with a specific token containing specific information associated with the flight operation procedure, depending on whether the flight operation procedure is associated with a particular aircraft system among a plurality of aircraft systems, wherein the dynamic conversation graph generator is configured to hand off communications to the system-oriented conversation module depending on whether the flight operation procedure is associated with the particular aircraft system. Article 6. The system-oriented conversation module is configured to perform a set of functions, and the set of functions is Receiving a token from the tokenizer, wherein the token includes information associated with the flight operation procedure, Query the database regarding the aforementioned flight operation procedure. Determining the specific aircraft system among the plurality of aircraft systems for performing the aforementioned flight operation procedure, and An interface system according to any one of Clauses 1 to 6, comprising communicating information relating to the flight operation procedure with the pilot via the tokenizer and the chatbot. Article 7. The command encoder / decoder further comprises, For communication with the specific aircraft system among the plurality of aircraft systems, the system-oriented conversation module receives and executes formatted instruction commands related to the flight operation procedure, and An interface system according to any one of Clauses 1 to 6, configured to perform the task of decoding the specific information from the specific aircraft system into a set of responses for communication with the system-oriented conversation module. Article 8. The unit further comprises a dynamic widget / form generation unit configured to perform a set of functions, the set of functions being: Receiving the token from the tokenizer or the Air Traffic Controller Pilot Data Link Communication (CPDLC) / ACARS (ACARS) text analyzer that receives the text, and An interface system as described in any one of Clauses 1 to 7, which includes generating a specific form, widget, or both in accordance with the aforementioned token. Article 9. The interface system according to any one of the clauses 1 to 8, further comprising a system-oriented conversation module configured to guide the conversation with the pilot to a specific aircraft system associated with the flight operation procedure, wherein the dynamic conversation graph generator is configured to hand off communication to the system-oriented conversation module depending on whether the flight operation procedure is associated with the specific aircraft system among a plurality of aircraft systems. Article 10. The system further comprises a dynamic widget / form generation unit configured to generate specific forms, widgets, or both associated with the flight operation procedure for presentation within a window on the display, The interface system described in any one of Clauses 1 to 9, wherein the specific form, widget, or both thereof is configured to interact with the pilot. Article 11. An interface system according to any one of Clauses 1 to 10, wherein a specific visual arrangement or alignment of a form, widget, or both associated with the aforementioned flight operation procedure correlates with the corresponding visual chatbot conversation text. Article 12. An interface system according to any one of the clauses 1 to 11, wherein the pilot's interaction with a specific form, widget, or both thereof includes at least one of the following: interaction by conversing with the chatbot, or interaction by the pilot touching the features of the specific form, widget, or both thereof. Article 13. An automatic speech recognition device configured to receive voice from the pilot and convert the voice into text for transmission to the chatbot, and The interface system according to any one of Clauses 1 to 12 further comprises a text-to-speech converter configured to receive text from the chatbot and convert the text into speech for transmission to the pilot by a speaker, wherein the speaker is located in the cockpit or in the pilot's headset. Article 14. The interface system according to any one of Clauses 1 to 13, wherein the chatbot is configured to conduct the conversation with the pilot using at least one of the following: an acoustic panel in the cockpit, a display in the cockpit, or a portable electronic device. Article 15. A method for communication in the cockpit, The chatbot engages in conversation with a pilot, the conversation including voice communication, visual communication using a display, or both. The dynamic conversation graph generation unit determines the flight operation procedure from the conversation with the pilot. In order to communicate with the pilot, the dynamic conversation graph generation unit provides the chatbot with information related to the flight operation procedure, and A method comprising the dynamic conversation graph generation unit responding to any requests received by the chatbot from the pilot during the conversation with the pilot. Article 16. Depending on whether the aforementioned flight operation procedure is associated with a specific aircraft system, the dynamic conversation graph generation unit hands off communication to a system-oriented conversation module, and The method according to clause 15, further comprising providing the chatbot with the information associated with the flight operation procedure via the system-oriented conversation module in order to communicate with the pilot. Article 17. To detect keywords in the conversation with the pilot using tokens from a tokenizer, wherein the keywords identify a specific flight operation procedure. To identify a node corresponding to the keyword in the conversation with the pilot, the search involves searching for multiple nodes in a dynamic conversation graph, where each node corresponds to a different flight operation procedure. Depending on whether the node corresponding to the keyword has an owner, the handoff of communication to a system-oriented conversation module, wherein the owner is a specific aircraft system associated with the flight operation procedure, and The method of Clause 15 or 16, further comprising determining the identity of any adjacent node to the node in the dynamic conversation graph, in accordance with whether the node corresponding to the keyword does not have an owner, wherein the identity of any adjacent node is communicated to the pilot by the chatbot in the conversation. Article 18. The tokenizer converts the text received from the chatbot into tokens, and The method according to any one of the clauses 15 to 17, further comprising converting the token received by the tokenizer into text to be sent to the chatbot to have the conversation with the pilot. Article 19. The method according to any one of claims 15 to 18, further comprising providing the tokenizer with a system-oriented conversation module a token containing the information associated with the flight operation procedure, depending on whether the flight operation procedure is associated with a particular aircraft system, and the dynamic conversation graph generator is configured to hand off communications to the system-oriented conversation module depending on whether the flight operation procedure is associated with a particular aircraft system. Article 20. The method according to any one of the clauses 15 to 19, further comprising generating a specific form, widget, or both associated with the flight procedure for presentation within a window on a display, wherein the specific form, widget, or both are configured to interact with the pilot, and the pilot's interaction with the specific form, widget, or both includes at least one of the following: interaction by conversing with the chatbot, or interaction by the pilot touching features of the specific form, widget, or both.

[0097] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible embodiments of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a particular logical function. In some alternative implementations, the functions described within a block may manifest in a different order than that shown in the diagram. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or sometimes in reverse order, depending on the related functions. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, may be implemented by a special-purpose hardware-based system that performs a particular function or operation, or by a combination of special-purpose hardware and computer instructions.

[0098] The terms used herein are for illustrative purposes only and are not intended to limit the examples provided herein. In this specification, the singular forms “a, an” and “the” are intended to include the plural forms as well (unless the context clearly indicates otherwise). Furthermore, the terms “include, includes” and “comprises and / or comprising,” as used in this specification, identify the presence of the features, entities, steps, actions, elements, and / or components described, but are not intended to exclude the presence or addition of one or more other features, entities, steps, actions, elements, components, and / or groups thereof.

[0099] All means or steps, corresponding structures, materials, actions, and equivalents, as well as functional elements in the following claims, are intended to include any structures, materials, or actions that, in combination with any other claimed elements, perform a function. The description of this embodiment is presented for illustrative and explanatory purposes only and is not intended to be exhaustive or to limit to the embodiments disclosed. Those skilled in the art will understand that numerous modifications and changes can be made without departing from the scope and spirit of the embodiment.

[0100] While specific embodiments are illustrated and described herein, those skilled in the art will recognize that the specific embodiments shown may be substituted with any configuration intended to achieve the same objective, and that these embodiments may have different uses in other environments. This application is intended to cover any modifications or variations of the present disclosure. The following claims are not intended in any way to limit the scope of the examples in the present disclosure to the specific embodiments described herein.

Claims

1. A cockpit communication interface system (100), A chatbot (102) configured to converse (602) with a pilot (103), wherein the conversation (602) includes voice communication, visual communication using a display (106), or both. The system includes a dynamic conversation graph generation unit (128) configured to perform a set of functions (130), the set of functions (130) being: Determining the flight procedure (306) from the conversation (602) with the pilot (103), To communicate with the pilot (103), provide the chatbot (102) with information associated with the flight operation procedure (308), and This includes responding to any requests (310) received by the chatbot (102) from the pilot (103) during the conversation (602) with the pilot (103), The interface system (100) further comprises a tokenizer (126), and the tokenizer (126) is Converting the text (156) received from the chatbot (102) into tokens (302, 403) transmitted by the tokenizer (126) (510), and The tokenizer (126) is configured to convert the token received by the tokenizer (126) into text (156) to be sent to the chatbot (102) to have a conversation (602) with the pilot (103), (546) The set of functions (130) performed by the dynamic conversation graph generation unit (128) is: Detecting a keyword (604) in the conversation (602) with the pilot (103) using the token (302) from the tokenizer (126) (514), wherein the keyword (604) identifies a specific flight operation procedure, In order to identify the node (702) corresponding to the keyword (604) in the conversation (602) with the pilot (103), the search (516) of multiple nodes (702-706) in the dynamic conversation graph (700), wherein each node (702-706) corresponds to a different flight operation procedure. Depending on whether the node (702) corresponding to the keyword (604) has an owner, hand off the communication (522) to a system-oriented conversation module (134), wherein the owner is a specific aircraft system (136) associated with the flight operation procedure, and An interface system (100) further comprising determining the identity (524) of any adjacent nodes (704a to 704d) to the node (702) in the dynamic conversation graph (700) in accordance with the fact that the node (702) corresponding to the keyword (604) does not have an owner, wherein the identity of any adjacent nodes (704a to 704d) is communicated to the pilot (103) by the chatbot (102) in the conversation (602).

2. The set of functions (130) performed by the dynamic conversation graph generation unit (128) is: Searching the adjacent nodes (704a to 704d) of the dynamic conversation graph (700) in order to identify a specific adjacent node (704) corresponding to another keyword (604) during the conversation (602) with the pilot (103) (552), and The interface system (100) according to claim 1, further comprising handing off communication (558) to the system-oriented conversation module (134) depending on whether the particular adjacent node (704) has an associated owner.

3. The interface system (100) according to claim 1, further comprising a system-oriented conversation module (134) configured to provide the tokenizer (126) with a specific token containing specific information associated with the flight operation procedure, depending on whether the flight operation procedure is associated with a specific aircraft system (136) among a plurality of aircraft systems (136a to 136n), wherein the dynamic conversation graph generation unit (128) is configured to hand off communication to the system-oriented conversation module (134) depending on whether the flight operation procedure is associated with the specific aircraft system (136).

4. The system-oriented conversation module (134) is configured to perform a set of functions, and the set of functions is Receiving a token (403) from the tokenizer (126) (402), wherein the token (403) contains information associated with the flight operation procedure, The aforementioned flight operation procedure is to query the database (140) (404), Determining a specific aircraft system (136) from among the plurality of aircraft systems (136a to 136n) for performing the aforementioned flight operation procedure (406), and The interface system (100) according to claim 3, comprising communicating information relating to the flight operation procedure with the pilot (103) via the tokenizer (126) and the chatbot (102) (416).

5. The system further comprises a command encoder / decoder (142), and the command encoder / decoder (142) is For communication with a specific aircraft system (136) among the plurality of aircraft systems (136a to 136n), the system-oriented conversation module (134) receives and executes formatted instruction commands related to the flight operation procedure (574), and The interface system (100) according to claim 3, configured to perform the task of decoding the specific information from the specific aircraft system (136) into a set of responses (582) for communication with the system-oriented conversation module (134).

6. The system further comprises a dynamic widget / form generation unit (144) configured to perform a set of functions, the set of functions being: Receiving the tokens (149, 152) from the tokenizer (126) or from the air traffic controller pilot (103) data link communications (CPDLC) / aircraft air-ground data communications system (ACARS) text (156) analyzer (802), and The interface system (100) according to claim 1, comprising generating a specific form, widget, or both (1116) in accordance with the aforementioned tokens (149, 152).

7. The interface system (100) according to any one of claims 1 to 6, further comprising a system-oriented conversation module (134) configured to guide the conversation (602) with the pilot (103) to a specific aircraft system (136) associated with the flight operation procedure, wherein the dynamic conversation graph generation unit (128) is configured to hand off communication to the system-oriented conversation module (134) depending on whether the flight operation procedure is associated with a specific aircraft system (136) among a plurality of aircraft systems (136a to 136n).

8. An automatic speech recognition device (122) configured to receive voice from the pilot (103) and convert the voice into text (156) for transmission to the chatbot (102), and The system further comprises a text-to-speech converter (124) configured to receive text (156) from the chatbot (102) and convert the text (156) into speech for transmission to the pilot (103) via a speaker (112), wherein the speaker (112) is located in the cockpit (105) or in the pilot's (103) headset. The interface system (100) according to claim 1, wherein the chatbot (102) is configured to have a conversation (602) with the pilot (103) using at least one of the following: an acoustic panel (104) in the cockpit (105), a display (106) in the cockpit (105), or a portable electronic device (108).

9. A method for communication in the cockpit (500, 1100), The chatbot (102) engages in a conversation (602) with the pilot (103), wherein the conversation (602) includes voice communication, visual communication using a display, or both. The dynamic conversation graph generation unit (128) determines the flight operation procedure (1104) from the conversation (602) with the pilot (103). In order to communicate with the pilot (103), the dynamic conversation graph generation unit (128) provides the chatbot (102) with information related to the flight operation procedure (1106), and The dynamic conversation graph generation unit (128) includes responding to any requests (1108) received by the chatbot (102) from the pilot (103) during the conversation (602) with the pilot (103), Detecting (514) a keyword (604) in the conversation (602) with the pilot (103) using a token (149) from the tokenizer (126), wherein the keyword (604) identifies a specific flight operation procedure, In order to identify the node (702) corresponding to the keyword (604) in the conversation (602) with the pilot (103), the search (516) of multiple nodes (702-706) in the dynamic conversation graph (700), wherein each node (702-706) corresponds to a different flight operation procedure. Depending on whether the node (702) corresponding to the keyword (604) has an owner, the handoff of communication (522, 1110) to a system-oriented conversation module (134), wherein the owner is a specific aircraft system (136) associated with the flight operation procedure, and A method comprising determining the identity (524) of any adjacent nodes (704a to 704d) to the node (702) in the dynamic conversation graph (700) in accordance with the fact that the node (702) corresponding to the keyword (604) does not have an owner, wherein the identity (524) of any adjacent nodes (704a to 704d) is communicated to the pilot (103) by the chatbot (102) in the conversation (602).

10. Depending on whether the aforementioned flight operation procedure is associated with a specific aircraft system (136), the dynamic conversation graph generation unit (128) hands off communication to a system-oriented conversation module (1110), and The method according to claim 9, further comprising providing the chatbot (103) with the information associated with the flight operation procedure via the system-oriented conversation module (134) (1114) in order to communicate with the pilot (103).

11. The tokenizer (126) converts the text (156) received from the chatbot (102) into tokens (149) (510), and The method according to claim 9, further comprising converting the token (149) received by the tokenizer (126) into text (156) to be sent to the chatbot (102) to have a conversation (602) with the pilot (103).

12. The method according to claim 11, further comprising providing the token containing the information associated with the flight operation procedure to the tokenizer (126) by a system-oriented conversation module (134) (586), in which case the flight operation procedure is associated with a particular aircraft system (136), and the dynamic conversation graph generation unit (128) is configured to hand off communication to the system-oriented conversation module (134) in which case the flight operation procedure is associated with the particular aircraft system (136).

Images