Method for assisting the piloting of an aircraft

Abstract

The invention relates to a method for piloting an aircraft, comprising the following steps implemented by an electronic piloting assistance device (1): detection of an event, said event being an identification event of a note class from among a set of predefined identification events of N note classes, N being an integer ≥ 2; following said detection, triggering the acquisition of a voice signal being transmitted or received by the pilot and associated with said note class, and processing of said acquired voice signal, said processing being a function of the note class and comprising at least the transformation of the voice signal into text and the generation of a note based on the text. Figure for the abstract: Fig. 1

Description

Title of the invention: Method for assisting in piloting an aircraft Technical field

[0001] The invention lies in the field of assistance in piloting an aircraft, civil or military. Prior art

[0002] Today, in an aircraft, the pilot (or more generally a member of the crew) uses a flight log, a notebook or a sheet on which he notes with a pen the information that he considers important to keep for later use, during the flight or after the flight. The pilot or the crew can also use a digital touch interface ("EFB" for Electronic Flight Bags) for the same purpose, using the keyboard as the input mode.

[0003] However, these two solutions do not allow you to write anything down when your hands are busy. In addition, this note-taking process consumes time and monopolizes the pilot's attention.

[0004] There is therefore a need to have a note-taking system that does not have these drawbacks. Summary of the invention

[0005] To this end, according to a first aspect, the present invention describes a method for assisting the piloting of an aircraft, comprising the following steps implemented by an electronic piloting assistance device:

[0006] detection of an event, said event being an event of identification of a note class among a set of predefined identification events of N note classes, N integer > 2;

[0007] following said detection, triggering of the acquisition of a voice signal being transmitted or received by the pilot and associated with said note class and processing of said acquired voice signal, said processing being a function of the note class and comprising at least the transformation of the voice signal into text and the generation of a note as a function of the text.

[0008] The method according to the invention thus fulfills certain current tasks of taking written notes which are today the responsibility of the pilot.

[0009] The proposed solution consists of a system for capturing and interpreting the voice (of the pilot, of a control tower agent, etc.) in different modes, storing the information provided and displaying it on a screen for future use. These notes are determined by transcribing the voice and presented to the pilot in a clear and unambiguous manner.

[0010] The solution meets the need to note information for memorization and later use (short-term consultation, in the minutes following the taking of notes or consultation during the flight or even consultation after the flight). It thus frees up time and attention resources previously devoted by the pilot to these tasks and also allows them to be carried out even when the pilot is busy with other tasks. This is particularly valuable for delicate piloting phases, and even more so in the case of a single-pilot crew or during the flight phases requiring the most note-taking: on the ground before takeoff and in flight during preparation for the approach.

[0011] In embodiments, such a method will further comprise at least one of the following features:

[0012] - the detection of an event of identification of a class of note is a function of at least at least one element among the detection of a keyword in a voice signal being transmitted or received by the pilot, the detection of manual or virtual actuation of a Human Machine Interface element in the aircraft;

[0013] - the signal to be acquired and processed is selected according to the note class identified by the detected identification event;

[0014] - the method for assisting in piloting an aircraft comprises at least one of the characteristics characteristics among:

[0015] i / an identification event having been predefined to identify the emission of a voice signal by the pilot via his microphone to the control tower, the acquisition and processing following the detection of such an event being in this case carried out selectively on the voice signal emitted by the pilot;

[0016] ii / an identification event having been predefined to identify the broadcasting of an ATIS recording in the aircraft intended for the pilot, the acquisition and processing following the detection of such an event being in this case carried out selectively on the voice signal received by the pilot;

[0017] iii / an identification event has been predefined to identify a voice note command triggered by the pilot, the acquisition and processing following the detection of such an event being in this case carried out selectively on the voice signal emitted by the pilot;

[0018] - the method for assisting in piloting an aircraft comprises at least one of the dis following positions:

[0019] in the case of characteristic i: the detection of the identification event comprises the detection of the actuation of the Push-To-Talk by the pilot;

[0020] in the case of characteristic i or iii: the detection of the identification event comprises the detection of a keyword in a voice signal emitted by the pilot or the detection of the selection by the pilot of a Human-Machine Interface element;

[0021] in the case of characteristic ii: the detection of the identification event comprises the detection of a keyword in a voice signal received by the pilot or the detection of the selection by the pilot of a Human-Machine Interface element;

[0022] - said treatment comprises one or more steps selected according to the type of note class among: raw transcription of the voice signal, selective extraction in the text of data relating to flight parameters, determination of a summary of the text.

[0023] According to another aspect, the invention describes a computer program intended to be stored in the memory of a device for assisting the piloting of an aircraft and further comprising a microcomputer, said computer program comprising instructions which, when executed on the microcomputer, implement the steps of a method according to the first aspect of the invention.

[0024] The invention also describes a non-transitory computer-readable medium storing such a computer program.

[0025] According to another aspect, the invention describes an aircraft piloting assistance device, said device being adapted to detect an event, said event being an event of identification of a note class among a set of predefined identification events of N note classes, N integer > 2;

[0026] said device being further adapted, following said detection, to trigger the acquisition of a voice signal being transmitted or received by the pilot and associated with said note class and to process said acquired voice signal, said processing being a function of the note class and comprising at least the transformation of the voice signal into text and the generation of a note as a function of the text.

[0027] In embodiments, such a device will further comprise at least one of the following features:

[0028] - the detection of an event of identification of a class of note is a function of at least at least one element among the detection of a keyword in a voice signal being transmitted or received by the pilot, the detection of manual or virtual actuation of a Human Machine Interface element in the aircraft;

[0029] - the aircraft piloting assistance device is adapted to select the signal to be acquired and processed according to the note class identified by the detected identification event. Brief description of the drawings

[0030] The invention will be better understood and other characteristics, details and advantages will appear more clearly on reading the following description, given without limitation, and thanks to the appended figures, given by way of example.

[0031] [Fig-1] [Fig.l] is an illustration of a pilot assistance device in a embodiment of the invention;

[0032] [Fig.2] [Fig.2] represents steps of a pilot assistance method in an embodiment of the invention;

[0033] [Fig.3] [Fig.3] is an example of an ATIS type note generated in a rea ​​mode lization of the invention;

[0034] [Fig.4] [Fig.4] is an example of an ATC type note generated in a rea ​​mode lization of the invention;

[0035] [Fig.5] [Fig.5] is an example of a “Miscellaneous” type note generated in a mode of carrying out the invention.

[0036] Identical references may be used in different figures when they designate identical or comparable elements. Description of the embodiments

[0037] [Fig.l] is an illustration of a pilot assistance device 1 in one embodiment of the invention, hereinafter referred to as assistance device 1, on board an aircraft (not shown).

[0038] The assistance device 1 comprises, in one embodiment, an acquisition and classification block 10, called ACQ / CLASS block, a processing block 20 and a display block 30, called AFF block.

[0039] The ACQ / CLASS block 10 is adapted to acquire voice signals of various natures and to assign them a note class from among a plurality of predefined text note classes and to deliver the acquired voice signals associated with their class to the processing block 20.

[0040] For example, voice signals occurring in the cabin include, but are not limited to, the following voice signals: ATC signals (ATC for "Air Traffic Controller"), ATIS voice signals, and various other voice signals.

[0041] ATC voice signals are the exchanges between the pilot and the control tower. In particular, when the pilot of the aircraft wishes to provide voice information to ATC, the pilot presses a button, called a Push-To-Talk button located for example on his headset or control stick or on the instrument panel of the aircraft, to switch from reception mode to transmission mode on a half-duplex link at a predefined frequency HF (“High Frequency”) or VHF (“Very High Frequency”) and to communicate the desired information vocally.

[0042] ATIS (Automatic Terminal Information Service) is an automatic broadcast service that allows pilots to receive continuous information on the busiest airports in aircraft.

[0043] ATIS voice messages are, for example, broadcast by a telephone answering machine, on a predefined radio frequency, called the ATIS frequency, and contain the following information: essential training, such as weather data, the runway(s) in use, the available approach, and any information needed by pilots. Pilots typically listen to ATIS through headsets or cockpit speakers before contacting the control tower, reducing controller workload and decreasing frequency occupancy time.

[0044] For example, in the case considered as an example, the ACQ / CLASS 10 block is adapted to receive as input the following voice signals emitted in the cockpit or at least some of them: for example those emitted by the pilot (for example, the microphone worn by the pilot has an output connected to an input of the ACQ / CLASS 10 block) and those emitted to the pilot (at least some of the loudspeakers in the cockpit have an output connected to a respective input of the ACQ / CLASS 10 block, the audio headphones worn by the pilot have an output connected to a respective input of the ACQ / CLASS 10 block).

[0045] In the embodiment described here, it has been considered, for illustration purposes only, that the information to be the subject of notes is of N=3 distinct types, listed below: - ATC type notes: radio communications carried out on VHF and HF frequencies for exchanges with ATC, in the direction Pilot to Control Tower; - ATIS type notes: ATIS information; - “Miscellaneous” type notes: various notes, “at the request” of the pilot.

[0046] For the first two types of notes (ATC and ATIS), in the embodiment considered, the information provided in the voice signal must be integrated into a note and the key information is to be highlighted.

[0047] The various notes, at the request of the pilot, are for example related in particular to: - the temporal monitoring of the flight: “block time”, “Take Off time”, “landing time”, “flight time”...; - information relating to departure, arrival and taxi (e.g.: parking at destination); - fuel management (e.g. fuel quantity readings): block at such time, fuel flow, fuel used; - potentially, information following the receipt of weather information (METAR / TAF), NOTAMs, airport information sheets, or information from audio listening to the ATIS (Example: QNH, wind (speed and direction), runway in use).

[0048] The processing block 20 is adapted to generate text notes: it is adapted in particular to transpose into a text note, a voice signal classified by the ACQ / CLASS block according to the class which has been assigned. In the embodiment considered, the processing block 20 comprises a sub-block for transposing voice into text 21, named STT (for the English Speech To Text), an interpretation block 22, named INTERP (for example a component for processing and understanding natural language, acronym NLP / NLU for "Natural Language Processing / Natural Language Understanding" in English), and a note management sub-block 23, named GEST_NOTES.

[0049] The AFF block 30 is adapted to display the text notes generated by the processing block 20 and comprises an HMI (Human Machine Interface) provided for example with a screen and a keyboard (or a touch screen).

[0050] The assistance device 1 is adapted, in the embodiment considered, to implement the steps, represented in [Fig.2], of a method 100 for assisting the piloting of the aircraft. The blocks that compose it are thus adapted to implement the steps incumbent on them, of the method 100 for assisting the piloting of the aircraft. In one embodiment, the steps or at least some of these steps are implemented by the execution, on a processor of the assistance device 1, of software instructions stored in a memory of said assistance device 1.

[0051] The method 100 comprises a step 101 of acquiring and classifying voice signals, carried out by the ACQ / CLASS block 10 which, in the different stages of the flight of the aircraft, receives as input the voice signals emitted by the pilot and the signals received by the pilot, as described above.

[0052] The ACQ / CLASS block 10 detects a class identification event among a predefined set of identification events of N classes of text notes and following this detection, it triggers the acquisition of a voice signal and associates this voice signal with the class which corresponds to this identification event.

[0053] The identification element of a predefined class can be of various natures: - actuation of a specific HMI button (this is understood to mean the actuation of a hardware button or the selection on a screen - of the cockpit for example - of a field associated with a predefined function); and / or - one or more keywords pronounced in a voice signal; and / or - the origin of a voice signal (for example, microphone of the pilot headset or earphone of the pilot headset); and / or - of the flight phase (takeoff, cruise, landing).

[0054] The ACQ / CLASS 10 block of course includes technical means enabling it to carry out this detection (voice recognition on the voice signals coming from the different collected sources, HMI button actuation detection, etc.). In one embodiment, it is connected to one or more remote control blocks which carry out this detection and notify the ACQ / CLASS 10 block in real time.

[0055] The triggering and the end of the acquisition are defined for each type of note.

[0056] Thus, in one embodiment, in a step 101a, as soon as it is detected (step 101 a_l) that the pilot initiates a communication on an HF or VHF channel to the control tower (for example via the detection of a Push To Talk button in the cockpit or the detection of a dedicated selection element on the HMI for selecting the radio transmission channel and used by the pilot), the ACQ / CLASS 10 block begins the acquisition of the voice signal emitted by the pilot (corresponding for example to the signal at the output of the pilot headset microphone): following this detection, the voice signal is recorded and further associated with the “ATC” class by the ACQ / CLASS 10 block (step 101a_2). The acquisition of the audio stream is stopped (step 101a_3) when it is detected that the pilot ends the communication to the control tower by selecting one of the means for stopping the transmission (if the PTT has been previously selected, release or use of a dedicated selection means on the radio transmission channel selection HMI).

[0057] For example, in one embodiment, in a step 101b, as soon as it is detected (step 101b_l) that an ATIS recording is broadcast (for example by detection of a predefined keyword spoken at the start of an ATIS recording or detection of the pilot actuating a Push button type selection button (single press or a Push To Note type button (button to be held down for the duration of ATIS listening) etc. depending on the configuration chosen), the ACQ / CLASS block 10 begins the acquisition of the broadcast voice signal (i.e. for example the signal output from the earphones of the pilot headset or a loudspeaker in the cabin): it records the voice signal and further associates it with the “ATIS” class (step 101b_2).The acquisition of the audio stream is stopped (step 101b_3) when the end of the ATIS broadcast is detected (similarly by detection again of a predefined keyword spoken at the end of ATIS recording or detection of the actuation or release by the pilot of a button or detection of a silence of a duration greater than a predefined threshold X in the signal supplied at the input of the earphone / loudspeaker (for example X equal to or greater than 300ms, etc. depending on the configuration chosen).

[0058] In one embodiment, in a step 101c, as soon as it is detected (step 101c_l) that the pilot commands a Miscellaneous type note-taking (for example by detecting a dedicated predefined keyword spoken by the pilot or detecting the actuation by the pilot of a Push button type selection button (single press) or a Push To Note type button (button to be held down while the pilot speaks his voice message) etc. depending on the configuration chosen), the ACQ / CLASS block 10 begins the acquisition of the voice signal (i.e. for example at the signal at the output of the pilot headset microphone) by recording the voice signal and further associates it with the “Miscellaneous” class (step 101c_2). The acquisition of the audio stream is stopped (step 101c_3) when the end of the signal to be transformed into a note of type “Miscellaneous” is detected (for example by detection of a predefined keyword spoken or detection of the actuation or release by the pilot of a button or detection of a silence of a duration greater than a predefined threshold in the signal supplied at the microphone input etc. depending on the configuration chosen).

[0059] In a step 102, any voice signal acquired in step 101 with its associated class is delivered by the ACQ / CLASS block to the STT block 21 which transposes it into text.

[0060] The STT block 21 performs a level 0 textual transcription (in the example considered corresponding to the raw text) consisting of a simple textual transcription of what was said. This raw text with the associated note class is then delivered to the INTERP block 22 which performs a first formatting and thus delivers a level 1 textual transcription of the voice signal. In the embodiment, the level 0 textual transcription will for example only be available / consultable a posteriori, in the file of notes recorded during a flight. The raw text will not be displayed in step 104, but is recorded in a memory of the assistance device 1 in association with the corresponding voice signal also stored.

[0061] This first formatting transcribes for example the level 0 text into aeronautical language (example: similar to the information on published maps, METAR message, ...) taking into account acronyms, capital letters, commonly used shortcuts. For example, it indicates: - C for Charlie; - 27L for twenty seven left; - weather information: SCT for Scattered, BKN for Broken;

[0062] In one embodiment, the INTERP block 22 then selects one or more treatments to be performed on the level 0 text from among treatments of several types (level treatment no. i, i = 1 to Niv) according to the associated note class and performs this / these selected treatment(s). Thus, certain level 0 texts will not be subject to subsequent level treatments and others will be.

[0063] In one embodiment:

[0064] - the first level or level 0 corresponds to the “raw” output of the speech to text without formatting or interpretation;

[0065] - level 1 corresponds to a formatting of the previous level: acronym aero nautical (SCT for scattered), aeronautical alphabet (C for Charlie), Arabic numerals for numbers (1 for one), and / or text punctuation;

[0066] level 2 corresponds to an interpretation of the data from the previous level: type / value (parameter QNH=1012 for “QNH 1012), command.

[0067] . Considering the example where the level 2 processing includes an extraction with interpretation of certain data of the note, this processing is carried out only for ATC and ATIS type notes. Additional data such as that a summary, an English translation are further determined and added in embodiments of the invention for this level 2.

[0068] In a step 103, a note is generated by the GEST_NOTES block 23 as a function of each level 1 text associated with its note class and the corresponding level 2 text when it exists and stored in a memory (not shown). Each note is for example a function of the associated note type. The GEST_NOTES block 23 creates the note so that it can be displayed as described below (numbering of the notes, date of the note, content of the folded and unfolded forms of the note, etc.).

[0069] In a step 104, the note obtained at the output of the GEST_NOTES block 23 is delivered for display by the AFF block 30. In one embodiment, the display is specific to the note class. The AFF block 30 displays the notes (or at least some of them) for example, in real time, in an HMI dedicated to the notes of the current flight. They can also be consulted in deferred time.

[0070] Each note is numbered. Several numbering logics are possible: a single numbering regardless of the type of note or a numbering by type of note

[0071] Each note is dated. The notes are classified by time. The notes are presented in an order chosen by the user (modifiable at any time): from oldest to most recent, from most recent to oldest).

[0072] In another embodiment, an indication of the flight phase has been added by the GEST_NOTES block 23 in the chronological sequence to help find a note more easily. This flight phase information has for example been obtained by the GEST_NOTES block 23 from a cockpit avionics module (example: Flight Management System).

[0073] The three types of notes are thus differentiated at the level of the HMI of the display block 30. A note appears, in the list of consultable notes, once the voice signal at the origin of this note has ended (for example within the following half minute).

[0074] Each note has, for example, two display forms: collapsed form (synthetic, summarized, essential information only) and unfolded form. For example, the collapsed form varies depending on the type of note.

[0075] In one embodiment, the display is: - ATC type notes: folded and unfolded form: presentation of text level 2 and text level 1; - ATIS type notes: folded form: level 2 text; unfolded form: level 2 text and level 1 text; - Miscellaneous type notes: Level 1 only

[0076] It is possible to modify, via the HMI of the display block 30, each note once that it is completed and available in the list of notes. The HMI then presents the note selected for modification in a specific format allowing modification.

[0077] It is possible to modify a note: - by selection, then manual modification (touch interaction), whatever the type of notes; or - by selection, then modification by voice, for example only for notes of type “Miscellaneous”

[0078] It is possible to delete a note with the possibility of recovering it from a trash can if necessary.

[0079] The AFF 30 display block therefore manages several display formats for the data of a given flight: - nominal mode: display of notes in folded form - Selection mode: display of the selected note in unfolded form - edit mode: display of the selected note in edit mode - “Deleted items” mode: display of deleted notes.

[0080] Optionally, a mode allows only the notes of a type of notes chosen from the three types to be displayed (filtering function).

[0081] Optionally, a mode allows only the notes associated with a chosen flight phase to be displayed (when the note is associated with the flight phase, see above).

[0082] The transition from one mode to another is done by user action (HMI interaction).

[0083] The AFF 30 display block manages access to several note files. There is a file per flight including the flight notes. Accessing the file for a given flight allows you to view the notes for the given flight after the fact.

[0084] A new notes file is created at the start of each flight. At the end of a flight, the notes for that flight are therefore archived in the same file and can be consulted later.

[0085] The creation of the notes file at the start of the flight and its archiving at the end of the flight are triggered by manual or automatic command (for example depending on the state of the aircraft, the flight phase).

[0086] As an illustration:

[0087] - an example of an ATIS type note generated in an embodiment of the invention is represented in [Fig.3];

[0088] - an example of an ATC type note generated in an embodiment of the invention is represented in [Fig.4];

[0089] an example of a note of type “Miscellaneous” generated in an embodiment of the invention is represented in [Fig.5].

[0090] The first note in [Fig.4] from the top is in unfolded form; the other notes are in their unfolded form. In Figures 3, 4, 5, the acronym ALT is used to indicate altitude; the acronym HDG is used to indicate heading; the acronym FL is used to indicate flight level; the acronym QNH is used to characterize atmospheric pressure; the acronym TURN is used to indicate the direction of rotation.

[0091] The pilot assistance device presented above thus makes it possible to transcribe (and annotate where appropriate) the voice on board an aircraft, intelligently, depending on the context. It reduces the workload of the pilot (or more generally of a member of the crew) in his routine tasks; in particular it:

[0092] - facilitates note-taking regardless of whether the pilot has his hands full or No ;

[0093] - restores the noted information in a structured and clear manner.

[0094] Only three classes of voice signals have been considered here. In other embodiments, N distinct classes are considered, with N any integer greater than or equal to two, with implementation of a note processing specific to each class. For example, a class corresponds to ATC exchanges from the control tower to the pilot etc. A class is defined according to inputs of the Human Machine Interface (HMI) and / or avionics type.

[0095] The method may be implemented by executing software instructions on a processor, as described. Alternatively, it may be implemented by dedicated hardware, typically a digital integrated circuit, either specific (ASIC) or based on programmable logic (e.g. FPGA / Field Programmable Gate Array).

Claims

Claims

1. Method for assisting the piloting of an aircraft, comprising the following steps implemented by an electronic piloting assistance device (1): detection of an event, said event being an event of identification of a note class among a set of predefined identification events of N note classes, N integer > 2; following said detection, triggering the acquisition of a voice signal currently being transmitted or received by the pilot and associated with said note class and processing said acquired voice signal, said processing being a function of the note class and comprising at least the transformation of the voice signal into text and the generation of a note as a function of the text.

2. Method for assisting in piloting an aircraft according to claim 1, according to which the detection of an event of identification of a class of note is a function of at least one element among the detection of a keyword in a voice signal being transmitted or received by the pilot, the detection of the manual or virtual actuation of a Human Machine Interface element in the aircraft.

3. Method for assisting the piloting of an aircraft according to claim 1 or 2, according to which the signal to be acquired and processed is selected according to the class of note identified by the detected identification event.

4. Method for assisting in piloting an aircraft according to any one of the preceding claims, comprising at least one of the characteristics among: i / an identification event having been predefined to identify the emission of a voice signal by the pilot via his microphone to the control tower, the acquisition and the processing following the detection of such an event being in this case carried out selectively on the voice signal emitted by the pilot; ii / an identification event having been predefined to identify the broadcasting of an ATIS recording in the aircraft to the pilot, the acquisition and the processing following the detection of such an event being in this case carried out selectively on the voice signal received by the pilot; iii / an identification event has been predefined to identify a voice note command triggered by the pilot, the acquisition and processing following the detection of such an event being in this case carried out selectively on the voice signal emitted by the pilot.

5. Method for assisting in piloting an aircraft according to claim 4, comprising at least one of the following provisions: in the case of characteristic i: the detection of the identification event comprises the detection of the actuation of the Push-To-Talk by the pilot; in the case of characteristic i or iii: the detection of the identification event comprises the detection of a keyword in a voice signal emitted by the pilot or the detection of the selection by the pilot of a Human-Machine Interface element; in the case of characteristic ii: the detection of the identification event comprises the detection of a keyword in a voice signal received by the pilot or the detection of the selection by the pilot of a Human-Machine Interface element.

6. Method for assisting in piloting an aircraft according to any one of the preceding claims, according to which said processing comprises one or more steps selected according to the type of note class from among: raw transcription of the voice signal, selective extraction in the text of data relating to the flight parameters, determination of a summary of the text.

7. Computer program, intended to be stored in the memory of a pilot assistance device (1) of an aircraft and further comprising a microcomputer, said computer program comprising instructions which, when executed on the microcomputer, implement the steps of a method according to one of the preceding claims.

8. Aircraft piloting assistance device (1), said device being adapted to detect an event, said event being an event of identification of a note class from a set of predefined identification events of N note classes, N integer > 2; said device being further adapted, following said detection, to trigger the acquisition of a voice signal being transmitted or received by the pilot and associated with said note class and to process said acquired voice signal, said processing being a function of the note class and comprising at least the transformation of the voice signal into text and the generation of a note as a function of the text.

9. Aircraft pilot assistance device (1) according to claim 8, in which the detection of an event of identification of a class of note is a function of at least one element among the detection of a keyword in a voice signal being transmitted or received by the pilot, the detection of the manual or virtual actuation of a Human Machine Interface element in the aircraft.

10. Pilot assistance device (1) for an aircraft according to claim 8 or 9, adapted to select the signal to be acquired and processed as a function of the note class identified by the detected identification event.

Images