Flight status recognition system and method

Abstract

The patent discloses a flight state recognition system and method, and relates to the cross field of computer software and avionics system.The main technical scheme of the present application is as follows: constructing a text database;collecting pilot voice data and performing voice recognition;inputting the voice data;matching the input voice data with the constructed text database, calculating the matching degree, and recognizing the flight state of the airplane by using the matching degree.The pilot voice recognition flight state process can quickly infer the real flight state of the airplane, recognize the dangerous conditions that the airplane may encounter during the flight process, improve the response speed of the abnormal flight state recognition, and implement fault disposal according to the abnormal recognition result.

Description

TECHNICAL FIELD

[0001] The present application relates to the intersection of computer software and avionics systems, and in particular, to a flight state recognition system and method. BACKGROUND

[0002] Traditional aircrafts in the process of flight when encountering some failures of the aircraft, most of them are triggered by the abnormal signal of the airborne data, and the pilot takes the corresponding operation according to the abnormal signal. The existing voice recognition system has not yet recognized and classified the pilot's voice, and has not yet researched the flight state based on the voice recognition result. Some prior art systems attempt to enable aircraft pilots to quickly and reliably interact with various avionics systems and cockpit components through voice recognition systems, but fail to involve various aspects of aircraft operation, such as operating environment, reliability and accuracy requirements, time-sensitive characteristics of some aircraft operations and other safety issues. Therefore, these systems do not involve giving the pilot feedback on the flight state of the aircraft based on the voice recognition result, and cannot ensure that the pilot can quickly respond correctly.

[0003] Generally, the pilot will make announcements, check single confirmations and other work according to the different states of the aircraft during the execution of the aircraft flight task, so there will be a large amount of voice in the flight process, and the text of these voices is relatively fixed, and the semantic category is basically consistent. Real-time recognition of these voice information can identify whether the pilot's voice is in accordance with the normal operation process, and can more quickly and accurately identify the flight state of the aircraft. If there is a flight anomaly, the flight anomaly recognition result given by the flight state recognition system can more quickly and efficiently handle dangerous situations. SUMMARY

[0004] In order to solve the above problems, the embodiment of the present application provides a flight state recognition method, which comprises:

[0005] A text database is constructed, and the flight state is identified based on the voice recognition result of the cockpit voice in the flight process, and a text database for comparison with the voice recognition result needs to be established;

[0006] The input voice data is processed.

[0007] The text database comprises a first text database and a second text database, the input voice data is a voice recognition result, and the text database is used for matching with the voice recognition result.

[0008] The first text database includes first type data and second type data; the second text database includes third type data and fourth type data; the first type data is given a high weight; the second type data is given a low weight; the first text database includes flight orders in various states of an airplane; the flight orders include fixed text and non-fixed text; the first type data is fixed text; the second type data is non-fixed text; the second text database includes airplane components and fault states; the third type data includes an engine and a flap; and the fourth type data includes "abnormal", "damaged" and "failed".

[0009] Before processing the input voice data, the method further includes:

[0010] collecting voice data;

[0011] performing voice recognition on the collected voice data;

[0012] obtaining the input voice data;

[0013] processing the input voice data includes at least one of the following: normal flight state recognition and abnormal flight state recognition.

[0014] The normal flight state recognition includes:

[0015] calculating a matching degree of the input voice data and the first text database;

[0016] judging whether the matching degree is greater than a first threshold.

[0017] The calculation of the matching degree of the input voice data and the first text database includes:

[0018] matching the input voice data with the first text database; the input voice data can be segmented, and then matched with data in the first text database; data in the first text database that matches the segmented input voice data is extracted;

[0019] calculating an edit distance of the matched data;

[0020] calculating a weighted result of the edit distance as the matching degree according to the weight of the first type data and / or the second type data; the first type data is fixed text; the second type data is non-fixed text; and the low weight of the second type data can reduce the influence on the weighted result;

[0021] In a case where the matching degree is greater than or equal to a first threshold value, an identification result of normal flight identification is output: the flight state is normal; in a case where the matching degree is less than the first threshold value, the following processing is performed on the input voice data: abnormal flight state identification.

[0022] The abnormal flight state identification includes:

[0023] The input voice data is segmented into first type keywords and second type keywords; the first type keywords include: engine, flap; the second type keywords include: abnormal, damage, failure; the first type keywords and third type data are used to determine the fault position, and the second type keywords and fourth type data are used to determine the fault state;

[0024] In a case where the first type keywords and the third type data are matched successfully, the edit distance between the second type keywords and the matched fourth type data is calculated as the matching degree;

[0025] In a case where the matching degree is greater than or equal to a second threshold value, an identification result of the abnormal flight identification is output: the flight state is abnormal; in a case where the matching degree is less than the second threshold value, an identification result of the abnormal flight identification is output: irrelevant to the flight state;

[0026] In a case where the first type keywords and the third type data are not matched successfully, an identification result of the abnormal flight identification is output: irrelevant to the flight state.

[0027] A normal flight state identification method includes:

[0028] A normal flight text database is constructed;

[0029] The normal flight text database includes: first type data and second type data; the first type data is given a high weight;

[0030] The matching degree between input voice data and the normal flight text database is calculated;

[0031] It is judged whether the matching degree is greater than a threshold value;

[0032] In a case where the matching degree is greater than or equal to the threshold value, an identification result is output: the flight state is normal;

[0033] In a case where the matching degree is less than the threshold value, an identification result is output: the flight state is not normal.

[0034] An abnormal flight state identification method includes:

[0035] An abnormal flight text database is constructed;

[0036] The abnormal flight text database comprises: first type data and second type data.

[0037] The input voice data is segmented into first type keywords and second type keywords.

[0038] The first type keywords are matched with the first type data, and the second type keywords are matched with the second type data.

[0039] In the case that the first type keywords are successfully matched with the first type data, the edit distance between the second type keywords and the matched second type data is calculated as a matching degree.

[0040] In the case that the matching degree is greater than or equal to a threshold value, an identification result of the abnormal flight recognition is output: flight state abnormal; in the case that the matching degree is less than the threshold value, an identification result of the abnormal flight recognition is output: irrelevant to flight state.

[0041] In the case that the first type keywords are unsuccessfully matched with the first type data, an identification result of the abnormal flight recognition is output: irrelevant to flight state.

[0042] A flight state identification system, the system comprises a voice collection module, a voice recognition module, a state identification module, a display flight state terminal; wherein the voice collection module is used for collecting voice data of a pilot in an aircraft cockpit; the voice recognition module is based on a constructed text database, and voice recognition is performed on the collected voice data; the state identification module is based on the above voice recognition result, and the input voice data is matched with the text database, and the flight state is identified according to the relationship between the matching degree and the threshold value; the display flight state terminal is used for displaying the flight state judgment result to the user.

[0043] The beneficial effects of the present application are as follows:

[0044] The voice data of the pilot during the flight is classified, the flight state is determined through the voice data of the pilot, the real flight state of the aircraft can be quickly inferred, abnormal flight behavior can be quickly responded according to the identification result, and the abnormal identification result can be transmitted to an electronic flight bag and other modules to query corresponding fault solving schemes and provide a quick fault disposal procedure under the normal flight program with strict regulations in the manual content.

[0045] The cockpit voice during the flight of the aircraft is recognized and classified by using artificial intelligence related technology, the obtained voice information of the crew is processed through noise reduction, voice recognition and classification, whether the aircraft completes the flight plan according to the normal situation is identified, and the dangerous situation possibly encountered by the aircraft during the flight is identified, which is beneficial to improve the identification response speed of the abnormal flight state. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0047] Figure 1 is a flow chart of a flight state recognition method in an embodiment of the present application;

[0048] Figure 2 is a flow chart of a normal flight state recognition method in an embodiment of the present application;

[0049] Figure 3 is a flow chart of an abnormal flight state recognition method in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without any creative effort belong to the protection scope of the present application.

[0051] In an embodiment provided by the present application, a flight state recognition method is provided.

[0052] A first text database, i.e., a text database of a normal flight state of an airplane, is constructed. In order to match the voice recognition result, the normal flight state text database is constructed, and the voice recognition result of the pilot performing the flight task is matched with the information in the text database, so as to identify whether the current airplane is in the normal flight state. The content of the normal flight state text database mainly includes the pilot's call in the normal flight process. The content broadcasted by the pilot in the normal flight process is sorted, and the first type of data is labeled. The first type of data is fixed text, and the second type of data is labeled. The second type of data is non-fixed text. The fixed text and the non-fixed text are mainly derived from the flight manual of the airplane. The fixed text includes the operation process in the flight process, and the non-fixed text includes different content that may appear in different flight tasks, such as flight altitude data. For example, in the FCOM2 operation manual, there is a specific operation process about the normal flight process of the airplane model. The fixed text includes: the left seat call before takeoff: “check the single-line lower part”; the takeoff program PF call: “takeoff”, “adjust the airplane thrust”, “check at 80 kt”, “takeoff at V1”, “collect the wheel”, the PM call before 60 kt: “takeoff thrust setting”, the PF call at 80 kt: “check at 80”, the PF call at V1: “V1”, and the call at the positive rate of climb: “positive rate of climb”. The non-fixed text includes: in the approach program, the PM call “DA” to establish a suitable visual reference, the call “visible runway”, “visible approach light”, and the report of other variable parameter values in the execution program according to the flight state of the airplane, such as the speed variable and the state of the airplane aileron. The labeling of the fixed text and the non-fixed text uses technical means, which are not limited. For paper manuals, the fixed text and the non-fixed text can be manually extracted. For electronic manuals, the fixed text and the non-fixed text can be extracted according to the structured document format, and then manually checked to ensure accuracy. The text database is used to match the voice recognition result, and the edit distance calculation and other operations are performed.

[0053] The first type of data is given a high weight, and the second type of data is given a low weight. The factors for determining the weight include: the proportion of the fixed text and the non-fixed text in the flight manual, the first threshold, and the edit distance result.

[0054] The text content of the flight manual is unchanged. When the text library is established, the proportion of the fixed text and the non-fixed text of the flight order in the flight manual is counted. The weight range is set according to the proportion of the two, and the fixed text is the main reference for setting the weight. If there is no non-fixed text in the execution operation program under normal circumstances, the matching degree of the fixed text should be 100%. The voice recognition result has a certain error, and the matching degree of the fixed text is close to 100%.

[0055] The determination of the weight of the fixed text and the non-fixed text and the first threshold value is related, and the threshold value is determined by pre-experiment, the speech recognition result data is matched with the first text library with the assigned weight, the weighted result of the edit distance is calculated, the final threshold value is determined according to the results of multiple experiments, the final threshold value obtains a weight of the fixed text, and the weight of the non-fixed text is determined again according to the obtained weight of the fixed text.

[0056] The weight is determined according to the edit distance result, including that the fixed text edit distance is small, and the non-fixed text is assigned a low weight; the non-fixed text in the speech recognition result is not successfully matched with the non-fixed text in the flight order, for example, the angle of the aircraft cannot be matched, and the non-fixed text is assigned a low weight.

[0057] Since the proportion of the fixed text in the flight order is different in different flight stages, for example, takeoff, approach, landing and the like, the weight of the fixed text and the non-fixed text in each flight stage can be obtained according to the pre-stored flight orders arranged in sequence, and therefore, the weight changes with the flight stage, and similarly, the pre-stored first threshold value in each flight stage can also be obtained according to the pre-stored flight orders arranged in sequence.

[0058] A second text database is constructed, the second text database is established by using expert experience and fault content information in a flight manual, main content is aircraft fault information text, and the third type of data and the fourth type of data are labeled; the third type of data is an aircraft function or an aircraft component keyword; the fourth type of data is an abnormality recognition keyword; the aircraft function or the aircraft component keyword includes an engine and a flap; and the abnormality recognition keyword includes “abnormal”, “damage” and “failure”.

[0059] The pilot's speech data in the flight process is collected. The collection includes collecting the pilot's speech data in the flight process by using a recording device.

[0060] The collected speech data is denoised, and whether the speech data is denoised depends on the airborne environment and the access source. Since the pilot's speech data collected in the flight process has a certain intensity of background noise, when the background noise intensity is large, it will have a significant impact on the subsequent speech recognition effect. In this case, the collected speech data needs to be filtered, and an amplifier is added to improve the signal-to-noise ratio, and the signal-to-noise ratio includes extracting the noise spectrum, then performing a reverse compensation operation on the noise-containing speech according to the noise spectrum, thereby obtaining the denoised speech data, so as to reduce noise interference.

[0061] The voice data after noise reduction is subjected to voice recognition to obtain a voice recognition result. The voice recognition includes: first, pre-processing the voice signal after noise reduction, adding pilot voice training samples in a pre-trained Chinese general voice model, training model optimization parameters, and extracting feature parameters based on a neural network, then training and recognizing the acoustic model, language model and dictionary using the feature parameters, and finally comparing the feature parameters with the trained acoustic model, language model and dictionary, calculating the corresponding probability through rules, selecting the result matched with the maximum probability of the feature parameters, and obtaining the voice recognition text. The training refers to obtaining model parameters and evaluating the pilot language recognition ability in the voice recognition model, matching with the first text database and the second text database, and optimizing the pilot language fitting ability and generalization ability; the recognition is a process of traversing the first text database and the second text database; the acoustic model represents the pronunciation of a language based on a neural network, which can recognize the pilot voice model and the characteristics of the cockpit environment through training; the language model is a probabilistic model of word regularization of the first text database and the second text database; the dictionary contains a large number of professional terms and pronunciation rules specific to the field of aircraft piloting.

[0062] The voice text recognition result is the obtained input voice data, and the input voice data is subjected to normal flight state recognition.

[0063] The matching degree of the voice recognition result with the first text database is calculated; the voice recognition result is matched with the first text database, including:

[0064] The voice recognition result is subjected to word segmentation search, and the data amount in the first text database is huge, so the voice recognition result can be subjected to word segmentation, and the word segmentation search method includes: jieba Chinese word segmentation and deep learning algorithm word segmentation. For example, the keywords such as approach and take-off in the voice recognition result can be searched in the first text database for flight orders matching the keywords after word segmentation, the matching flight orders are extracted for edit distance calculation, the non-matching flight orders are not subjected to edit distance calculation, and the voice recognition result is subjected to word segmentation to save the calculation amount and speed up the subsequent calculation of edit distance.

[0065] The voice recognition result is not subjected to word segmentation, but is directly matched with the first text database to extract the matching data of the voice recognition result and the first text database; the edit distance of the matching data is calculated, and the weighted result is calculated as the matching degree.

[0066] The edit distance of the matching data is calculated; the weighted result of the edit distance is calculated as the matching degree according to the weight of the first type of data and / or the second type of data, including:

[0067] The weighted result is calculated according to the high weight of the first type of data and the low weight of the second type of data;

[0068] The first type of data is matched, and the weighted result of the first type of data is calculated; the second type of data is matched, and the weighted result of the first type of data is used as the matching degree if the matching is successful; and the weighted result is calculated according to the high weight of the first type of data and the low weight of the second type of data if the matching is unsuccessful;

[0069] The first type of data is fixed text; and the second type of data is non-fixed text;

[0070] The edit distance of the fixed text in the speech recognition result and the fixed text in the matched text database is calculated, the edit distance weighted result of the fixed text is calculated according to the weight of the fixed text given when the first text database is constructed, and it is judged whether the non-fixed text can be matched, including numbers, flap states, etc. For example, the data in the first text database is the aircraft elevation angle 5°. If the speech recognition result matches the data in the database, the non-fixed text has no influence on the matching degree, and the edit distance weighted result of the fixed text is used as the matching degree. If the non-fixed text aircraft elevation angle in the speech recognition result is impossible, it is considered that the speech recognition result has an error, and the non-fixed text cannot be matched. At this time, the edit distance weighted result of the non-fixed text is calculated according to the low weight of the non-fixed text given when the first text database is constructed, and the edit distance weighted result of the fixed text and the non-fixed text is used as the matching degree. The low weight of the non-fixed text is used to ensure that the non-fixed text does not affect the calculation result of the edit distance of the fixed text, eliminate the influence of the non-fixed text on the matching degree, and prevent interference with the recognition process. The setting is not strict, and there are many determining factors.

[0071] The calculated weighted result is used as the matching degree, and it is judged whether the matching degree is greater than a first threshold value; if the matching degree is greater than or equal to the first threshold value, it is recognized as a normal flight state, and if the matching degree is less than the first threshold value, an abnormal flight state recognition is performed.

[0072] The abnormal flight state recognition includes:

[0073] The speech recognition result is subjected to semantic recognition, and the non-standard aircraft abnormal state expression can be recognized and identified as a fault;

[0074] The voice recognition result is segmented into words, and is divided into a first type of keyword and a second type of keyword, the first type of keyword is a part keyword, including: engine, flap; the second type of keyword is a fault keyword, including: "abnormal", "damage", "failure"; the first type of keyword and the third type of data are used to judge the fault position, the second type of keyword and the fourth type of data are used to judge the fault state; the segmented voice recognition result is traversed to match the second text database, the first type of keyword and the third type of data are matched, the second type of keyword and the fourth type of data are matched, the first type of keyword and the third type of data are matched successfully, the fault state is established, and the edit distance of the second type of keyword and the matched fourth type of data is taken as a matching degree; in the case that the matching degree is greater than or equal to a second threshold value, the recognition result of the abnormal flight recognition is output: flight state abnormal; in the case that the matching degree is less than the second threshold value, the recognition result of the abnormal flight recognition is output: irrelevant to flight state; in the case that the first type of keyword and the third type of data are not matched successfully, the recognition result of the abnormal flight recognition is output: irrelevant to flight state.

[0075] The first type of keyword and the third type of data are matched successfully, and the second type of keyword and the fourth type of data are matched, to identify the fault position and the fault state. After this step is completed, an electronic manual input end can be connected, the corresponding operation is searched in the electronic flight manual according to the fault position and the fault state, and a solution is provided for the user.

[0076] When the second text database is constructed, the fault keywords can be given weights, different fault levels are given different weights, and the fault state level of the fault position is judged.

[0077] The determination method of the first threshold value and the second threshold value includes:

[0078] The established text database is selected for testing, and the value of the threshold value is corrected;

[0079] A simple neural network is built to obtain reasonable threshold values; generally, in order to ensure the rationality and robustness of the threshold value, the determination of the threshold value is related to the constructed text database, preferably, the reasonable threshold value is obtained through pre-experiment, including: a group of data is selected from the constructed first text database and second text database for experiment, and the reasonable threshold value is deduced according to the result, so that the rationality and robustness of the threshold value setting can be ensured, and a high accuracy can be ensured in subsequent verification and use.

[0080] One embodiment provided by the application specifically includes:

[0081] The text database is constructed, the first text database is constructed, the first type of data in the first text database is given a high weight, the second type of data is given a low weight, and the first threshold value is determined through pre-experiment; the second text database is constructed, and the second threshold value is determined;

[0082] inputting a speech recognition result,

[0083] segmenting the speech recognition result, and distinguishing flight phases such as approach, take-off, landing, etc.

[0084] extracting a flight command related to the flight phase of the speech recognition result;

[0085] calculating an edit distance between the speech recognition result and the extracted flight command, and comparing a weighted result with a first threshold value, wherein the weighted result of the edit distance is a final matching degree;

[0086] if the matching degree is greater than or equal to the first threshold value, the speech recognition result is normal flight state;

[0087] if the matching degree is less than the first threshold value, an abnormal flight state is determined;

[0088] segmenting the speech recognition result, and dividing the speech recognition result into a first keyword and a second keyword, wherein the first keyword is matched with third data, and after the first keyword and the third data are successfully matched, an edit distance between the second keyword and matched fourth data is calculated as a matching degree; in a case where the matching degree is greater than or equal to a second threshold value, an abnormal flight recognition result is output: flight state is abnormal; in a case where the matching degree is less than the second threshold value, the abnormal flight recognition result is output: irrelevant to flight state.

[0089] in a case where the first keyword and the third data are not successfully matched, the abnormal flight recognition result is output: irrelevant to flight state.

[0090] The embodiment of the present application provides a normal flight state recognition method,

[0091] The normal flight text database is constructed; the content of the normal flight state text database is mainly the pilot's call in the normal flight process, and the content required to be broadcast by the pilot in the normal flight process is sorted, and the first type of data and the second type of data are marked. The first type of data is fixed text, and the second type of data is non-fixed text. The fixed part includes the operation process in the flight process, and the non-fixed text part includes different results that may occur in the pilot's execution of different flight tasks, such as flight altitude data and the like. For example, there is a specific operation process about the normal flight process of the aircraft model in the FCOM2 operation manual. The fixed text includes: the left seat call before take-off "check single-line lower part", the take-off program PF call: "take-off", "adjust the thrust of the aircraft", "check" at 80kt, "take-off" at V1, "collect wheels", the PM call: "take-off thrust setting" before 60kt, the PF call check at 80kt, the PF call take-off at V1, and "positive rate of climb" at the positive rate of climb. The non-fixed text includes: in the approach program, the PM calls "DA" to establish a suitable visual reference, calls "visible runway" and "visible approach light", and reports other variable parameter values such as speed variable, aircraft flap state and the like during the execution of the program. The fixed text and the non-fixed text are marked by technical means, which are not limited. For paper manuals, they can be manually extracted, and for electronic manuals, they can be extracted according to the structured document format, and then manually checked to ensure accuracy. The first type of data is given a high weight; the voice recognition result of the pilot executing the flight task is matched with the information in the text database, so as to identify whether the current aircraft is in the normal flight state.

[0092] Collecting voice data; including collecting the voice data generated by the pilot during the flight process by using a recording device;

[0093] Voice recognition is performed on the collected voice data;

[0094] The collected voice data can be de-noised before voice recognition. Whether to de-noise depends on the airborne environment and the access source of the voice data. Since the pilot's voice data collected during the flight of the aircraft will have a certain intensity of background noise, when the background noise intensity is large, it will have a significant impact on the subsequent voice recognition effect. Therefore, the collected voice data needs to be filtered, and an amplifier is added to improve the signal-to-noise ratio. The improvement of the signal-to-noise ratio includes: extracting the frequency spectrum of the noise, and then performing a reverse compensation operation on the noise-containing voice according to the frequency spectrum of the noise, so as to obtain the de-noised voice data, thereby reducing noise interference.

[0095] The voice recognition comprises: firstly, pre-processing the noise-reduced voice signal, adding pilot voice training samples in a pre-trained Chinese general voice model, training model optimization parameters, and extracting feature parameters based on a neural network; then, training and recognizing an acoustic model, a language model and a dictionary by using the feature parameters; and finally, comparing the feature parameters with the trained acoustic model, language model and dictionary, calculating corresponding probabilities by rules, selecting a result matched with the maximum probability of the feature parameters, and obtaining a voice recognition text.

[0096] The voice text recognition result is used as input voice data,

[0097] The matching degree of the input voice data and the normal flight text database is calculated, and the input voice data is searched and matched with the normal flight text database.

[0098] The voice recognition result is segmented and searched, and the segmentation search method comprises: jieba Chinese segmentation, deep learning algorithm segmentation, and matching with the normal flight text database.

[0099] The matching data in the input voice data and the normal flight text database is extracted, the edit distance of the voice recognition result and the matching voice data in the normal flight text database is calculated, the weighted result is calculated as the matching degree, and it is judged whether the matching degree is greater than a threshold value; in the case that the matching degree is greater than or equal to the threshold value, the result data is output; the result data is a normal flight state; in the case that the matching degree is less than the threshold value, the result data is output; the result data is an abnormal flight state.

[0100] The embodiment of the application provides an abnormal flight state recognition method,

[0101] An abnormal flight text database is constructed, and the abnormal flight text database includes first type data and second type data. The first type data and the second type data are labeled according to expert experience and failure content description in a flight manual. The first type data includes keywords of aircraft functions and / or components, such as an engine and a flap. The second type data includes “abnormal”, “damage” and “failure”. When the exchange between pilots appears data in the abnormal flight text database, the context of the voice recognition result is searched. If the words “abnormal”, “damage” and “failure” appear, it is determined that the probability of the aircraft being in an abnormal flight state is increased.

[0102] Voice data is collected, and the collected voice data is subjected to voice recognition.

[0103] The collected voice data can be subjected to noise reduction before voice recognition. Whether to reduce noise depends on the airborne environment and the access source of the voice data. Since the voice data of the pilot collected during the flight of the aircraft has a certain intensity of background noise, when the intensity of the background noise is large, the subsequent voice recognition effect is obviously affected. Therefore, the collected voice data needs to be filtered, and an amplifier is added to improve the signal-to-noise ratio. The improvement of the signal-to-noise ratio includes extracting the frequency spectrum of the noise, and then performing a reverse compensation operation on the noise-containing voice according to the frequency spectrum of the noise, so as to obtain the voice data after noise reduction, thereby reducing noise interference.

[0104] The voice recognition includes: first, the voice signal after noise reduction is preprocessed, pilot voice training samples are added in the pre-trained Chinese general voice model, the model parameters are trained and optimized, and the feature parameters are extracted based on the neural network. Then, the acoustic model, the language model and the dictionary are trained and recognized by using the above feature parameters. Finally, the feature parameters are compared with the trained acoustic model, language model and dictionary, the corresponding probability is calculated through rules, the result matched with the maximum probability of the feature parameters is selected, and the text of voice recognition is obtained. The training refers to obtaining model parameters, and evaluating the pilot language recognition ability in the voice recognition model. The abnormal flight text database is matched to optimize the fitting ability and generalization ability of the pilot language. The recognition is a process of traversing the abnormal flight text database. The acoustic model represents the pronunciation of a language based on the neural network. The pilot voice model and the characteristics of the cockpit environment can be recognized by training. The language model is a probability model of normal flight text database word regularization. The dictionary contains a large number of professional terms and pronunciation rules specific to the aircraft driving field.

[0105] The element information can be extracted from the pilot voice on the basis of voice recognition, a plurality of elements are analyzed in association, and the voice recognition result is subjected to semantic recognition by using technical means such as word analysis, information extraction, time causality and emotion recognition, and some colloquial words representing faults are recognized as the fourth type of data through semantic recognition.

[0106] Obtaining input voice data;

[0107] The voice recognition result is subjected to word segmentation search, and the word segmentation search method comprises: jieba Chinese word segmentation, deep learning algorithm word segmentation, and matching with an abnormal flight text database,

[0108] In the case that the first type of keyword matches the first type of data successfully, the edit distance between the second type of keyword and the matched second type of data is calculated as a matching degree;

[0109] In the case that the matching degree is greater than or equal to a threshold value, the recognition result of the abnormal flight recognition is output: flight state abnormality; in the case that the matching degree is less than the threshold value, the recognition result of the abnormal flight recognition is output: irrelevant to flight state;

[0110] In the case that the first type of keyword does not match the first type of data successfully, the recognition result of the abnormal flight recognition is output: irrelevant to flight state.

[0111] The embodiment of the present application provides a flight state recognition system, the system comprises a voice collection module, a voice recognition module, a state recognition module, and a display flight state terminal; wherein the voice collection module is used for collecting voice data of a pilot in an aircraft cockpit; the voice recognition module is used for performing voice recognition on the collected voice data on the basis of a constructed text database; the state recognition module is used for matching the input voice data with the text database on the basis of the voice recognition result, and recognizing the flight state according to the relationship between the matching degree and the threshold value; and the display flight state terminal is used for displaying the flight state judgment result to a user.

[0112] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included.

[0114] The division of the modules is for illustration only, and in actual applications, the above functions can be completed by different functional units or modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0115] The above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A flight state recognition method characterized by comprising: The method comprises: constructing a text database; the text database comprises: a first text database, a second text database; the first text database comprises: first type data, second type data; the second text database comprises: third type data, fourth type data; processing input voice data; Before processing the input voice data, the method further comprises: collecting voice data; performing voice recognition on the collected voice data; obtaining input voice data, which is a voice recognition result; processing the input voice data comprises at least one of the following: normal flight state recognition, abnormal flight state recognition; The abnormal flight state recognition comprises: segmenting the input voice data into first type keywords and second type keywords; matching the first type keywords with the third type data and the second type keywords with the fourth type data; the first type keywords and the third type data are used to determine the fault position, and the second type keywords and the fourth type data are used to determine the fault state; In the case that the first type keywords match the third type data successfully, the edit distance between the second type keywords and the matched fourth type data is calculated as a matching degree; In the case that the matching degree is greater than or equal to a second threshold, the recognition result of the abnormal flight recognition is output: flight state abnormal; in the case that the matching degree is less than the second threshold, the recognition result of the abnormal flight recognition is output: irrelevant to flight state; In the case that the first type keywords do not match the third type data successfully, the recognition result of the abnormal flight recognition is output: irrelevant to flight state.

2. The method of claim 1, wherein, The first type data is given a high weight.

3. The method of claim 1, wherein, The normal flight state recognition comprises: calculating the matching degree of the input voice data and the first text database; judging whether the matching degree is greater than a first threshold.

4. The method according to claim 2 or 3, characterized in that, Calculating the matching degree of the input voice data and the first text database comprises: matching the input voice data with the first text database; calculating the edit distance of the matching data; calculating the weighted result of the edit distance as the matching degree according to the weight of the first type data and / or the second type data.

5. The method of claim 3, wherein In the case that the matching degree is greater than or equal to the first threshold, the recognition result of the normal flight recognition is output: flight state normal; in the case that the matching degree is less than the first threshold, the input voice data is processed as follows: abnormal flight state recognition.

6. A normal flight state recognition method characterized by comprising: The method comprises: constructing a normal flight text database, which comprises: first type data, second type data; the first type data is given a high weight; constructing a second text database, which comprises: third type data, fourth type data; collecting voice data, and performing voice recognition on the collected voice data, and the voice recognition result is input voice data; processing the input voice data at least one of the following: normal flight state recognition, abnormal flight state recognition; calculating the matching degree of the input voice data and the normal flight text database; judging whether the matching degree is greater than a threshold; In a case where the matching degree is greater than or equal to a threshold value, an identification result: flight state normal is outputted; In a case where the matching degree is less than the threshold value, an identification result: flight state abnormal is outputted; The abnormal flight state identification comprises: Tokenizing input voice data into first type keywords and second type keywords; The first type keywords are matched with third type data, and the second type keywords are matched with fourth type data; the first type keywords and the third type data are used to determine a fault position, and the second type keywords and the fourth type data are used to determine a fault state; In a case where the first type keywords are successfully matched with the third type data, an edit distance between the second type keywords and matched fourth type data is calculated as a matching degree; In a case where the matching degree is greater than or equal to a second threshold value, an identification result of the abnormal flight identification: flight state abnormal is outputted; in a case where the matching degree is less than the second threshold value, an identification result of the abnormal flight identification: irrelevant to flight state is outputted; In a case where the first type keywords are unsuccessfully matched with the third type data, an identification result of the abnormal flight identification: irrelevant to flight state is outputted.

7. An abnormal flight state recognition method characterized by comprising: The method comprises: A first text database is constructed, and the first text database comprises first type data and second type data; An abnormal flight text database is constructed, and the abnormal flight text database comprises third type data and fourth type data; The first text database comprises flight orders in various states of an airplane; the flight orders comprise fixed texts and non-fixed texts; the first type data are the fixed texts; the second type data are the non-fixed texts; a second text database comprises airplane components and fault states; the third type data comprise engines and flaps; and the fourth type data comprise abnormalities, damages and failures; Tokenizing input voice data into first type keywords and second type keywords; the first type keywords are position keywords, and comprise engines and flaps; the second type keywords are fault keywords, and comprise abnormalities, damages and failures; the first type keywords and the third type data are used to determine a fault position, and the second type keywords and the fourth type data are used to determine a fault state; The first type keywords are matched with the third type data, and the second type keywords are matched with the fourth type data; In a case where the first type keywords are successfully matched with the third type data, an edit distance between the second type keywords and matched fourth type data is calculated as a matching degree; In a case where the matching degree is greater than or equal to a second threshold value, an identification result of the abnormal flight identification: flight state abnormal is outputted; in a case where the matching degree is less than the second threshold value, an identification result of the abnormal flight identification: irrelevant to flight state is outputted; In a case where the first type keywords are unsuccessfully matched with the third type data, an identification result of the abnormal flight identification: irrelevant to flight state is outputted.

8. An airborne state recognition system, characterized by The system comprises a voice collection module, a voice recognition module, a state recognition module, and a display flight state terminal; the voice collection module is configured to collect voice data of a pilot in a cockpit of an airplane; the voice recognition module is configured to perform voice recognition on the collected voice data based on a constructed text database; the state recognition module is configured to match the input voice data with the text database based on the voice recognition result, and to recognize a flight state according to a relationship between a matching degree and a threshold value; The display flight state terminal is configured to display a flight state judgment result to a user; The text database comprises a first text database and a second text database; the first text database comprises first type data and second type data; the second text database comprises third type data and fourth type data; the input voice data is a voice recognition result; The recognized flight state comprises at least one of normal flight state recognition and abnormal flight state recognition; The abnormal flight state recognition comprises: segmenting the input voice data into first type keywords and second type keywords; matching the first type keywords with the third type data and matching the second type keywords with the fourth type data; the first type keywords and the third type data are configured to judge a fault position, and the second type keywords and the fourth type data are configured to judge a fault state; in a case where the matching of the first type keywords with the third type data is successful, calculating an edit distance between the second type keywords and the matched fourth type data as a matching degree; in a case where the matching degree is greater than or equal to a second threshold value, outputting a recognition result of the abnormal flight recognition: flight state abnormal; in a case where the matching degree is less than the second threshold value, outputting a recognition result of the abnormal flight recognition: irrelevant to a flight state; in a case where the matching of the first type keywords with the third type data is unsuccessful, outputting a recognition result of the abnormal flight recognition: irrelevant to a flight state.

Images