A flight state information judgment method and system based on cockpit audio data analysis

Abstract

The application discloses a flight state information judgment method and system based on cockpit audio data analysis, comprising the following steps: 1, using a cockpit recorder or / and a protection recorder to obtain audio data in the cockpit; 2, performing data preprocessing and noise suppression processing on the audio data in the cockpit to convert non-verbal information into sampling points with a specified sampling speed, extract audio data corresponding to each sampling point, and perform noise suppression processing on the extracted audio data; 3, using an alarm sound detection operation law to detect the category and accurate time of each alarm sound, and using an engine torque evaluation operation law to evaluate the engine torque. The technical scheme of the application solves the problem that the existing analysis method for audio data in a helicopter cockpit cannot accurately extract sound due to the large number of categories, different characteristics and various noises of audio in the cockpit.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of maintenance support, and particularly relates to a flight state information judgment method and system based on cockpit audio data analysis. BACKGROUND

[0002] Safety is an extremely important index requirement of aviation equipment, and the aviation industry at home and abroad has always been committed to improving the safety of helicopters, but the accident rate of helicopters is still higher than that of fixed-wing aircraft.

[0003] At present, a large number of researches have been implemented at home and abroad to understand the specific state of the cockpit when the helicopter event / accident occurs, especially the analysis of the audio data of the cockpit to supplement the work of the flight data recorder. The audio data of the cockpit includes various sounds, such as the conversation of the flight crew, the alarm sound, the movement sound of the control surface or the control surface, the trigger sound of the switch and the sound of the engine. The various categories of sounds in the helicopter cockpit all have relatively unique characteristics, such as frequency and duration, and in addition, these sounds are mixed with various noises, such as wind noise. The above factors all become the reasons restricting the analysis of the cockpit audio data.

[0004] At present, the industry has always used traditional methods such as subjective "discrimination and hearing" and single signal analysis to monitor the state, but the traditional method is not easy to complete the accurate extraction of the sound. Therefore, how to accurately identify the flight state information is extremely important for improving the safety of equipment, and it is also urgently needed to carry out technical research. SUMMARY

[0005] The purpose of the present application is to solve the above problems. The embodiment of the present application provides a flight state information judgment method and system based on cockpit audio data analysis, so as to solve the problem that the existing analysis method for the audio data in the cockpit of the helicopter is difficult to realize the accurate extraction of the sound due to the fact that there are many categories of audio in the cockpit, the characteristics are different, and various noises are mixed.

[0006] The technical scheme of the present application: the embodiment of the present application provides a flight state information judgment method based on cockpit audio data analysis, comprising:

[0007] Step 1, using a cockpit recorder or / and a protective recorder, acquiring audio data in the cockpit;

[0008] Step 2, performing data preprocessing and noise suppression processing on the audio data in the cockpit to convert the non-verbal information into sampling points with a specified sampling speed, extracting the audio data corresponding to each sampling point, and performing noise suppression processing on the extracted audio data;

[0009] Step 3, using the alarm sound detection operation law to detect the category and accurate time of each alarm sound, and using the engine torque evaluation operation law to perform engine torque evaluation.

[0010] Optionally, in the flight state information judgment method based on cockpit audio data analysis as described above, the manner of obtaining the audio data in the cockpit in step 1 comprises:

[0011] Manner 1, using a cockpit recorder to obtain the audio data in the cockpit, comprising: four channels in the cockpit recorder work simultaneously, channel 1 records non-verbal information, i.e. the alarm sound in the cockpit area, channel 2 and channel 3 record the voices of the pilot and co-pilot respectively, and channel 4 records the sound information of the radio panel;

[0012] Manner 2, using a protection recorder to obtain the audio data in the cockpit, comprising: the protection recorder has one channel working, which is used to record all the sounds in the cockpit, and the recorded sound includes the alarm and engine sound in the audio data.

[0013] Optionally, in the flight state information judgment method based on cockpit audio data analysis as described above, the manner of data preprocessing in step 2 comprises:

[0014] Step 21, dividing the audio data into verbal information and non-verbal information: the verbal information includes the voices of the pilot and co-pilot, and the non-verbal information includes other noise information and background sound;

[0015] Step 22, information conversion preprocessing: converting the non-verbal information into a set of sampling points with a specified sampling speed, and extracting the audio data corresponding to each sampling point.

[0016] Optionally, in the flight state information judgment method based on cockpit audio data analysis as described above, the manner of noise suppression processing in step 2 comprises:

[0017] Step 23, using a high-low pass filter to suppress noise of the extracted audio data of each sampling point to retain audio data within a specified frequency range;

[0018] Among them, the filtered data is used for alarm sound detection, and the unfiltered audio data is used for engine torque evaluation.

[0019] Optionally, in the flight state information judgment method based on cockpit audio data analysis as described above, the detection of each alarm sound in step 3 comprises:

[0020] For alarm sound detection, the category of the alarm sound is identified by the highest correlation between the alarm sound in the obtained cockpit audio data and the sound in the alarm sound database generated using the alarm characteristics.

[0021] Optionally, in the flight state information determination method based on cockpit audio data analysis, the engine torque evaluation in step 3 comprises:

[0022] For engine torque evaluation, the correlation between the engine sound main frequency range and the known engine parameters in the noise and uncertainty range is obtained to determine the main frequency of the engine.

[0023] Optionally, in the flight state information determination method based on cockpit audio data analysis, the way of detecting each warning sound in step 3 comprises:

[0024] S31, identify the characteristics of the warning sound, including: using Fourier transform and high power spectral density to identify the characteristics of each warning sound in an ideal environment, and detecting the occurrence time of each warning sound in an ideal environment; before analyzing the warning sound data, analyze the characteristics of each warning sound in an ideal environment to generate a warning sound database;

[0025] S32, frequency domain analysis of the warning sound, the category of the collected warning sound is obtained by STFT analysis method, and the occurrence time thereof is detected, the audio data is converted into an audio signal matrix by operation, and each element in the matrix represents the time and frequency of the audio signal;

[0026] S33, compare the similarity between the warning sound recorded by the audio data in the cockpit and the warning sound in the warning sound database containing all possible sounds of the relevant helicopter; the correlation between the Pth warning sound in the warning sound database and the audio data recorded in the sampling time t is represented as:

[0027] ;

[0028] In the formula, n is the number of frequencies in the high power spectral density, is the PSD value of the Pth warning sound in the warning sound data at time t in frequency j, and is the PSD value of the recorded warning frequency j in the audio data at sampling time t.

[0029] Optionally, in the flight state information determination method based on cockpit audio data analysis, the way of engine torque evaluation in step 3 comprises:

[0030] S41, in each segment, discard the frequency with low power spectral density, and distinguish the remaining frequencies by using clustering, each cluster represents the main continuous frequency in the audio data;

[0031] S42, correlate the flight data and the cluster of each main persistent frequency by Pearson correlation to determine the frequency corresponding to the known engine parameter; the correlation is represented as:

[0032] ;

[0033] wherein, is the Pearson correlation coefficient, represents the engine parameter value, represents the frequency from the signal processing technique, and represent and corresponding mean values, respectively; the value is between 0 and 1, the higher the value, the higher the correlation between the engine parameter value and the given frequency cluster;

[0034] S43, establish the correlation between the engine parameter and the audio data based on the audio sample, and use the segmented regression analysis to represent the statistical model as:

[0035] ;

[0036] ;

[0037] ;

[0038] wherein, α and β are estimated values, and the statistical model describes the engine parameter and the noise under the corresponding main frequency;

[0039] S44, establish the statistical model to represent the relationship between the main frequency (x) and the engine parameter (y), and estimate the flight parameter based on the statistical model under the known audio characteristics of the cockpit by the following formula:

[0040] ;

[0041] wherein, is the evaluation parameter, is the main frequency of the recorded audio data in the cockpit.

[0042] The embodiment of the present application also provides a flight state information judgment system based on cockpit audio data analysis, comprising: a memory and a processor, and a cockpit recorder or / and a protection recorder;

[0043] wherein, the memory is configured to save executable instructions;

[0044] The processor is specifically configured to implement the flight state information determination method based on cockpit audio data analysis as claimed in any one of the preceding clauses by executing the executable instructions stored in the memory using a cockpit recorder or / and a protection recorder.

[0045] The application has the following beneficial effects: The application provides a flight state information determination method and system based on cockpit audio data analysis, specifically a method for detecting alarm sounds and inferring engine state information. In the flight state information determination method, two sub-targets are achieved, i.e., alarm sound detection and engine related parameter inference. Based on the two sub-targets, cockpit audio analysis is divided into two sub-audio processing operation analysis. On the one hand, for alarm detection, a signal processing technology and a control chart mutation detection technology are combined to accurately identify or determine the alarm type and detect the start and end time of the same alarm. On the other hand, for engine torque information inference, a single processing technology is combined with a data mining method to determine the main frequency in the noise audio data, and then a statistical model is established to associate the engine related parameters (related flight parameters) with the audio information. Based on the detection of alarm sound type and occurrence time and the inference of engine related parameters, the audio data in the cockpit can supplement the data recorded by the flight data recorder, so that the state in the cockpit can be better understood during flight, especially when an accident / event occurs. Therefore, the application helps to analyze the causes of the event / accident, thereby improving the safety of the equipment and having certain engineering application value. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings are used to provide a further understanding of the technical solutions of the application, constitute a part of the specification, and are used to explain the technical solutions of the application together with the embodiments of the application, and do not constitute a limitation on the technical solutions of the application.

[0047] Figure 1 A flowchart of a flight state information determination method based on cockpit audio data analysis provided by the application;

[0048] Figure 2 A schematic diagram of frequency cluster identification in the flight state information determination method based on cockpit audio data analysis provided by the application. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] As explained in the background article above, given the high helicopter accident rate, there's a growing interest in analyzing cockpit audio data to supplement flight data recorders. However, due to the relatively unique characteristics of different sound types in cockpit audio data, as well as the presence of noise, existing methods for analyzing helicopter cockpit audio data make it difficult to accurately extract sound.

[0051] To address the above issues, an embodiment of the present invention provides a flight status information judgment method and system based on cockpit audio data analysis. Multiple cockpit audio analysis algorithms are used to detect the sound category and precise time of each alarm triggering, and the cockpit audio data is used to evaluate engine-related parameters.

[0052] The present invention provides the following specific embodiments that can be combined with each other. The same or similar concepts or processes may not be described in detail in some embodiments.

[0053] Figure 1 The present invention provides a flowchart of a method for determining flight status information based on cockpit audio data analysis. Figure 1 As shown, the flight status information determination method provided by the embodiment of the present invention includes:

[0054] Step 1: Use the cockpit voice recorder and / or the protection recorder to obtain the cockpit audio data;

[0055] Step 2: Preprocessing and noise suppression are performed on the cockpit audio data to convert non-verbal information into sampling points with a specified sampling rate, extracting the audio data corresponding to each sampling point, and performing noise suppression on the extracted audio data.

[0056] Step 3: Use the alarm sound detection operation law to detect the type and precise time of each alarm sound, and use the engine torque evaluation operation law to perform engine torque evaluation.

[0057] In one implementation of the embodiment of the present invention, in step 1, the following two methods can be used to obtain the cockpit audio data, including:

[0058] The method 1 uses the cockpit recorder to obtain the audio data in the cockpit, and includes: four channels of the cockpit recorder work simultaneously, channel 1 records non-verbal information, the non-verbal information is the warning sound in the cockpit area, channel 2 and channel 3 record the voice of the driver and the co-pilot respectively, and channel 4 records the sound information of the radio panel.

[0059] The method 2 uses the protection recorder to obtain the audio data in the cockpit, and includes: the protection recorder has one channel working, which is used to record all the sounds in the cockpit, and the recorded sound includes the warning and engine sound in the audio data.

[0060] In an implementation of the embodiment of the present application, the method of data preprocessing in step 2 can include:

[0061] Step 21, the audio data is divided into verbal information and non-verbal information: the verbal information includes the voice of the driver and the co-pilot, and the non-verbal information includes other noise information and background sound;

[0062] Step 22, information conversion preprocessing: the non-verbal information is converted into a set of sampling points with a specified sampling speed, and the audio data corresponding to each sampling point is extracted.

[0063] Further, in the implementation of step 2, the method of noise suppression processing can include:

[0064] Step 23, using a high-low pass filter to suppress noise of the extracted audio data of each sampling point, so as to retain the audio data in the specified frequency range;

[0065] Wherein, the filtered data is used for the warning sound detection, and the unfiltered audio data is used for the engine torque evaluation.

[0066] The step 3 of the embodiment of the present application includes two parts:

[0067] The first part is to detect each warning sound in step 3, which includes:

[0068] For the warning sound detection, the category of the warning sound is identified by the highest correlation between the warning sound in the obtained cockpit audio data and the sound in the warning sound database generated by using the warning characteristics;

[0069] The second part is the engine torque evaluation in step 3, which includes:

[0070] For the engine torque evaluation, the main frequency range of the engine sound is correlated with the known engine parameters in the noise and uncertainty range, so as to determine the main frequency of the engine.

[0071] In an implementation of the embodiment of the present application, the method of detecting each warning sound in step 3 includes:

[0072] S31, identifying the characteristics of the alarm sound, including: using Fourier transform and high power spectral density to identify the characteristics of each alarm sound in an ideal environment, and detecting the occurrence time of each alarm sound in an ideal environment; before analyzing the alarm sound data, analyzing the characteristics of each alarm sound in an ideal environment to generate an alarm sound database;

[0073] S32, performing frequency domain analysis on the alarm sound, obtaining the category of the collected alarm sound and detecting their occurrence time through STFT analysis method, converting the audio data into a matrix of audio signals through operation, and each element in the matrix represents the time and frequency of the audio signal;

[0074] S33, comparing the similarity between the alarm sound recorded by the audio data in the cockpit and the alarm sound in the alarm sound database containing all possible sounds of the relevant helicopter; the correlation between the Pth alarm of the alarm sound database and all recorded alarm sounds in the audio data corresponding to the sampling time t is represented as:

[0075] ; (4)

[0076] Wherein, n is the number of frequencies in the high power spectral density, is the PSD value of the Pth alarm in the alarm sound data at time t and frequency j, and is the PSD value of the recorded alarm frequency j in the audio data at the sampling time t.

[0077] In an implementation manner of the embodiment of the application, the engine torque evaluation manner in step 3 comprises:

[0078] S41, in each segment, discard the frequency with low power spectral density, and distinguish the remaining frequencies by using clustering, each cluster representing a main sustained frequency in the audio data;

[0079] S42, associating the flight data and the cluster of each main sustained frequency through Pearson correlation to determine the frequency corresponding to the known engine parameter; the correlation is represented as:

[0080] ; (5)

[0081] In the formula, is the Pearson correlation coefficient, represents the engine parameter value, represents the frequency from the signal processing technology, and represent and the respective mean values; The value is between 0 and 1, The higher the value, the higher the correlation between the engine parameter value and the given frequency cluster;

[0082] S43, based on the audio sample, the correlation between the engine parameter and the audio data is established, and the statistical model is expressed by using the segmented regression analysis as:

[0083] ; (6)

[0084] ; (7)

[0085] ; (8)

[0086] Wherein, alpha and beta are estimated values, and the statistical model describes the engine parameter and the noise under the corresponding main frequency;

[0087] S44, the statistical model is established to represent the relationship between the main frequency (x) and the engine parameter (y), and the flight parameter is estimated by the following formula based on the statistical model under the condition that the audio characteristics of the cockpit are known:

[0088] ; (9)

[0089] In the formula, is the evaluation parameter, is the main frequency of the recorded audio data in the cockpit.

[0090] The flight state information judgment method based on cockpit audio data analysis provided by the embodiment of the application is a method for conveniently detecting alarm sound and inferring engine state information. In order to realize two sub-targets, alarm detection and engine related parameter inference, the cockpit audio analysis operation is divided into two sub-audio processing operation laws based on the two sub-targets. On the one hand, for alarm detection, the alarm detection operation scheme is a combination of signal processing technology and control chart mutation detection technology to accurately identify or determine the alarm type and detect the start and end time of the same alarm; on the other hand, for engine torque information inference, a single processing technology is combined with data mining method to determine the main frequency in the noise audio data, and then a statistical model is established to associate the engine related parameters (related flight parameters) with the audio information. Based on the detection of alarm sound type and occurrence time and the inference of engine related parameters, the audio data in the cockpit can supplement the data recorded by the flight data recorder, so that the state in the cockpit can be better understood during flight, especially when an accident / event occurs; thereby helping to analyze the causes of the event / accident, and improving the safety of the equipment, which has certain engineering application value.

[0091] Based on the flight state information determination method based on cockpit audio data analysis provided in the above embodiments of the present application, an embodiment of the present application further provides a flight state information determination system based on cockpit audio data analysis, comprising a memory and a processor, and a cockpit recorder or / and a protective recorder.

[0092] The memory is configured to save executable instructions.

[0093] The processor is specifically configured to use the cockpit recorder or / and the protective recorder to implement the flight state information determination method based on cockpit audio data analysis provided in any of the above embodiments when executing the executable instructions saved in the memory.

[0094] The implementation of the flight state information determination method based on cockpit audio data analysis provided in the embodiments of the present application is schematically described below through a specific embodiment.

[0095] Embodiment

[0096] The embodiment provides a flight state information determination method based on cockpit audio data analysis. The method detects the sound category and accurate time of each alarm through a plurality of audio analysis operation laws, and evaluates the engine related parameters by using the cockpit audio data. The main method flow is as follows, as shown in the flow of Figure 1

[0097] Step one, formulate audio analysis operation law;

[0098] S1.1, determine the input of operation law: cockpit audio data of alarm and engine sound as the input of operation law;

[0099] S1.2, audio data preprocessing: convert the audio data information into points with a specified sampling speed, and obtain the audio data of the sampling points;

[0100] S1.3, suppress noise: suppress noise by using high and low band pass filter, suppress background noise such as wind noise and human voice. The alarm sound detection operation law uses filtered data, and the engine torque evaluation operation law uses non-filtered audio data.

[0101] S1.4, find the correlation of audio data: for alarm sound detection, identify the type of alarm sound by obtaining the highest correlation between the alarm sound of the cockpit audio data and the sound of the alarm sound database generated by using the alarm characteristics; for engine torque evaluation, determine the main frequency of the engine by obtaining the correlation of the main frequency range of the engine sound with the known engine parameters in the noise and uncertainty range;

[0102] ​S1.5, Statistical Model: Regression method is used to produce statistical model;

[0103] S1.6, Detecting the type and time of the warning, and establishing the relevant flight parameters.

[0104] Step 2, Obtain audio data in the cockpit;

[0105] S2.1, Obtain cockpit audio data; this step can be performed in two ways, i.e. using cockpit voice recorder and protective recorder; wherein:

[0106] First, cockpit voice recorder generally has four channels working simultaneously to record cockpit sound; among them, channel 1 records non-verbal information, i.e. cockpit area warning sound, channel 2 and channel 3 record the voices of the pilot and co-pilot respectively, and channel 4 records the sound information of the radio panel;

[0107] Second, protective recorder generally has only one channel working, which records all sounds. Unlike cockpit voice recorder audio data, the audio data obtained by protective recorder includes warning sound and engine sound.

[0108] Step 3, Preprocess and denoise cockpit audio data;

[0109] 3.1, Divide audio data into verbal information and non-verbal information: verbal information includes the voices of the pilot and co-pilot, while non-verbal information includes other noise information and background sound;

[0110] 3.2, Signal conversion preprocessing: convert non-verbal information into a set of sampling points with a specified sampling speed, and extract the audio data corresponding to each sampling point for further analysis;

[0111] 3.3, Suppress noise: use high and low pass filter to suppress noise of the extracted audio data of each sampling point, and band pass filter can effectively suppress unwanted frequencies, only retaining audio data within the specified frequency range; for engine torque evaluation, use unfiltered sound.

[0112] Step 4, Analyze cockpit audio data;

[0113] Cockpit audio analysis algorithm is composed of two sub-algorithms. One is warning sound detection, and the other is engine torque evaluation.

[0114] First, warning sound detection includes the following steps:

[0115] a) First, create a warning sound database.

[0116] The characteristics of each alarm sound in an ideal environment are identified using Fourier transform and high power spectral density, and then the occurrence time of each alarm sound in an ideal environment is detected. The alarm sound in an ideal environment refers to the alarm sound in an environment without background sound and noise. Before analyzing the alarm sound data, the characteristics of each alarm sound in an ideal environment are analyzed to generate an alarm sound database. The database is generated in the following manner: the characteristics of a specific sound can be identified using the main frequency of the alarm sound, and the main frequency signal can be expressed as a distribution in a frequency range. Based on the characteristics of the alarm sound being a periodic signal, discrete Fourier transform is used, which is discretized based on the specified sampling speed in step 2, and the alarm sound data at each discrete point forms the alarm sound database. This is given by the following formula.

[0117] , n = 0, 1, 2,..., N - 1; (1)

[0118] where N is the number of samples, n is the sampling point, X(k) is the amplitude of the alarm sound signal at frequency k, x[n] represents the frequency value corresponding to the sampling point n; P(k) is the high power spectral density (PSD) at frequency k; and the PSD is expressed as:

[0119] ; (2)

[0120] b) Then, frequency domain analysis is performed on the alarm sound.

[0121] The category of the alarm sound corresponding to the sampling point and the occurrence time thereof are obtained by STFT analysis, and the audio data is converted into a matrix of audio signals by calculation, and each element in the matrix represents the time and frequency of the audio signal.

[0122] ; (3)

[0123] In the formula, i is the time index, k is the frequency, and w(n-i) is a window function used to modify the value of x(n).

[0124] c) Finally, statistical value calculation. The correlation using the Euclidean distance can compare the similarity between the alarm sound recorded in the audio data in the cockpit and the alarm sound in the alarm sound database containing all possible sounds of the relevant helicopter. The correlation of the Pth alarm in the alarm sound database with all recorded alarm sounds in the audio data at sampling time t can be provided by the following formula.

[0125] ; (4)

[0126] where n is the number of frequencies in the high power spectral density, is the PSD value of the Pth alarm in the alarm sound data at time t and frequency j, and is the PSD value of the alarm frequency j recorded in the audio data at the sampling time t. It is to be noted that the correlation analysis operation law presented herein calculates the mean value of all values of the correlation between the alarms in the database and the alarms recorded in the audio data, and uses the mean value to calculate the statistical value.

[0127] Secondly, engine torque evaluation, including the following steps:

[0128] a) Firstly, the audio data is processed using a signal processing technique applied to 60s segments (i.e. equation 3 above). Within each segment, frequencies with low power spectral density are discarded and the remaining frequencies are distinguished by using clustering. This step eliminates indirect source noise and the density-based clustering method is advantageous in identifying irregularly shaped clusters and flexible in handling different sizes of data, which can represent the main persistent frequencies within the audio data; as shown in FIG. 1, which is a schematic diagram of frequency cluster identification in the flight state information determination method based on cockpit audio data analysis provided by the embodiment of the present application, Figure 2 wherein the blue, cyan and purple colors represent clusters respectively. Figure 2

[0129] b) Then, the flight data and the cluster of each main frequency are linked to determine the frequencies corresponding to known engine parameters (if any). This is achieved using the Pearson correlation given by the following equation:

[0130] ; (5)

[0131] wherein, is the Pearson correlation coefficient, represents the engine parameter value, represents the frequency from the signal processing technique, and represent the respective mean values; and the value is between 0 and 1, and the higher the value indicates that the engine parameter value has a high correlation with the given frequency cluster.

[0132] c) Further, the correlation (relationship) between the engine parameters and the audio data is established based on a large number of audio samples, and a statistical model is represented using segmented regression analysis. The engine parameter is treated as an independent variable (i.e. y), and the corresponding frequency is treated as a dependent variable (i.e. x), so that this linear regression is defined as:

[0133] ; (6)

[0134] ; (7) ​​

[0135] ; (8)

[0136] Where α and β are estimated quantities. This statistical model describes the engine parameters and noise at the corresponding main frequency;

[0137] d) Finally, a statistical model is developed to provide the relationship between the main frequency (i.e., x) and the engine parameters (i.e., y). Based on this, given the cockpit audio characteristics, the flight parameters can be estimated using the following equation.

[0138] ; (9)

[0139] Where, is the evaluation parameter, is the dominant frequency of interest in the recorded cockpit sound.

[0140] Although the embodiments disclosed herein are as described above, the contents are merely provided to facilitate understanding of the present invention and are not intended to limit the present invention. Any person skilled in the art may make any modifications and variations in the form and details of the embodiments without departing from the spirit and scope of the present invention. However, the scope of patent protection of the present invention shall remain subject to the scope defined by the appended claims.

Claims

1. A flight state information determination method based on cockpit audio data analysis, characterized by, The method comprises the following steps: Step 1, obtaining audio data in the cockpit by using a cockpit recorder or / and a protective recorder; Step 2, pre-processing and noise suppression processing are performed on the audio data in the cockpit to convert non-verbal information into sampling points with a specified sampling speed, extract audio data corresponding to each sampling point, and perform noise suppression processing on the extracted audio data; Step 3, detecting the category and accurate time of each alarm sound by using an alarm sound detection operation law, and performing engine torque evaluation by using an engine torque evaluation operation law; In step 3, the detection of each alarm sound comprises: S31, identifying the characteristics of the alarm sound, including: identifying the characteristics of each alarm sound in an ideal environment by using Fourier transform and high power spectral density, and detecting the occurrence time of each alarm sound in an ideal environment; analyzing the characteristics of each alarm sound in an ideal environment to generate an alarm sound database before analyzing the alarm sound data; S32, performing frequency domain analysis on the alarm sound, obtaining the category of the collected alarm sound and detecting the occurrence time thereof by using an STFT analysis method, and converting the audio data into a matrix of audio signals by operation, wherein each element in the matrix represents the time and frequency of the audio signal; S33, comparing the similarity between the alarm sound recorded by the audio data in the cockpit and the alarm sound database containing all possible sound alarm sounds of the related helicopter; the correlation between the Pth alarm sound in the alarm sound database and all recorded alarm sounds in the audio data corresponding to the sampling time t is represented as: ； where n is the number of frequencies in the high power spectral density, is the PSD value of the pth alert in the alert sound data at time t and frequency j, and is the recorded PSD value of the alert frequency j in the audio data at the sampling time t. In step 3, the engine torque evaluation comprises: S41, discarding the frequencies with low power spectral density in each segment, and distinguishing the remaining frequencies by using clustering, wherein each cluster represents a main sustained frequency in the audio data; S42, correlating the flight data and the cluster of each main sustained frequency by using Pearson correlation to determine the frequency corresponding to the known engine parameter; the correlation is represented as: ； wherein, is the Pearson correlation coefficient, represents an engine parameter value, represents a frequency from signal processing techniques, and represents and corresponding mean values, respectively; the value is between 0 and 1, the higher the value, the higher the correlation of the engine parameter value with the given frequency cluster; S43, establishing the correlation between the engine parameter and the audio data based on the audio sample, and representing the statistical model by using segmented regression analysis as: ； ； ； Wherein, α and β are estimated values, and the statistical model describes the engine parameter and noise under the corresponding main frequency; S44, establishing a statistical model to represent the relationship between the main frequency (x) and the engine parameter (y), and estimating the flight parameter based on the statistical model under the condition that the audio characteristics of the cockpit are known by the following formula: ； wherein is an evaluation parameter, is a dominant frequency of the recorded in-cabin audio data.

2. The flight status information determination method based on cockpit audio data analysis according to claim 1, characterized in that, In step 1, the method for obtaining audio data in the cockpit comprises: Method 1, obtaining audio data in the cockpit by using a cockpit recorder, including: four channels in the cockpit recorder work simultaneously, channel 1 records non-verbal information, i.e. alarm sound in the cockpit area, channel 2 and channel 3 record the voices of the pilot and the co-pilot respectively, and channel 4 records the sound information of the radio panel; Method 2, obtaining audio data in the cockpit by using a protective recorder, including: the protective recorder has one channel working, which is used to record all the sounds in the cockpit, and the recorded sound includes the alarm and engine sound in the audio data.

3. The flight status information determination method based on cockpit audio data analysis according to claim 2, characterized in that, The data pre-processing manner in step 2 includes: Step 21, dividing the audio data into language information and non-language information: the language information includes positive and negative driver voices, and the non-language information includes other noise information and background sound; Step 22, information conversion pre-processing: converting the non-language information into a set of sampling points with a specified sampling speed, and extracting the audio data corresponding to each sampling point.

4. The flight status information determination method based on cockpit audio data analysis according to claim 3, characterized in that, The noise suppression processing manner in step 2 includes: Step 23, using a high-low pass filter to perform noise suppression processing on the audio data of each extracted sampling point to retain audio data within a specified frequency range; Wherein, the filtered data is used for alarm sound detection, and the unfiltered audio data is used for engine torque evaluation.

5. The flight status information determination method based on cockpit audio data analysis according to claim 3, characterized in that, The detection of each alarm sound in step 3 includes: For alarm sound detection, the category of the alarm sound is identified by obtaining the highest correlation between the alarm sound in the cockpit audio data and the sound in the alarm sound database generated using the alarm characteristics.

6. The flight status information determination method based on cockpit audio data analysis according to claim 3, characterized in that, The engine torque evaluation in step 3 includes: For engine torque evaluation, the correlation between the engine sound frequency range and the known engine parameters in the noise and uncertainty range is obtained to determine the engine frequency.

7. A flight status information determination system based on cockpit audio data analysis, characterized by, It includes: A memory and a processor, and a cockpit recorder or / and a protection recorder; Wherein, the memory is configured to save executable instructions; The processor is specifically configured to use the cockpit recorder or / and the protection recorder to implement the flight state information judgment method based on cockpit audio data analysis as claimed in any one of claims 1-6 when executing the executable instructions saved in the memory.

Images