Airport on-road speech analysis and safety management method, system, terminal and computer storage medium
Patent Information
- Application Number
- CN202410287791.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-03-13
Smart Images

Figure CN118116373B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, system, terminal and computer storage medium for airport on-road speech analysis and safety management. Background Art
[0002] Runway intrusion prevention is one of the core risks of airport control. When passing through the runway or going to the runway for work, frontline staff need to communicate with the airport tower and enter the runway only after obtaining permission from the tower. Frontline staff may not express themselves clearly or use non-standard communication, which may lead to misjudgment by airport tower control personnel, thus causing unsafe runway intrusion incidents. Summary of the invention
[0003] To this end, the purpose of the present invention is to provide an airport runway voice analysis and safety management method, system, terminal and computer storage medium, which can perform voice analysis on the call records of front-line personnel and tower control personnel, identify non-standard call records, warn relevant personnel, alleviate the risk of runway intrusion prevention, ensure the safety of runway operations, and reduce the probability of unsafe incidents such as runway intrusion.
[0004] In order to achieve the above object, the present invention adopts the following technical solutions:
[0005] In a first aspect, the present invention first proposes a method for airport on-road speech analysis and safety management, the method comprising:
[0006] The recording equipment will connect the generated airport voice recording files to the OA network through the flight area network segment and transmit them to the safety status monitoring system server in real time;
[0007] The safety status monitoring system server uses a python interpreter and a watch program. When the safety status monitoring system server receives data consisting of a new recording file, the watch program notifies the data analysis program in the upper voice analysis server. The upper voice analysis server receives the upper voice recording file from the safety status monitoring system server and applies the data analysis program to perform upper voice recognition analysis.
[0008] After the Shangdao voice analysis server completes the Shangdao voice recognition analysis, it broadcasts the signal analyzed by the Shangdao voice recognition through port 8011 through UDP, and stores the analyzed data in the MySQL database;
[0009] After receiving the signal that the upper voice recognition and analysis is completed, the security status monitoring system server reads the analyzed data from the MySQL database, and performs an upper voice standard judgment operation on the analyzed data according to the scoring rules, and judges the total risk value of the calls in the upper voice recording file based on the scoring rules, and then counts and annotates the results of the voice index items.
[0010] In a second aspect, the present invention further proposes an airport on-road voice analysis and safety management system, comprising a dedicated hardware network architecture, the dedicated hardware network architecture comprising a terminal and a server, the terminal comprising a recording device, the server comprising an OA network, a MySQL database, and an on-road voice analysis server and a safety status monitoring system server both connected to the OA network and the MySQL database, the terminal being connected to the OA network via an airfield network segment;
[0011] A dedicated hardware network architecture is used to execute the above-mentioned airport on-road speech analysis and safety management method.
[0012] In a third aspect, the present invention further proposes a terminal, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned airport on-road voice analysis and safety management method.
[0013] Fourth aspect: The present invention also proposes a computer storage medium storing a plurality of instructions, which are suitable for being loaded by a processor and executing the above-mentioned airport on-road speech analysis and safety management method.
[0014] The technical effects to be achieved by the present invention include:
[0015] The present invention constructs a voice analysis system based on the call records of front-line personnel and tower control personnel in an airport environment. The system includes a recording device, a terminal connected to the OA network of a server through an airfield network segment, and connected to a server with an on-site voice analysis server and a safety status monitoring system, and a real-time voice event monitoring and content recognition mechanism suitable for a specific environment is set to identify non-standard call records, and then cooperate with a scoring system based on a time domain value and a risk value set based on an airport environment to determine and identify unsafe events in a specific environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0017] Figure 1It is a hardware network architecture diagram provided by the specific implementation method of this application;
[0018] Figure 2 It is a data communication logic flow chart provided by an embodiment of the present application;
[0019] Figure 3 It is a flowchart of the upper road speech recognition analysis process provided by the embodiment of the present application;
[0020] Figure 4 It is a flowchart of the compliance judgment process of the upper road speech recognition result provided in the embodiment of the present application;
[0021] Figure 5 It is a schematic diagram of the hardware structure of the terminal provided in the embodiment of the present application. DETAILED DESCRIPTION
[0022] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0023] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0024] The present invention proposes a method, system, terminal and computer storage medium for airport on-road speech analysis and safety management. The system includes a hardware network architecture, such as Figure 1 shown.
[0025] The hardware network architecture includes two parts: the terminal and the server. The terminal includes a recording device, which can be a dedicated mobile terminal or a general call device. The terminal is connected to the server's OA network by wireless communication through the flight zone network segment. The server's OA network is connected to the on-road voice analysis server and the safety status monitoring system server through the wired network architecture to form a complete server system. The server system is also equipped with a MySQL database connected to the on-road voice analysis server and the safety status monitoring system server respectively. The database is an independently set management system that inherits the database management device and the hardware storage structure. The dedicated hardware network architecture is used to execute the airport on-road voice analysis and safety management method to realize the analysis and safety management of the airport on-road voice data.
[0026] like Figure 2 As shown, this embodiment shows the specific implementation method of the airport on-road voice analysis and safety management method:
[0027] The recording device will connect the generated voice recording files to the OA network through the flight zone network segment in real time, and transmit the recording files to the safety status monitoring system server;
[0028] The safety status monitoring system server uses the python interpreter and the watch program. When the safety status monitoring system server receives data consisting of a new recording file, the watch program notifies the data analysis program in the upper voice analysis server. The upper voice analysis server receives the upper voice recording file from the safety status monitoring system server and applies the data analysis program to perform upper voice recognition analysis.
[0029] After the Shangdao speech analysis server completes the Shangdao speech recognition analysis, it broadcasts the signal analyzed by the Shangdao speech recognition through UDP through port 8011, and stores the analyzed data in the MySQL database;
[0030] After receiving the signal that the Shangdao voice recognition and analysis is completed, the security status monitoring system server reads the analyzed data from the MySQL database, and performs the Shangdao voice standard judgment operation on the analyzed data according to the scoring rules, and judges the total risk value of the calls in the Shangdao voice recording file based on the scoring rules, and then counts and annotates the results of the voice index items.
[0031] In some specific implementations, the security status monitoring system server installs a python interpreter and runs the watch program to monitor in real time whether new recording files are generated. When data constituting a new recording file is generated, the watch program will notify the data analysis program in the Shangdao voice analysis server to receive the recording file from the security status monitoring system server and perform the Shangdao voice recognition and analysis operation.
[0032] After the Shangdao voice analysis server analyzes the recording file, it will broadcast UDP through port 8011 and store the analyzed data in the MySQL database.
[0033] After receiving the UDP broadcast of port 8011 through Risk, the security status monitoring system server connects to the MySQL database and retrieves the parsed data. After that, the security status monitoring system server performs the upper voice standard judgment operation on the parsed data according to the scoring rules, judges the total risk value of the call based on the scoring rules, and then counts and annotates the results of the voice indicator items.
[0034] In some embodiments, the above speech recognition analysis process is as follows: Figure 3 As shown, it includes the following two steps:
[0035] Step 1: On-channel voice cutting step, in which a frame cutting model is constructed, the airport on-channel voice signal recording file is input into the frame cutting model, frame cutting is performed, and the fundamental frequency characteristics and time domain characteristics of the call data in the recording file are extracted;
[0036] Step 2 is the runway operation call speech recognition step. This step identifies the content corresponding to the fundamental frequency features and time domain features extracted in the previous voice cutting step by building a language and semantic correlation model applied to special scenarios, and improves the recognition effect through model design.
[0037] Specifically, in step 1, the implementation method of the upper voice cutting step is as follows:
[0038] Split a large speech file into independent sentences for subsequent processing. The speech segmentation algorithm uses GMM (Gaussian Mixture Mode) to model speech and noise, and judges speech and noise by corresponding probabilities. The advantage of this algorithm is that it is unsupervised and does not require strict training.
[0039] The implementation process involves the following sub-steps:
[0040] (1) Collecting speech signal sample sets. Collecting speech signal samples at the airport. The samples include speech signal sample sets consisting of speech signals of multiple voices, multiple speakers, multiple speaking speeds, accents, holidays and weekdays.
[0041] (2) Collect noise signal sample sets. Collect the specific noise on the airport runway that needs to be detected. The samples include the takeoff noise of various aircraft types, the vehicle driving noise around the runway, the noise under different weather conditions, etc., as the noise signal sample set.
[0042] (3.1) Framing: The audio files in the speech signal sample set and the noise signal sample set are pre-emphasized and then framed to obtain speech frame signals and noise frame signals respectively. The frame length of each frame is fixed and can be 10ms, 20ms, 30ms, etc.
[0043] (3.2) Windowing, and obtain MFCC features through discrete Fourier transform.
[0044] After segmentation, there will be discontinuities at the beginning and end of each frame, so the more frames are segmented, the greater the error with the original signal. The purpose of windowing is to make the framed signal continuous, and each frame will show the characteristics of a periodic function. Common window functions include rectangular window, Hamming window, Hanning window, etc. In speech signal processing, the Hamming window is usually used, and its formula is as follows:
[0045] ,
[0046] A set of bandpass filters is set from dense to sparse according to the critical bandwidth in the frequency band from low frequency to high frequency to filter the input signal. The energy signal output by each bandpass filter is used as the basic feature of the signal. After adaptive processing, this feature can be used as the input feature of the speech. Among them, n is the time index, L is the window length, and w[n] represents the windowing process.
[0047] (3.3) The data in the framed and windowed speech signal sample set and the noise signal sample set are mixed to generate mixed noise data, and a total of N noisy speech strips are obtained to simulate the interfered speech samples in the actual environment, which are used as the test set of the model for testing after the model is trained.
[0048] (4) Model training: First, extract the feature set for GMM training from the MFCC features. Specifically, extract the speech feature set for the speech signal sample set, and extract the noise feature set for the noise signal sample set. The speech feature set and the noise feature set include the feature sets of the speech signal sample set and the noise signal sample set, and the feature sets include the fundamental frequency features and time domain features extracted from each frame of the speech signal and the noise signal.
[0049] When extracting speech signal features, the spatial correlation of the spectrograms of the speech signal sample set and the noise signal sample set is extracted through a convolutional neural network, and the temporal correlation of the speech signal sample set and the noise signal sample set as well as the correlation between speech signal frames are extracted through a recurrent neural network. Finally, the output of the speech signal frame is combined with the text label through the CTC function to train the GMM model.
[0050] Then, the noise and speech models of the specific GMM to be trained are trained separately, where the noise and speech models are expressed as follows:
[0051] ,
[0052] Among them, p(xk|z,rk) represents the noise or speech value calculated under the condition of the k-th sampling value using the expectation and the k-th sub-model probability; sita represents the covariance, uz represents the mean, xk represents the k-th sampling value, z represents the expectation, rk represents the k-th sub-model probability, and pi is 3.14.
[0053] The processing logic is as follows: Supported frame length: supports judgment of 10ms, 20ms and 30ms voice frames.
[0054] (5) The calculation steps for voice cutting are as follows:
[0055] First, calculate the subband energy:
[0056] B(m,n) represents the energy of subband m in the nth frame, and the calculation formula is as follows:
[0057] ,
[0058] in, is the number of sample points in the subband, Indicates the energy value of the subband sample point in the nth frame, is the total number of sub-bands; in the speech of airport runway operation calls, there are many frames with large short-term energy, which makes the background noise estimation value too high, which easily causes the threshold to be set too high, resulting in the low-energy speech part being misjudged as noise. By calculating the sub-band energy, a frame signal is divided into multiple sub-bands when calculating the short-term energy, and the energy of each sub-band is calculated. The interference of noise on the threshold setting is eliminated by limiting the sub-band energy. The sub-bands are divided into 80~250Hz, 250~500Hz, 500~1000Hz, 1000~2000Hz, 2000~3000Hz, and 3000~4000Hz. The above sub-band energy feature vectors need to be calculated separately.
[0059] The specific method of limiting the subband energy is: when B(m,n)≥(E n / a), assign B(m,n) to 0; where a is a constant that needs to be obtained through multiple experiments. In the runway operation call speech recognition scenario, a takes a value of 0.5; E n Indicates its short-time energy.
[0060] Let BE(n) be the sub-band short-time energy of each frame calculated from the sub-band energy, and its noise average energy is:
[0061] ,
[0062] in, is the number of sample points in the subband.
[0063] Determine the signal type: The Gaussian mixture model is used to calculate the probabilities of speech and non-speech respectively, and the type of signal (speech or non-speech) is determined using the hypothesis testing method.
[0064] Step 2: The implementation method of the runway operation call voice recognition step is as follows:
[0065] The entire runway operation call speech recognition process includes three sub-steps: speech signal feature extraction step, acoustic feature modeling sub-step and language model decoding sub-step.
[0066] The implementation process involves the following sub-steps:
[0067] (1) Speech signal feature extraction sub-step: Extract the on-track speech signal containing information such as the noise background model, speaker's speaking speed, and speaker's pronunciation characteristics. Under the premise of fully considering the unique characteristics of runway operation call speech, a special speech signal framing method and a speech signal feature extraction method based on MFCC (MFCC: Mel Frequency Cepstrum Coefficient, MFCC) are proposed. This method forms a speech signal spectrum diagram by extracting one-dimensional and two-dimensional dynamic differential features of speech signal features, so as to improve the accuracy of the feature parameters in describing the original speech signal features in the time domain and frequency domain.
[0068] The specific method for extracting the one-dimensional dynamic differential features of the speech signal features in the frequency domain and the two-dimensional dynamic differential features combined with the time domain is: using the differential spectrum of the static features based on the frequency cepstrum coefficients to describe the dynamic characteristics of the speech. The dynamic and static features are combined to improve the recognition performance of the runway operation call speech. The calculation of the dynamic differential features of the frequency uses the following formula:
[0069] ,
[0070] in: Indicates cepstral coefficients; represents the order of the cepstral coefficients; Represents the time difference of the one-dimensional derivative, which can be 1 or 2.
[0071] The two-dimensional dynamic differential features based on the one-dimensional dynamic differential features in the time domain are divided into different frames in the time domain. The calculations are obtained using the following formula:
[0072] ,
[0073] in, Indicates cepstral coefficients; represents the order of the cepstral coefficients; Represents the time difference of the one-dimensional derivative, which can be 1 or 2.
[0074] (2) Acoustic feature modeling sub-step: The acoustic feature is constructed using an end-to-end network structure based on convolutional neural networks, recurrent neural networks, and CTC functions. The specific method is as follows: first, the spatial correlation of the speech signal spectrum extracted by the convolutional neural network is calculated, then the temporal correlation of the speech signal extracted by the recurrent neural network and the correlation between speech signal frames are calculated, and finally, the output of the speech signal frame is combined with the text label through the CTC function, and the dynamic programming method is used to estimate the output text label sequence with the maximum probability.
[0075] (3) Language model decoding sub-step: Establish the voice call procedures and writing rules in the upper voice application environment, as well as the language and semantic correlation between the proper nouns involved (including but not limited to airline names, airport taxiway names, airport runway names, parking stand names, fixed configuration vocabulary, etc.), build a corpus, and calibrate the recognition results of the acoustic model based on the differences between the recognition results and the standard results in the corpus.
[0076] In some implementations, the main function of the language model is to elevate the speech recognition results to a specific application scenario, and to improve the performance of the latest speech recognition through the language and semantic relevance in the specific scenario.
[0077] Specific examples are as follows:
[0078] Corpus sample: (calling party), [responding party], {number / code / direction}, "keyword / word"
[0079] Ground handling: [Tower], {36R} (ground handling) "Calling";
[0080] Tower: {36R} (ground control), {West} [Tower], "Please speak";
[0081] Ground staff: [Tower], let me confirm the operating direction of this field.
[0082] Tower: {36R} (field management), "this field" {south} "towards" {north} "operating".
[0083] Ground Operations: "Received", "Moving" {South} "towards" {North}. [Tower], I have "arrived" {E8} "outside", "preparing to perform" {36R} "runway" comprehensive "inspection", two "vehicles" "enter" from {E8}, "need to enter" {36R} "runway sensitive area", estimated to take XX minutes;
[0084] Tower: {36R} (ground control), {36R} "Wait outside the runway" and "Change frequency" to "{Runway check}";
[0085] Ground Operations: [Tower], {36R} (ground operations) "Received", {36R} "Wait outside the runway" and "Change frequency" "{Runway check}";
[0086] The ground management switched to the "runway inspection" channel;
[0087] Operations: Tower, 36R Operations, I am outside E8, preparing to perform a comprehensive inspection of runway 36R. Two vehicles will enter from E8 and need to enter the sensitive area of runway 36R. It is estimated to take XX minutes.
[0088] Tower: 36R ground operations, tower, waiting outside runway 36R;
[0089] Airfield Operations: Tower, waiting outside runway 36R, 36R airfield operations received;
[0090] (wait)……………………
[0091] Tower: 36R, the tower is calling.
[0092] Ground Operations: Tower, 36R ground operations officer, please speak.
[0093] Tower: 36R Operations, runway 36R is clear for inspection (up);
[0094] Ground Operations: Runway 36R is ready for inspection (up), 36R Ground Operations received;
[0095] Ground Operations: Tower, two vehicles have entered Runway 36R;
[0096] Tower: 36R field service, tower received.
[0097] (Road inspection)…………………………(Inspection completed)
[0098] Ground services: Tower, runway inspection shows no abnormalities, runway 36R pavement is suitable.
[0099] Tower: 36R field operations, tower received, code XXX.
[0100] (Airfield management "Airfield Management" intercom channel: AOCC, the track inspection vehicle has left runway 36L. AOCC: 36L Airfield Management, AOCC received, end monitoring)
[0101] In some specific implementations, the implementation method of the upper voice standard judgment operation step is as follows: Figure 4 As shown, the method for judging the content recognized in the runway operation call voice recognition step and storing each group of judgment results in a database to apply the scoring rules to judge the total risk value includes:
[0102] Input the voice files that need to be judged, group the voice files and sort the dialogues in order, verify each group of dialogues sentence by sentence according to the standard terminology database, and get the judgment result.
[0103] In some specific implementations, the sentence-by-sentence verification method is: determining whether the standardized terms are included;
[0104] At the end of the call, determine whether the last two sentences of each conversation meet the repeated wording rules. If the judgment results are both yes, it is in compliance with the regulations;
[0105] The step of judging whether the wording is repeated includes: calculating the similarity m between the last two sentences, and the similarity m calculation formula is:
[0106] ,
[0107] When m is greater than 2, it is considered a repeated word, where q is the total number of words that exist in both string 1 and string 2, s is the total number of words that exist in string 1 but not in string 2, r is the total number of words that exist in string 2 but not in string 1, and Kq, Kr and Ks are the weights of q, r and s respectively.
[0108] In some specific implementations, the content of sentence-by-sentence verification includes: 1. Whether it contains standardized terms and vocabulary; 2. At the end of the call, whether the last two sentences of each conversation meet the repetition rule; Get the similarity of the last two sentences. If it is greater than 2, it is considered to be repeated words (for standard vocabulary, please refer to the standard call example above). Among them, q is the total number of words that exist in both string 1 and string 2, s is the total number of words that exist in string 1 but not in string 2, and r is the total number of words that exist in string 2 but not in string 1. Kq, Kr and Ks are the weights of q, r and s respectively. According to the actual calculation situation, set Kq=2, Kr=Ks=1. If the judgment results are all yes, it meets the specifications.
[0109] Finally, the total risk value is determined based on the judgment results through specific scoring rules. The specific scoring rules are as follows: based on the airport environment requirements, a time threshold for the judgment time range of the on-channel voice recording file is set, and a value is assigned to each on-channel voice call within the time threshold to determine whether it meets the judgment results. The time threshold is within 24 hours. The result that meets the specifications is assigned a risk value of 1 point, and the result that does not meet the specifications is assigned a risk value of 7 points.
[0110] Figure 5 A schematic diagram of the hardware structure of a terminal 600 provided in some embodiments of the present application is shown.
[0111] The terminal 600 may include a processor 601 and a memory 602 storing computer program instructions.
[0112] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0113] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 602 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory. The memory 602 may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Therefore, typically, the memory 602 includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it may perform the operations described in any of the above-mentioned embodiments of the airport on-road voice analysis and safety management methods.
[0114] The processor 601 implements any one of the airport on-road speech analysis and safety management methods in the above-mentioned embodiments by reading and executing computer program instructions stored in the memory 602 .
[0115] In one example, the terminal 600 may further include a communication interface 603 and a bus 610. Figure 5 As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 610 and communicate with each other.
[0116] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0117] Bus 610 includes hardware, software or both, and couples the components of online data traffic billing equipment to each other. For example, but not limitation, the bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnect (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. Where appropriate, bus 610 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.
[0118] In addition, in combination with the airport on-road speech analysis and safety management method in the above embodiments, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any of the airport on-road speech analysis and safety management methods in the above embodiments is implemented.
[0119] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0120] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0121] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0122] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method and system according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0123] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A method for analyzing and managing airport on-road speech and safety, characterized by: The recording equipment will connect the generated airport voice recording files to the OA network through the flight area network segment and transmit them to the safety status monitoring system server in real time; The safety status monitoring system server uses a python interpreter and a watch program. When the safety status monitoring system server receives data consisting of a new recording file, the watch program notifies the data analysis program in the upper voice analysis server. The upper voice analysis server receives the upper voice recording file from the safety status monitoring system server and applies the data analysis program to perform upper voice recognition analysis. After the Shangdao voice analysis server completes the Shangdao voice recognition analysis, it broadcasts the signal analyzed by the Shangdao voice recognition through port 8011 through UDP, and stores the analyzed data in the MySQL database; After receiving the signal that the Shangdao voice recognition and analysis is completed, the security status monitoring system server reads the analyzed data from the MySQL database, and performs a Shangdao voice standard judgment operation on the analyzed data according to the scoring rules, and judges the total risk value of the calls in the Shangdao voice recording file based on the scoring rules, and then counts and annotates the voice index item results; The above-mentioned speech recognition analysis process includes: Step 1, a voice cutting step, constructing a frame cutting model, inputting the airport voice recording file into the frame cutting model, performing frame cutting and extracting the fundamental frequency characteristics and time domain characteristics of the call data in the voice recording file; Step 2, a runway operation call speech recognition step, identifying the content corresponding to the fundamental frequency features and time domain features extracted in the on-track speech cutting step through a pre-built language and semantic relevance model applied to a special scenario; The frame cutting model is constructed in the following manner: Step (1), collecting samples of calls in the airport on-road voice signal recording file to form a voice signal sample set; Step (2), collecting a noise signal sample set; Step (3.1) framing: pre-emphasize and then frame the audio files in the speech signal sample set and the noise signal sample set, respectively, to obtain speech frame signals and noise frame signals; Step (3.2) windowing: the speech frame signal and the noise frame signal are subjected to discrete Fourier transform to obtain MFCC features, and windowing is performed using a Hamming window, the formula of which is as follows: , Where n is the time index, L is the window length, and w[n] represents the windowing process; Step (3.3), mixing the data in the speech signal sample set and the noise signal sample set after framing and windowing to generate mixed noise data, and obtaining a total of N noisy speech strips to be used as a test set for the model for testing after the model is trained; Step (4) model training, extracting a feature set for GMM training from the MFCC features, specifically, extracting a speech feature set and a noise feature set from the test set, wherein the speech feature set and the noise feature set include a feature set of a speech signal sample set and a noise signal sample set, and the feature set includes a fundamental frequency feature and a time domain feature; When extracting speech feature sets, the spatial correlation of the spectrograms of speech signal sample sets and noise signal sample sets is calculated through convolutional neural networks, and the temporal correlation of speech signal sample sets and noise signal sample sets as well as the correlation between speech signal sample sets are calculated through recurrent neural networks. Finally, the output of the speech signal sample set is combined with the text label through the CTC function to train the GMM model. Then the noise and speech models of the GMM to be trained are trained separately, where the noise and speech models are expressed as follows: , Where p(xk|z,rk) represents the noise or speech value calculated under the k-th sampling value using the expectation and the k-th sub-model probability, sita represents the covariance, uz represents the mean, xk represents the k-th sampling value, z represents the expectation, rk represents the k-th sub-model probability, and pi is 3.14; Step (5) upper voice cutting calculation, the upper voice cutting calculation step includes: calculating subband energy, B(m,n) represents the energy of subband m of the nth frame, and the calculation formula is as follows: , in, is the number of sample points in the subband, Indicates The energy value of the subband sample point within the frame, is the total number of subbands; It is also necessary to limit the excessive value of the sub-band energy. The limiting method is: when B(m,n)≥(E n / a), assign B(m,n) to 0; where a is a constant. In the runway operation call speech recognition scenario, a takes a value of 0.5; E n Indicates its short-term energy; Let BE(n) be the sub-band short-time energy of each frame calculated from the sub-band energy, and its noise average energy is: , in, is the number of sample points in the subband; Determine the signal type: Use the Gaussian mixture model to calculate the probabilities of speech and non-speech, and use the hypothesis testing method to determine the type of signal; The speech signal sample set includes speech signals of multiple voices, multiple speakers, multiple speaking speeds, accents, holidays and weekdays; the noise signal sample set includes takeoff noise of various aircraft models, vehicle driving noise around the runway, and noise signals under different weather conditions.
2. The method for airport on-road speech analysis and safety management as claimed in claim 1, characterized in that: When framing, the frame length of each frame is fixed, and the frame length is 10ms, 20ms or 30ms.
3. The method for airport on-road speech analysis and safety management as claimed in claim 1, characterized in that: The runway operation call voice recognition step comprises: Step (1) Speech signal feature extraction sub-step: extracting the upper speech signal including the noise background model, the speaker's speaking speed, and the speaker's pronunciation features, extracting the one-dimensional dynamic differential features of the frequency of the speech signal features and the two-dimensional dynamic differential features based on the one-dimensional dynamic differential features in the time domain based on the MFCC speech signal feature extraction method, and forming a speech signal spectrum diagram; The one-dimensional dynamic differential feature extraction method of the frequency is: using the differential spectrum of the static features based on the frequency cepstrum coefficients to calculate the dynamic differential characteristics of the speech signal features, and the calculation of the dynamic differential features of the frequency adopts the following formula: , in: Indicates cepstral coefficients, represents the order of the cepstral coefficients, Represents the time difference of the one-dimensional derivative, which can be 1 or 2; The two-dimensional dynamic differential feature based on the one-dimensional dynamic differential feature in the time domain is obtained by respectively calculating different sub-frames x in the time domain, and the calculation adopts the following formula: , in: Indicates The first frame cepstral coefficients, represents the order of the cepstral coefficients, Represents the time difference of the one-dimensional derivative, which can be 1 or 2; Step (2) Acoustic feature modeling sub-step: Calculate the spatial correlation of the speech signal spectrogram through a convolutional neural network, calculate the temporal correlation of the speech signal and the correlation between speech signals through a recurrent neural network, and finally combine the output of the speech signal with the text label through a CTC function, and use a dynamic programming method to estimate the output text label sequence with the maximum probability; Step (3) Language model decoding sub-step: Establish the voice call procedures, writing rules, and language and semantic correlation between the proper nouns involved in the upper voice application environment, build a corpus, and calibrate the recognition results of the acoustic model based on the differences between the recognition results and the standard results in the corpus.
4. The method for airport on-road speech analysis and safety management as claimed in claim 1, characterized in that: The above-mentioned voice standard judgment operation steps include: Input the audio files that need to be judged, group the audio files and sort the conversations in order; Verify each dialogue sentence by sentence according to the standard terminology database to obtain the judgment result; The total risk value is determined based on the judgment result through scoring rules, wherein the scoring rule is to set a time threshold for the judgment time range of the on-channel voice recording file based on the airport environment requirements, and assign a value to each on-channel voice call within the time threshold to determine whether it meets the judgment result.
5. The method for airport on-road speech analysis and safety management as claimed in claim 4, wherein the sentence-by-sentence verification method comprises: determining whether the standardized terms are included; At the end of the call, determine whether the last two sentences of each conversation meet the repeated wording rules. If the judgment results are both yes, it is in compliance with the regulations; in, The step of judging whether the last two sentences of each group of dialogues meet the repeated wording rule includes: calculating the similarity m of the last two sentences of each group of dialogues, and the similarity m calculation formula is: , When m is greater than 2, it is considered a repeated word, where q is the total number of words that exist in both string 1 and string 2, s is the total number of words that exist in string 1 but not in string 2, r is the total number of words that exist in string 2 but not in string 1, and Kq, Kr and Ks are the weights of q, r and s respectively.
6. An airport on-road voice analysis and safety management system, characterized by: It includes a dedicated hardware network architecture, the dedicated hardware network architecture includes a terminal and a server, the terminal includes a recording device, the server includes an OA network, a MySQL database, and an on-road voice analysis server and a safety status monitoring system server both connected to the OA network and the MySQL database, and the terminal is connected to the OA network through an airfield network segment; The dedicated hardware network architecture is used to execute the airport on-road speech analysis and safety management method described in any one of claims 1-5.
7. A terminal, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the airport on-road speech analysis and safety management method as described in any one of claims 1-5.
8. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executed by the airport on-road speech analysis and safety management method as described in any one of claims 1-5.
