A voice understanding and identity verification method and system for air-ground communication

Abstract

The application discloses a voice understanding and identity verification method and system for air-ground communication, and the method comprises the following steps: performing multi-level feature extraction on original air-ground communication voice to generate a high-dimensional mixed voice context embedding; the original voice comprises a mixed voice of multiple speakers; modeling the original voice based on a preset mask prediction network to generate a corresponding multi-speaker time-frequency mask; performing speaker prediction based on the mixed voice context embedding, the multi-speaker time-frequency mask and a preset processing strategy to obtain a voice segment set corresponding to each speaker; performing high-precision text transcription on the voice segment set corresponding to each speaker to obtain recognized text; and performing identity verification on the voice segment set corresponding to each speaker in combination with a pre-constructed registered voiceprint library and an identity recognition model to obtain a corresponding identity recognition result; the method has the beneficial effects that the application can improve the robustness of multi-speaker voice separation and recognition and adapt to an aviation environment.

Description

Technical Field

[0001] This invention belongs to the field of air traffic management and intelligent voice processing technology, specifically relating to a voice understanding and authentication method and system for air-to-ground communication. Background Technology

[0002] With the development of intelligent air traffic control systems, air-to-ground communication is gradually evolving towards intelligence and autonomy. Voice, as the core interactive medium for air traffic control, has become a key supporting technology due to its automatic understanding and reliable verification capabilities.

[0003] In existing air traffic control speech processing technologies, various speech analysis methods have been proposed. For example, early researchers applied traditional DNNs or LSTMs to air traffic control speech recognition. These methods are usually based on a phased processing framework and often employ a cascaded process in engineering implementation. In high-density communication environments (strong non-stationary aviation noise, such as engine noise and airflow interference), the boundary detection accuracy of these methods drops sharply. Furthermore, the cascaded "speaker segmentation-speech recognition-identity authentication" process is not only prone to error propagation but also increases inference latency due to repeated processing.

[0004] Subsequent scholars, such as Ma Guanglin et al., proposed end-to-end multi-task learning models based on Conformer. Although these models integrate convolutional and Transformer structures to take into account both local and global modeling, and combine Connectionist Temporal Classification (CTC) and attention mechanisms to improve recognition performance, they do not consider issues such as multi-speaker speech separation and identity verification. In complex air traffic control calls, they are prone to recognition errors due to speaker confusion and cannot handle common multi-speaker speech dialogues and identity recognition situations in real-world scenarios.

[0005] Furthermore, air-to-ground communication exhibits significant differences in acoustic and semantic features: ground control voice is typically clear and stable, while airborne voice is often superimposed with high-intensity noise and channel distortion. Existing general-purpose speech enhancement and recognition methods are not optimized for these acoustic characteristics of "multi-person, multi-turn, alternating high and low noise," resulting in insufficient robustness of the system in complex operating environments.

[0006] This reveals that existing technologies generally suffer from three major bottlenecks:

[0007] First, traditional speaker diarization (SD) methods have difficulty accurately distinguishing overlapping speech segments under strong noise interference (such as aircraft engines and airflow), resulting in input distortion for subsequent automatic speech recognition (ASR) and speaker verification (SV) modules.

[0008] Secondly, the system performs systematic cascade processing by taking SD, ASR and SV as independent modules, which not only introduces cumulative errors, but also causes computational redundancy due to repeated processing of the same speech signal, making it difficult to meet the stringent requirements for low latency in air traffic control scenarios;

[0009] Thirdly, general speech enhancement strategies do not fully model the unique acoustic characteristics of ground-air communication; that is, controllers' speech usually comes from low-noise environments, while pilots' speech is embedded in high-noise backgrounds. If this "high-noise-low-noise alternation" feature is not modeled specifically, the discriminative ability of speaker embedding will be seriously weakened;

[0010] Fourthly, the prior art lacks effective modeling capability for the terminology and expression modes unique to the aviation field, and is difficult to accurately understand the professional semantics in air traffic control scenarios; for special numerical pronunciations in the industry (e.g., "dong, yao, liang, guai" represent "0, 1, 2, 7" respectively), standard call terms (e.g., "Bravo", "Tango", "Yankee", "Romeo", etc.) and key command information (e.g., flight numbers, waypoints, codes, etc.), existing methods generally have such problems as low recognition accuracy, insufficient semantic parsing capability and incomplete structured extraction, thereby seriously affecting the accuracy and reliability of subsequent structured processing of air traffic control commands and automatic command generation for simulators.

[0011] Therefore, how to construct an integrated method for speech understanding and identity trustworthiness oriented to ground-air communication scenarios, realize the collaborative optimization of speech recognition, semantic parsing and identity verification, and support the trustworthy operation of large-scale air traffic control models has become a key technical problem that needs to be solved urgently. Summary of the Invention

[0012] In view of the technical defects mentioned in the Background, an object of embodiments of the present invention is to provide a speech understanding and identity verification method and system oriented to ground-air communication, which aims to solve at least one of the technical problems in the related art to a certain extent.

[0013] To achieve the above object, in a first aspect, the present invention provides a speech understanding and identity verification method oriented to ground-air communication, the method comprising:

[0014] Performing multi-level feature extraction on original speech of ground-air communication to generate high-dimensional hybrid speech context embedding; wherein the original speech comprises multi-speaker hybrid speech;

[0015] Modeling the original speech based on a preset constructed mask prediction network to generate corresponding multi-speaker time-frequency masks;

[0016] Speaker prediction is performed based on the hybrid speech context embedding, the multi-speaker time-frequency mask, and the preset processing strategy to obtain a set of speech segments corresponding to each speaker;

[0017] The speech segments corresponding to each speaker are transcribed with high precision to obtain the recognized text; wherein, during the transcription, the text is analyzed and verified using a rule base with air traffic control characteristics awareness.

[0018] The voice segments corresponding to each speaker are combined with a pre-built registered voiceprint library and identity recognition model for identity verification to obtain the corresponding identity recognition result; wherein, during verification, a segmentation reuse mechanism is adopted to reduce computational overhead.

[0019] As a specific implementation of this application, the steps for generating a high-dimensional hybrid speech context embedding include:

[0020] The mixed speech of multiple speakers is converted into a complex spectrum using a short-time Fourier transform;

[0021] A Mel filter bank is used to perform nonlinear frequency mapping on the amplitude spectrum to generate a Mel spectrum; wherein the amplitude spectrum is the modulus value of the complex spectrum;

[0022] The Mel spectroscopic features contained in the Mel spectrogram are then expanded in dimension and enhanced in feature through a deployed learnable network, and finally the hybrid speech context embedding is output.

[0023] As a specific implementation of this application, the method of modeling the original speech based on a pre-constructed mask prediction network to generate a corresponding multi-speaker time-frequency mask specifically includes:

[0024] The input raw speech is converted into a time spectrum using short-time Fourier transform;

[0025] The time-frequency spectrum is transmitted to the mask prediction network to obtain the corresponding multi-speaker time-frequency mask; wherein, the mask prediction network adopts the U-Net architecture, and the skip connections introduced in U-Net directly transmit the low-level fine-grained time-frequency structure of the encoding stage to the feature map of the corresponding scale in the decoding stage, thereby realizing the effective fusion of features at different scales.

[0026] As a specific implementation of this application, the mask prediction network, during training, is based on high-noise speech data collected in real air traffic control scenarios and mixed with clean speech recorded in a low-noise environment according to a random signal-to-noise ratio to construct training samples with real noise statistical characteristics.

[0027] Meanwhile, the supervised learning strategy adopted by the network is to predict the ideal ratio mask of the corresponding speaker from the mixed speech signal, thereby learning the ability to accurately estimate the energy distribution of speech components in complex noise background.

[0028] As a specific implementation of this application, the processing strategy for speaker prediction includes dual-path feature fusion, high-dimensional semantic enhancement, attention interaction modeling, feedforward network enhancement, and speaker prediction.

[0029] The dual-path feature fusion is used to perform element-wise dot product between the hybrid speech context embedding and the multi-speaker time-frequency mask to obtain a joint representation of each speaker; wherein, the dimensions of the hybrid speech context embedding are [1, 400, 512], where 1 represents the batch size, 400 represents the number of time frames, and 512 represents the frequency dimension; the dimensions of the multi-speaker time-frequency mask are [num_spk, 400, 512], where num_spk represents the number of speakers, 400 represents the number of time frames, and 512 represents the frequency dimension;

[0030] The high-dimensional semantic enhancement is used for:

[0031] The dimensionality of the fused features is compressed and reduced by linear mapping to decrease computational complexity.

[0032] Rotational position encoding is used to model the positional information of time series.

[0033] The attention interaction modeling is used to perform information interaction in the "speaker × time" dimension using a multi-head attention mechanism, so that each speaker's representation can refer to other speakers and the global context;

[0034] The feedforward network enhancement is used to improve the feature representation capability using a feedforward neural network, and to maintain training stability and keep the time dimension structure unchanged through residual connections.

[0035] The speaker prediction method uses LSTM to capture past and future temporal context, compresses the dimension through a linear layer, obtains the activation probability of different speakers in each time frame through a Sigmoid activation layer, and clusters to obtain the set of speech segments corresponding to each speaker.

[0036] As one specific implementation of this application, the high-precision text transcription is processed through a preset speech recognition process, specifically including:

[0037] The original text is obtained by automatic recognition through a speech recognition model that has been fine-tuned in the domain beforehand; wherein, the speech recognition model is fine-tuned and trained using a constructed aviation-specific corpus to adapt to the aviation communication context; wherein, the speech recognition model adopts the lightweight SenceVoiceSmall model.

[0038] Using regular expression templates and context rules, structured fields in aviation instructions are automatically identified and extracted from the original text, and all fields are checked for format validity.

[0039] The recognition results are analyzed and verified for semantic integrity and logical rationality of instructions using a rule base with air traffic control characteristics.

[0040] As a specific implementation of this application, the registered voiceprint database contains a mapping table of <user ID, registration embedding vector>, and the identity recognition model adopts a fine-tuned ECAPA-TDNN model; wherein, the ECAPA-TDNN model is fine-tuned in the training phase using air traffic control corpus to enhance its modeling ability for aviation terminology, communication style and channel characteristics.

[0041] During registration, the ECAPA-TDNN model is used to process the registered speech, extract the high-dimensional speaker embedding vector, and use it as the registration embedding vector, denoted by ej.

[0042] During verification, the following process is executed for each speaker's voice stream separated in real time:

[0043] All speech segments belonging to the same speaker i are spliced ​​together to generate a representative speech signal Si;

[0044] The speaker's voice Si is input into the finely tuned ECAPA-TDNN model, and the output is the embedding vector ei;

[0045] Calculate the similarity between ei and all registered embedding vectors ej in the registered voiceprint database; if there exists an i such that the similarity is greater than the threshold, then determine that the current speaker is a registered user ID and perform a match; otherwise, consider it an unknown speaker.

[0046] Secondly, the present invention also provides a voice understanding and authentication system for land-air communication, the system comprising:

[0047] The feature encoding module is used to perform multi-level feature extraction on the raw speech of land-air communication to generate a high-dimensional mixed speech context embedding; wherein, the raw speech includes multi-speaker mixed speech;

[0048] The mask modeling module models the original speech based on a pre-constructed mask prediction network and generates corresponding multi-speaker time-frequency masks.

[0049] The feature fusion and clustering module performs speaker prediction based on the hybrid speech context embedding, the multi-speaker time-frequency mask, and a preset processing strategy to obtain a set of speech segments corresponding to each speaker.

[0050] The speech recognition module is used to perform high-precision text transcription of the speech segments corresponding to each speaker to obtain the recognized text; wherein, during transcription, the rule base with air traffic control characteristics is used for analysis and verification.

[0051] The identity verification module is used to combine the set of voice segments corresponding to each speaker with a pre-built registered voiceprint library and identity recognition model to perform identity verification and obtain the corresponding identity recognition result; wherein, during verification, a segmentation reuse mechanism is adopted to reduce computational overhead.

[0052] The technical solution provided by the embodiments of the present invention has the following advantages:

[0053] 1. Improve the robustness of multi-speaker speech separation and recognition: Construct high-dimensional context embedding by using Mel spectrum and learnable network for feature encoding, effectively preserving the acoustic features of air-to-ground communication in aviation noise environment; Introduce multi-speaker mask modeling based on U-Net architecture, make full use of time-frequency image characteristics, realize adaptive modeling of complex acoustic environment of "high noise-low noise alternation", and generate high-fidelity, continuous time-frequency mask.

[0054] By combining a dual-path feature fusion mechanism with speaker prediction, speaker activation states are accurately modeled at the time frame level, effectively solving problems such as overlapping multiple speakers, frequent speaker switching, and incomplete speech segments. Compared with traditional methods based on energy thresholds or shallow clustering, it exhibits stronger stability and consistency in complex acoustic environments.

[0055] Meanwhile, this method does not rely on specific speakers or fixed scene distributions, has good cross-scene generalization ability, and can be promoted and applied in different air traffic control communication environments, other high-noise and multi-speaker scenarios, and has high engineering implementation value.

[0056] 2. Adapt to aviation communication context and improve recognition accuracy: By integrating air traffic control characteristics, the system specifically verifies key fields and command logic in the air-to-ground communication environment; and by using a constructed aviation-specific corpus, the speech recognition model has undergone full parameter fine-tuning of real air traffic control data on professional terminology, pronunciation habits, and channel characteristics in the air traffic control environment, thereby improving accuracy.

[0057] 3. Reduce computational overhead and meet the real-time requirements of high concurrency and low latency in air traffic control: By proposing a "segmentation multiplexing mechanism", embedding extraction and matching are performed only once for multiple speech segments of the same speaker, avoiding redundant calculations; compared with the segment-by-segment verification scheme, the throughput of speaker identification is significantly enhanced.

[0058] The speech recognition model adopts the lightweight SensenceVoiceSmall model and is fine-tuned with aviation corpus, effectively controlling inference latency and meeting the stringent real-time requirements of air-to-ground communication.

[0059] 4. Achieve efficient collaboration between each step, build an end-to-end trusted communication analysis link, share intermediate representations and joint optimization, and avoid information fragmentation; at the same time, the overall process does not require additional noise reduction preprocessing, retains the real channel noise as part of the identity features, and thus improves the speaker embedding ability in actual deployment. Attached Figure Description

[0060] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0061] Figure 1 This is the architecture diagram corresponding to the technical solution of this invention;

[0062] Figure 2 This is a flowchart of a voice understanding and authentication method for land-air communication provided in an embodiment of the present invention;

[0063] Figure 3 This is a flowchart of generating hybrid speech context embeddings provided in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of the structure of a mask prediction network provided in an embodiment of the present invention;

[0065] Figure 5 This is a flowchart of speaker prediction provided in an embodiment of the present invention;

[0066] Figure 6 This is a schematic diagram illustrating a specific implementation result provided by an embodiment of the present invention;

[0067] Figure 7 This is a principle block diagram of a voice understanding and authentication system for land-air communication provided in an embodiment of the present invention. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0070] Throughout this specification, references to "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with that embodiment or example is included in at least one embodiment of the invention. Therefore, the phrases "in an embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, or characteristics can be combined in one or more embodiments or examples in any suitable combination and / or sub-combination.

[0071] It should be noted that, unless otherwise stated, the technical terms used in this embodiment have the common meaning as understood in the relevant technical field.

[0072] The inventive concept of this invention is as follows: In air traffic control scenarios involving land and air communication, controllers need to conduct multiple rounds of interleaved voice communication with pilots on the same high-frequency channel. This communication is often affected by weather, airflow, and electromagnetic interference, leading to a decline in voice quality. To achieve accurate understanding and safe monitoring of the content of land-air communications, it is essential to first solve the problem of speaker separation and recognition in complex multi-person interactions and high-noise environments. Only by accurately distinguishing the identities of different speakers can the voice signals be correctly attributed and transcribed, thereby supporting subsequent semantic understanding, instruction repetition verification, and normative analysis. Therefore, multi-speaker voice separation and speaker recognition are key prerequisites for building reliable intelligent analysis of air traffic control voice signals.

[0073] The present invention provides a voice understanding and authentication method and system for air-to-ground communication, designed to address the aforementioned challenges. Figure 1 As shown, its overall principle is to complete the entire link from "multi-speaker original speech → separation → recognition → identity binding" through five steps: "1. Land-to-air communication feature encoding + 2. Multi-speaker mask modeling + 3. Dual-path feature fusion and adaptive speaker clustering + 4. Enhanced speech recognition with aviation command awareness + 5. Segmented multiplexing speaker authentication". This enables stable and reliable autonomous communication analysis capabilities in high-density airspace land-to-air communication environments.

[0074] Please refer to Figure 2 This invention provides a voice understanding and authentication method for air-to-ground communication, the method comprising:

[0075] S101, perform multi-level feature extraction on the original speech of the land-air communication to generate a high-dimensional mixed speech context embedding; wherein, the original speech includes multi-speaker mixed speech;

[0076] S102, The original speech is modeled based on a pre-constructed mask prediction network to generate a corresponding multi-speaker time-frequency mask;

[0077] S103, based on the hybrid speech context embedding, the multi-speaker time-frequency mask and the preset processing strategy, speaker prediction is performed to obtain a set of speech segments corresponding to each speaker;

[0078] S104, perform high-precision text transcription on the set of speech segments corresponding to each speaker to obtain the recognized text; wherein, during transcription, the text is analyzed and verified by a rule base with air traffic control characteristics.

[0079] S105, the set of voice segments corresponding to each speaker is combined with the pre-built registered voiceprint library and identity recognition model to perform identity verification and obtain the corresponding identity recognition result; wherein, during verification, a segmentation reuse mechanism is adopted to reduce computational overhead.

[0080] In application, step S101 can be understood as the aforementioned land-air communication feature encoding. This step performs multi-level feature extraction on the original land-air communication speech signal to generate a high-dimensional context embedding E (referred to as "hybrid speech context embedding"), including:

[0081] First, the mixed speech signal containing multiple speakers is input into the system and converted into a complex spectrum through Short-Time Fourier Transform (STFT);

[0082] Subsequently, the amplitude spectrum is nonlinearly frequency-mapped using the Mel filter bank to generate a Mel-spectrogram.

[0083] The Mel scale is a nonlinear frequency scale that simulates the characteristics of human auditory perception: it has high resolution in the low-frequency region and low resolution in the high-frequency region. This nonlinear compression is more in line with the characteristics of human speech perception, especially in the case of aviation noise, it can effectively preserve key frequency band information related to speaker identity. Therefore, compared with the original linear spectrum, the Mel spectrum is more suitable for subsequent speaker representation modeling.

[0084] The final Mel spectrogram has dimensions (Batch, Time, Frequency) = (1, 400, 80), where the number of time frames t = 400 and the frequency dimension f = 80, representing 80 Mel frequency bands, which serve as the initial input for high-dimensional acoustic feature extraction.

[0085] While the 80-dimensional spectral density effectively characterizes the time-frequency energy distribution of speech, its dimensionality is relatively low and it lacks the ability to explicitly model speaker discriminative information. More importantly, subsequent step 3 employs a Transformer-based architecture for multi-speaker modeling. Standard Transformer models typically require input embeddings to have a fixed and high dimensionality to support the self-attention mechanism in effectively capturing long-range contextual dependencies and to provide sufficient expressive capacity to distinguish subtle speaker differences.

[0086] Therefore, it is necessary to map the original 80-dimensional Mel spectrum features to a higher-dimensional embedding space that is adapted to Transformer through a learnable network.

[0087] To this end, this invention designs a lightweight and efficient "feature encoding" module as the main structure of the learnable network: the module uses a one-dimensional long short-term memory (LSTM) network as its core to model the temporal dependence of speech signals and improve the feature dimension; at the same time, layer normalization (LayerNorm) is introduced to stabilize the training process, a Gaussian error linear unit (GELU) is used as a smooth activation function, and a dropout mechanism is used to prevent overfitting.

[0088] This "feature encoding" module effectively performs dimensionality expansion and feature enhancement while preserving temporal context information; the specific network structure and parameter configuration are as follows: Figure 3 As shown:

[0089] The final output feature E∈R(1×400×512) is the “hybrid speech context embedding”, which will be used later.

[0090] Step S102 can be understood as the aforementioned multi-speaker mask modeling. This step serves the speaker prediction task, and its core objective is to model the mixed speech signal of multiple speakers and generate the corresponding multi-speaker time-frequency mask (Mask, M). This mask will serve as important prior information for subsequent speaker discrimination and speech recognition.

[0091] To achieve this goal, the present invention focuses on the following factors:

[0092] 1. Alternating High-Noise and Low-Noise Acoustic Environment: In air traffic control (ATC) scenarios, voice signals are interfered with by highly non-stationary noise. Specifically, controllers are typically in relatively quiet indoor environments, while pilots are exposed to complex noise conditions such as engine roar, airflow disturbances, and dense wireless communications. This type of noise exhibits strong time-varying and localized characteristics in both time and frequency dimensions, resulting in voice signals exhibiting alternating high-noise and low-noise characteristics.

[0093] 2. Image-based characteristics of the input representation: In this invention, the input speech signal is first converted into a time-frequency spectrum through a Short-Time Fourier Transform (STFT), with dimensions of time frame t × frequency f, which is formally equivalent to a single-channel grayscale image. This two-dimensional structure naturally fits the image processing paradigm. Therefore, this invention uses a Convolutional Neural Network (CNN) for modeling, rather than relying on sequence modeling methods (such as LSTM), to more efficiently capture local time-frequency patterns.

[0094] 3. Context Awareness and Multi-Scale Modeling Requirements: In multi-speaker mixed speech, the speech activities of different speakers are often highly intertwined in time and frequency, manifesting as energy overlap and rapid switching within local time-frequency regions. Effective time-frequency masking estimation not only needs to distinguish the overall feature differences between different speakers but also must accurately characterize fine-grained structures such as speech start-end boundaries, transient plosives, and weak-energy sounds. Therefore, the model needs to possess multi-scale context awareness capabilities: on the one hand, it should be able to capture global differences at the speaker level using a wide range of contextual information; on the other hand, it should be able to accurately locate active speech regions at the local scale, thereby achieving a balance between noise suppression and speech fidelity.

[0095] In this embodiment, the mask prediction network based on a preset structure models the original speech and generates a corresponding multi-speaker time-frequency mask, specifically including:

[0096] The input raw speech is converted into a time spectrum using short-time Fourier transform;

[0097] The time-frequency spectrum is transmitted to the mask prediction network to obtain the corresponding multi-speaker time-frequency mask; wherein, the mask prediction network adopts the U-Net architecture, and the skip connection introduced in U-Net directly transmits the low-level fine-grained time-frequency structure of the encoding stage to the feature map of the corresponding scale in the decoding stage, thereby realizing the effective fusion of features at different scales.

[0098] During training, the mask prediction network is based on high-noise speech data collected in real air traffic control scenarios and mixed with clean speech recorded in low-noise environments according to random signal-to-noise ratios to construct training samples with real noise statistical characteristics.

[0099] Meanwhile, the supervised learning strategy adopted by the network is to predict the ideal ratio mask of the corresponding speaker from the mixed speech signal, thereby learning the ability to accurately estimate the energy distribution of speech components in complex noise background.

[0100] The U-Net architecture (U-shaped Encoder–Decoder Network) serves as the core network structure for mask modeling.

[0101] The U-Net architecture, with its ability to jointly model local details and global context, is particularly well-suited for processing speech signals with complex time-frequency structures and highly non-stationary noise. Its encoder extracts high-resolution local features, such as harmonic structures, energy transition edges, and speech transient start points, while compressing the time-frequency resolution through layer-by-layer downsampling. The decoder, on the other hand, restores the time-frequency resolution through layer-by-layer upsampling, extracting high-level semantic information (such as speaker identity and speech activity state). The skip connections introduced in U-Net directly pass the low-level, fine-grained time-frequency structure from the encoding stage to the corresponding scale feature map in the decoding stage, achieving effective fusion of features at different scales. This design enhances the model's overall ability to distinguish speakers and suppress noise while maintaining the integrity of speech details, thereby generating more accurate, continuous, and robust multi-speaker time-frequency masks.

[0102] Therefore, the U-Net architecture can not only adaptively model non-stationary noise characteristics, but also preserve speech structure information to the greatest extent while effectively suppressing noise, thereby significantly improving the robustness of subsequent speaker prediction and automatic speech recognition (ASR) tasks.

[0103] The encoder-decoder architecture designed in the U-Net style in this embodiment of the invention is as follows:

[0104] The encoder consists of four repeating modules, each of which sequentially includes convolution, max pooling, batch normalization, and a non-linear activation function.

[0105] The decoder also consists of four modules, each of which performs upsampling, convolution, batch normalization, and activation operations in sequence.

[0106] At the same time, through skip connections (such as Figure 4 (As shown by the horizontal arrow in the image) cross-layer connections are made between feature maps of the same scale in the encoder and decoder to achieve multi-scale feature fusion.

[0107] The final output layer generates a multi-speaker time-frequency mask with values ​​in the range [0,1] using the Sigmoid activation function, which corresponds to the proportion of each target speaker.

[0108] The entire encoder-decoder structure constitutes a trainable multi-speaker mask prediction network, the detailed configuration of which is as follows: Figure 4 As shown.

[0109] During the training phase, high-noise speech data collected in real air traffic control scenarios is mixed with clean speech recorded in low-noise environments according to the random signal-to-noise ratio (SNR) to construct training samples with realistic noise statistical characteristics. This data construction method effectively simulates the complex and highly non-stationary noise conditions in actual air traffic control communications.

[0110] The network of this invention employs a supervised learning strategy to predict the ideal ratio mask (IRM) for the corresponding speaker from mixed speech signals, thereby learning the ability to accurately estimate the energy distribution of speech components in complex noise backgrounds. Unlike binary masks that use hard decision, IRMs are continuously valued soft masks, capable of more finely characterizing the energy ratio relationship between speech and noise in the time and frequency domains. They exhibit superior robustness and reconstruction accuracy under non-stationary noise conditions such as engine noise and airflow disturbances.

[0111] This invention uses IRM as the supervision target, enabling the noise mask prediction network to model the relative energy distribution of speech and noise more precisely, thereby preserving speech details to the greatest extent while suppressing noise.

[0112] IRM is defined as the ratio of clean speech to noise in the time-frequency domain amplitude spectrum, and its mathematical expression is as follows:

[0113] ;

[0114] in: This represents the STFT complex coefficients of the i-th speaker at time frame t and frequency frame f;

[0115] This represents the noise component at the corresponding time frame t and frequency frame f.

[0116] |⋅| indicates taking the amplitude spectrum (i.e., taking the modulus value);

[0117] The closer the mask value M(t,f) is to 1, the more dominant the speaker i is in that time-frequency unit; the closer it is to 0, the more dominant the noise and other speakers are.

[0118] This step yields the multi-speaker time-frequency mask M(num_spk,400,512).

[0119] Step S103 can be understood as the aforementioned dual-path feature fusion and adaptive speaker clustering. This step aims to predict the activation probability of each speaker in each time frame in mixed speech. Based on the results extracted in the first two steps, a joint feature with strong discriminative ability is constructed.

[0120] Multi-speaker time-frequency mask M (dimension is [num_spk,400,512]), where 400 is the number of time frames and 512 is the frequency point dimension;

[0121] The mixed speech context embedding E (dimensions [1, 400, 512]) is extracted holistically from the mixed speech and is used to represent global consistency and contextual information across speakers.

[0122] The specific steps are encapsulated into the processing strategy described below:

[0123] 1. Dual-path fusion: Element-wise fusion (dot product) of M and E is used to obtain a joint representation of each speaker, which preserves local speaker features while introducing global contextual information to improve feature consistency.

[0124] 2. High-dimensional semantic enhancement: First, the 512-dimensional features are compressed to 256 dimensions through linear mapping to reduce computational complexity; Rotary Position Embedding (ROPE) is used to model the positional information of the time series, which facilitates subsequent processing to capture order dependencies;

[0125] Specifically, RoPE encodes position information into the Transformer feature representation by introducing rotation operations in a two-dimensional subspace. For a hidden layer of dimension d (d=256 in this scheme), the feature dimension is divided into d / 2 two-dimensional subspaces, each corresponding to an independent rotation frequency. The two-dimensional rotation matrix is ​​represented as follows:

[0126]

[0127] in, Let represent the rotation angle corresponding to the k-th two-dimensional rotation subspace when the position index is i, and its definition is as follows:

[0128]

[0129] Where: i represents the position index of the input feature in the time series, k represents the two-dimensional rotation subspace number, with a value range of 0, 1, ..., d / 2−1, and d represents the hidden layer dimension;

[0130] For an input feature with position index i, RoPE constructs a rotation matrix based on the position index and rotation angle, mapping the input feature to the rotated representation space, thereby encoding the position information, as shown below:

[0131]

[0132] 3. Attention Interaction Modeling: Multi-Head Attention (MHA) is used to perform information interaction in the "speaker × time" dimension, so that each speaker's representation can refer to other speakers and the global context, thereby improving the discriminativeness;

[0133] 4. Feed-Forward Neural Network: A feed-forward neural network (FFN) is used to improve feature representation capabilities. Residual connections are used to maintain training stability and preserve the temporal structure.

[0134] 5. Speaker Prediction: LSTM is used to capture past and future temporal context. Dimensionality is compressed (256→1) through a linear layer, followed by a Sigmoid activation layer to obtain the activation probabilities of different speakers in each time frame; the specific network structure is as follows... Figure 5 As shown.

[0135] Step S104 can be understood as the aforementioned enhanced speech recognition for aviation command awareness. This step requires high-precision text transcription of the speech segments corresponding to each speaker. To this end, the present invention proposes an enhanced speech recognition process for aviation command awareness (i.e., a preset speech recognition process), the core objective of which is to accurately and efficiently convert aviation-related speech signals into text in complex communication environments.

[0136] 1. ASR Model Selection and Domain Fine-tuning

[0137] This invention selects SensenceVoiceSmall as the basic ASR architecture, mainly based on the following advantages:

[0138] Low inference latency: The model has a small number of parameters and high computational efficiency, meeting the stringent real-time requirements in air-to-ground communication (latency <300ms).

[0139] Strong robustness: Natively supports multiple accents, noisy and narrowband channel voice, and performs stably in real air traffic control communication environments;

[0140] Open source and customizable: Provides a complete training interface, making it easy to introduce domain data for fine-tuning.

[0141] Based on this, the present invention has collected and constructed an aviation-specific corpus, collected and cleaned a large number of real air-to-ground communication recordings, covering typical scenarios such as standard ICAO English, Chinese control instructions, pilot repetitions and emergency call signs;

[0142] Standardized annotation: The speech is transcribed word by word, and key aviation terms (such as hole, turn, 1, approach, etc.) are carefully verified;

[0143] Fine-tuning training: Based on the SenceVoiceSmall pre-trained weights, the above aviation corpus (over 70,000 entries) was used to fine-tune all parameters, so that the model could fully learn the vocabulary distribution, pronunciation habits and communication protocols in the air traffic control field;

[0144] In application, the original text is obtained by automatic recognition through a speech recognition model that has been fine-tuned in advance for the domain. The speech recognition model is fine-tuned and trained using a constructed aviation-specific corpus to adapt to the aviation communication context. According to actual tests, the accuracy of the fine-tuned model on the test set reached 99.4%.

[0145] 2. Empty pipe characteristic sensing

[0146] (1) Key field structured extraction and format validation

[0147] Using regular expression templates and context rules, structured fields in aviation instructions are automatically identified and extracted from raw ASR text, including:

[0148] For height levels (e.g., “decrease to 6,000” → 6,000), check if it is within the standard height level range (the standard height level usually needs to be within 10,000).

[0149] Heading (e.g., "turn left heading two turns to zero" → 270) is forcibly converted to a three-digit number format of 0–360, and invalid values ​​(e.g., "400") are removed.

[0150] Use a speed command (e.g., “indicator speed 230 knots” → 230) to verify if the speed is within the typical speed range of the model.

[0151] Flight call signs (such as “Air China 981” → CCA981) are standardized and restored by combining airline prefixes and number pronunciation rules (hole=0, "1", turn=7).

[0152] All fields undergo format validity checks. If the recognition result violates aviation communication regulations (e.g., heading "366 degrees" or altitude "53,000"), the system will trigger an error correction process.

[0153] (2) Instruction logic consistency verification

[0154] Aviation instructions often appear in fixed phrases (e.g., "approach clearance" usually includes runway number, glide slope approach heading, altitude restrictions, etc.). The semantic completeness and logical rationality of the recognition results are analyzed using a built-in air traffic control instruction template library (based on ICAO Doc 4444 and standard Chinese civil aviation terminology).

[0155] This type of verification does not rely on additional voice input, but is based solely on built-in custom text regularization rules. It has low computational overhead but significantly corrects the accuracy of instructions. The rule base with air traffic control characteristics awareness mentioned above can be understood as a domain rule base that integrates aviation communication specifications, air traffic control standard instruction templates, aviation domain knowledge, and flight operation constraints. This rule base utilizes the strongly standardized expression characteristics in air traffic control communication to perform structured parsing, format verification, and logical consistency checks on ASR recognition results.

[0156] It should be noted that the air traffic control characteristics mentioned mainly refer to the inherent domain features of air traffic control voice, such as standardized expressions, aviation numerical expression rules, parameter value range constraints, and command context logic relationships. For example, field rules such as heading not exceeding 360 degrees, altitude not exceeding 6000 meters, fixed waypoints, and fixed flight call signs require the legality judgment of the recognition results; through command template matching, non-existent fields, abnormal parameters, and semantic conflicts are detected, thereby improving the reliability of air traffic control voice recognition results.

[0157] This mechanism has effectively corrected approximately 8.3% of the high-risk keyword identification error rate in actual testing.

[0158] Step S105 can be understood as the aforementioned segmented multiplexing speaker authentication. This step aims to efficiently and accurately complete the speaker authentication (SV) task based on the preceding speech separation and speaker clustering results. Its core idea is "segmented multiplexing": that is, for multiple speech segments of the same speaker appearing in a multi-speaker mixed speech, only one identity embedding extraction and matching operation is performed, thereby significantly reducing the system's computational overhead while ensuring recognition accuracy. It is particularly suitable for air-to-ground communication scenarios with high concurrency and low latency requirements.

[0159] The speaker segmentation result output in the third step above is a set of speech segments s={S1,S2,...,SN} organized by speaker ID, where each Si={xi1,xi2,...,xik} represents several speech segments belonging to the i-th speaker.

[0160] Step S105 is divided into two stages: registration and verification.

[0161] register:

[0162] First, each authorized user (such as a specific controller or pilot) provides one or more clear speech samples. The registered speech is then processed using the ECAPA-TDNN (Emphasized Channel Attention, Propagation and Aggregation TimeDelay Neural Network) model to extract high-dimensional speaker embedding vectors.

[0163] Fine-tuning and adaptation: To improve adaptability in the aviation domain, the ECAPA-TDNN model uses air traffic control corpus for domain fine-tuning during the training phase, enhancing its ability to model aviation terminology, communication styles and channel characteristics.

[0164] Construct a mapping table: Store each user's unique identifier ID and its corresponding embedding vector in the database to form a mapping table <user ID, registration embedding vector> for subsequent matching; for ease of description, ej represents the registration embedding vector.

[0165] It is important to note that this section does not include a speech noise reduction module. This is because in high-density airspace autonomous control scenarios, controller speech is typically in a low-noise environment, while pilot speech contains a significant amount of background noise (such as engine noise and radio interference). In real-world environments, preserving this type of real noise actually helps improve the discriminativeness of speaker embedding—since noise has become part of the speaker's acoustic characteristics, directly using the original noisy speech for embedding extraction yields a more robust identity representation.

[0166] verify:

[0167] For each speaker's speech stream separated in real time, the following process is executed:

[0168] For all segments {xi1,...,xik} belonging to the same speaker i, concatenate them to generate a representative speech signal Si to improve embedding stability;

[0169] The speaker's voice Si is input into the finely tuned ECAPA-TDNN model, and the output is the embedding vector ei;

[0170] Calculate the similarity between ei and all registered embeddings ej in the registry;

[0171] If there exists an i such that the similarity is greater than the threshold τ, then the current speaker is determined to be a registered user ID; otherwise, the speaker is considered an unknown speaker.

[0172] Because this invention performs embedding extraction and matching operations only once for each speaker (rather than repeatedly verifying each speech segment), it can significantly reduce the number of model inferences in multi-round, alternating speech dialogues. For example, in a dialogue containing 5 alternating speeches, traditional segment-by-segment verification requires 5 SV calculations, while this method only requires 2, reducing computational overhead by 60%, significantly improving system throughput, and meeting the requirements of high-concurrency operation.

[0173] That is, the registered voiceprint database contains a mapping table of <user ID, registration embedding vector>, and the identity recognition model adopts a fine-tuned ECAPA-TDNN model; wherein, the ECAPA-TDNN model is fine-tuned in the training phase using air traffic control corpus to enhance its ability to model aviation terminology, communication style and channel characteristics.

[0174] During registration, the ECAPA-TDNN model is used to process the registered speech, extract the high-dimensional speaker embedding vector, and use it as the registration embedding vector, denoted by ej.

[0175] During verification, the following process is executed for each speaker's voice stream separated in real time:

[0176] All speech segments belonging to the same speaker i are spliced ​​together to generate a representative speech signal Si;

[0177] The speaker's voice Si is input into the finely tuned ECAPA-TDNN model, and the output is the embedding vector ei;

[0178] Calculate the similarity between ei and all registered embedding vectors ej in the registered voiceprint database; if there exists an i such that the similarity is greater than the threshold, then determine that the current speaker is a registered user ID and perform a match; otherwise, consider it an unknown speaker.

[0179] In practical implementation, taking an airport air-to-ground communication recording as an example, the recording duration is over 90 seconds, including one air traffic controller and three crew members. The original audio contains both engine noise and radio noise. After the original recording is input, an enhanced audio signal is obtained through a trainable mask prediction network, followed by speaker segmentation, resulting in the following segmentation results:

[0180] spk0: 0~7s, 20~30s, 41~51s, 53~67s;

[0181] spk1: 9~16s;

[0182] spk2: 35~41s;

[0183] SPK3: 70~76s;

[0184] Further automatic speech recognition was performed on each segment to obtain the recognized text:

[0185] spk0: Approaching 588, continuing to descend, corrected to maintain 45;

[0186] SPK1: Approach radar identified, rising to standard air pressure of 4,500;

[0187] spk2: 14.85, goodbye to hole 166;

[0188] spk3: Goodbye, Jinjin 83, Tianfu 6559;

[0189] Speaker verification is then performed: spk* is compared with the registered voiceprint database; the identity recognition results are as follows:

[0190] spk0: Controller;

[0191] spk1: Unit 1;

[0192] spk2: Unit 2;

[0193] spk3: Unit 3.

[0194] Implementation examples are as follows Figure 6 As shown.

[0195] The advantages of implementing the above plan are as follows:

[0196] 1. Improve the robustness of multi-speaker speech separation and recognition: Construct high-dimensional context embedding by using Mel spectrum and learnable network for feature encoding, effectively preserving the acoustic features of air-to-ground communication in aviation noise environment; Introduce multi-speaker mask modeling based on U-Net architecture, make full use of time-frequency image characteristics, realize adaptive modeling of complex acoustic environment of "high noise-low noise alternation", and generate high-fidelity, continuous time-frequency mask.

[0197] By combining a dual-path feature fusion mechanism with speaker prediction, speaker activation states are accurately modeled at the time frame level, effectively solving problems such as overlapping multiple speakers, frequent speaker switching, and incomplete speech segments. Compared with traditional methods based on energy thresholds or shallow clustering, it exhibits stronger stability and consistency in complex acoustic environments.

[0198] Meanwhile, this method does not rely on specific speakers or fixed scene distributions, has good cross-scene generalization ability, and can be promoted and applied in different air traffic control communication environments, other high-noise and multi-speaker scenarios, and has high engineering implementation value.

[0199] 2. Adapt to aviation communication context and improve recognition accuracy: By integrating air traffic control characteristics, the system specifically verifies key fields and command logic in the air-to-ground communication environment; and by using a constructed aviation-specific corpus, the speech recognition model has undergone full parameter fine-tuning of real air traffic control data on professional terminology, pronunciation habits, and channel characteristics in the air traffic control environment, thereby improving accuracy.

[0200] 3. Reduce computational overhead and meet the real-time requirements of high concurrency and low latency in air traffic control: By proposing a "segmentation multiplexing mechanism", embedding extraction and matching are performed only once for multiple speech segments of the same speaker, avoiding redundant calculations; compared with the segment-by-segment verification scheme, the throughput of speaker identification is significantly enhanced.

[0201] The speech recognition model adopts the lightweight SensenceVoiceSmall model and is fine-tuned with aviation corpus, effectively controlling inference latency and meeting the stringent real-time requirements of air-to-ground communication.

[0202] 4. Achieve efficient collaboration between each step, build an end-to-end trusted communication analysis link, share intermediate representations and joint optimization, and avoid information fragmentation; at the same time, the overall process does not require additional noise reduction preprocessing, retains the real channel noise as part of the identity features, and thus improves the speaker embedding ability in actual deployment.

[0203] Reference Figure 7 This invention also provides a voice understanding and authentication system for air-to-ground communication, the system comprising:

[0204] The feature encoding module is used to perform multi-level feature extraction on the raw speech of land-air communication to generate a high-dimensional mixed speech context embedding; wherein, the raw speech includes multi-speaker mixed speech;

[0205] The mask modeling module models the original speech based on a pre-constructed mask prediction network and generates corresponding multi-speaker time-frequency masks.

[0206] The feature fusion and clustering module performs speaker prediction based on the hybrid speech context embedding, the multi-speaker time-frequency mask, and a preset processing strategy to obtain a set of speech segments corresponding to each speaker.

[0207] The speech recognition module is used to perform high-precision text transcription of the speech segments corresponding to each speaker to obtain the recognized text; wherein, during transcription, the rule base with air traffic control characteristics is used for analysis and verification.

[0208] The identity verification module is used to combine the set of voice segments corresponding to each speaker with a pre-built registered voiceprint library and identity recognition model to perform identity verification and obtain the corresponding identity recognition result; wherein, during verification, a segmentation reuse mechanism is adopted to reduce computational overhead.

[0209] Furthermore, the mask prediction network based on a pre-constructed model models the original speech to generate corresponding multi-speaker time-frequency masks, specifically including:

[0210] The input raw speech is converted into a time spectrum using short-time Fourier transform;

[0211] The time-frequency spectrum is transmitted to the mask prediction network to obtain the corresponding multi-speaker time-frequency mask; wherein, the mask prediction network adopts the U-Net architecture, and the skip connections introduced in U-Net directly transmit the low-level fine-grained time-frequency structure of the encoding stage to the feature map of the corresponding scale in the decoding stage, thereby realizing the effective fusion of features at different scales.

[0212] The processing strategy for speaker prediction includes dual-path feature fusion, high-dimensional semantic enhancement, attention interaction modeling, feedforward network enhancement, and speaker prediction.

[0213] It should be noted that for a more detailed description of the workflow of the system implementation, please refer to the aforementioned method implementation section, which will not be repeated here.

[0214] The entire scheme employs a dual-path feature fusion mechanism: This step is a signal feature modeling mechanism for land-air communication, constructing a speech representation method that combines aviation noise characteristics. By introducing contextual semantic embedding and adaptive feature enhancement mechanisms, it improves speech expression capabilities in complex channel environments and significantly enhances speaker boundary detection accuracy under non-stationary aviation noise.

[0215] 2. Trainable mask-driven shared enhancement module: Synchronously supports ASR and SV tasks through unified speech enhancement output, avoiding redundant calculations and reducing system latency;

[0216] 3. Segmented and reusable authentication strategy: Based on the speaker segmentation results output by SD, identity matching is performed only once for each speech source, which greatly reduces the computational overhead of the SV module and improves overall efficiency;

[0217] 4. Over 70,000 pieces of real industry data were manually annotated for ASR fine-tuning.

[0218] Compared to existing solutions, this invention, through customized acoustic feature modeling and multi-task collaborative optimization architecture, significantly improves the accuracy of professional terminology recognition, speaker recognition and verification speed and precision in complex air traffic control and air-to-ground communication voice environments while ensuring efficient reasoning, providing key technical support for the intelligent evolution of high-density airspace autonomous control systems.

[0219] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can also be implemented in other ways. The method embodiments described above are merely illustrative. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in a different order than those marked in the accompanying drawings. In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. When each module is used, it collects and stores information only with the full authorization of the relevant user or organization and in compliance with relevant laws and regulations, and protects the security and privacy of the data, strictly prohibiting unauthorized access; the processing of data will be carried out within the scope prescribed by law and will not exceed the authorized purpose and scope; at the same time, the authorizing party has the right to access, correct, delete, restrict processing, refuse, etc. of its personal data; and strictly abides by applicable laws and regulations and conducts compliance review.

[0220] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0221] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A voice understanding and authentication method for air-to-ground communication, characterized in that, The method includes: Multi-level feature extraction is performed on the raw speech of land-air communication to generate a high-dimensional hybrid speech context embedding; wherein, the raw speech includes multi-speaker hybrid speech; The original speech is modeled based on a pre-constructed mask prediction network to generate corresponding multi-speaker time-frequency masks; Speaker prediction is performed based on the hybrid speech context embedding, the multi-speaker time-frequency mask, and the preset processing strategy to obtain a set of speech segments corresponding to each speaker; The speech segments corresponding to each speaker are transcribed with high precision to obtain the recognized text; wherein, during the transcription, the text is analyzed and verified using a rule base with air traffic control characteristics awareness. The voice segments corresponding to each speaker are combined with a pre-built registered voiceprint library and identity recognition model for identity verification to obtain the corresponding identity recognition result; wherein, during verification, a segmentation reuse mechanism is adopted to reduce computational overhead.

2. The voice understanding and authentication method for air-to-ground communication as described in claim 1, characterized in that, The specific steps for generating high-dimensional hybrid speech context embeddings include: The mixed speech of multiple speakers is converted into a complex spectrum using a short-time Fourier transform; A Mel filter bank is used to perform nonlinear frequency mapping on the amplitude spectrum to generate a Mel spectrum; wherein the amplitude spectrum is the modulus value of the complex spectrum; The Mel spectroscopic features contained in the Mel spectrogram are then expanded in dimension and enhanced in feature through a deployed learnable network, and finally the hybrid speech context embedding is output.

3. The voice understanding and authentication method for air-to-ground communication as described in claim 2, characterized in that, The pre-constructed mask prediction network models the original speech and generates corresponding multi-speaker time-frequency masks, specifically including: The input raw speech is converted into a time spectrum using short-time Fourier transform; The time-frequency spectrum is transmitted to the mask prediction network to obtain the corresponding multi-speaker time-frequency mask; wherein, the mask prediction network adopts the U-Net architecture, and the skip connections introduced in U-Net directly transmit the low-level fine-grained time-frequency structure of the encoding stage to the feature map of the corresponding scale in the decoding stage, thereby realizing the effective fusion of features at different scales.

4. The voice understanding and authentication method for air-to-ground communication as described in claim 3, characterized in that, During training, the mask prediction network is based on high-noise speech data collected in real air traffic control scenarios and mixed with clean speech recorded in low-noise environments according to random signal-to-noise ratios to construct training samples with real noise statistical characteristics. Meanwhile, the supervised learning strategy adopted by the network is to predict the ideal ratio mask of the corresponding speaker from the mixed speech signal, thereby learning the ability to accurately estimate the energy distribution of speech components in complex noise background.

5. A voice understanding and authentication method for air-to-ground communication as described in claim 4, characterized in that, The processing strategy for speaker prediction includes dual-path feature fusion, high-dimensional semantic enhancement, attention interaction modeling, feedforward network enhancement, and speaker prediction. The dual-path feature fusion is used to perform element-wise dot product between the hybrid speech context embedding and the multi-speaker time-frequency mask to obtain a joint representation of each speaker; wherein, the dimensions of the hybrid speech context embedding are [1, 400, 512], where 1 represents the batch size, 400 represents the number of time frames, and 512 represents the frequency dimension; the dimensions of the multi-speaker time-frequency mask are [num_spk, 400, 512], where num_spk represents the number of speakers, 400 represents the number of time frames, and 512 represents the frequency dimension; The high-dimensional semantic enhancement is used for: The dimensionality of the fused features is compressed and reduced by linear mapping to decrease computational complexity. Rotational position encoding is used to model the positional information of time series. The attention interaction modeling is used to perform information interaction in the "speaker × time" dimension using a multi-head attention mechanism, so that each speaker's representation can refer to other speakers and the global context; The feedforward network enhancement is used to improve the feature representation capability using a feedforward neural network, and to maintain training stability and keep the time dimension structure unchanged through residual connections. The speaker prediction method uses LSTM to capture past and future temporal context, compresses the dimension through a linear layer, obtains the activation probability of different speakers in each time frame through a Sigmoid activation layer, and clusters to obtain the set of speech segments corresponding to each speaker.

6. The voice understanding and authentication method for air-to-ground communication as described in claim 1, characterized in that, The high-precision text transcription is processed through a preset speech recognition process, specifically including: The original text is obtained by automatic recognition through a speech recognition model that has been fine-tuned in the domain beforehand; wherein, the speech recognition model is fine-tuned and trained using a constructed aviation-specific corpus to adapt to the aviation communication context; wherein, the speech recognition model adopts the lightweight SenceVoiceSmall model. Using regular expression templates and context rules, structured fields in aviation instructions are automatically identified and extracted from the original text, and all fields are checked for format validity. The recognition results are analyzed and verified for semantic integrity and logical rationality of instructions using a rule base with air traffic control characteristics.

7. The voice understanding and authentication method for air-to-ground communication as described in claim 1, characterized in that, The registered voiceprint database contains a mapping table of <user ID, registration embedding vector>, and the identity recognition model adopts a fine-tuned ECAPA-TDNN model; wherein, the ECAPA-TDNN model is fine-tuned in the training phase using air traffic control corpus to enhance its ability to model aviation terminology, communication style and channel characteristics. During registration, the ECAPA-TDNN model is used to process the registered speech, extract the high-dimensional speaker embedding vector, and use it as the registration embedding vector, denoted by ej. During verification, the following process is executed for each speaker's voice stream separated in real time: All speech segments belonging to the same speaker i are spliced ​​together to generate a representative speech signal Si; The speaker's voice Si is input into the finely tuned ECAPA-TDNN model, and the output is the embedding vector ei; Calculate the similarity between ei and all registered embedding vectors ej in the registered voiceprint database; if there exists an i such that the similarity is greater than the threshold, then determine that the current speaker is a registered user ID and perform a match; otherwise, consider it an unknown speaker.

8. A voice understanding and authentication system for land-air communication, characterized in that, The system includes: The feature encoding module is used to perform multi-level feature extraction on the raw speech of land-air communication to generate a high-dimensional mixed speech context embedding; wherein, the raw speech includes multi-speaker mixed speech; The mask modeling module models the original speech based on a pre-constructed mask prediction network and generates corresponding multi-speaker time-frequency masks. The feature fusion and clustering module performs speaker prediction based on the hybrid speech context embedding, the multi-speaker time-frequency mask, and a preset processing strategy to obtain a set of speech segments corresponding to each speaker. The speech recognition module is used to perform high-precision text transcription of the speech segments corresponding to each speaker to obtain the recognized text; wherein, during transcription, the rule base with air traffic control characteristics is used for analysis and verification. The identity verification module is used to combine the set of voice segments corresponding to each speaker with a pre-built registered voiceprint library and identity recognition model to perform identity verification and obtain the corresponding identity recognition result; wherein, during verification, a segmentation reuse mechanism is adopted to reduce computational overhead.

9. A voice understanding and authentication system for land-air communication as described in claim 8, characterized in that, The pre-constructed mask prediction network models the original speech and generates corresponding multi-speaker time-frequency masks, specifically including: The input raw speech is converted into a time spectrum using short-time Fourier transform; The time-frequency spectrum is transmitted to the mask prediction network to obtain the corresponding multi-speaker time-frequency mask; wherein, the mask prediction network adopts the U-Net architecture, and the skip connections introduced in U-Net directly transmit the low-level fine-grained time-frequency structure of the encoding stage to the feature map of the corresponding scale in the decoding stage, thereby realizing the effective fusion of features at different scales.

Images