An identity verification method and system based on a large language model

Abstract

The embodiment of the application discloses a kind of identity verification system and method based on large language model;System includes: streaming joint identification module layer, for receiving original air traffic voice stream in real time, and it is carried out streaming joint perception processing, obtains preliminary identification result;Domain knowledge base layer, for providing structured database of visible search;LLM semantic correction and reasoning module layer, for receiving preliminary identification result, combined with structured database provided by domain knowledge base layer to complete cognitive type check, output final identification result.The application has the advantages that: through the innovative streaming joint identification architecture, the inherent error propagation and high delay problem of traditional level connection system is fundamentally overcome;Break through the limitation that the prior art only stays at "perception level";The thinking chain reasoning ability of large language model is used, and the understanding depth of long-term context and complex interaction logic of the system is greatly enhanced.

Description

Technical Field

[0001] This invention relates to the fields of air traffic management, intelligent voice processing, and AI large model technology, specifically to an authentication method and system based on a large language model. Background Technology

[0002] In recent years, typical published patents related to speaker recognition or speaker classification in the field of civil aviation air traffic control include "A Pre-trained Multi-task Air Traffic Control Speech Recognition Method and Device" (Publication No.: CN115206293A) and "A Speaker Role Recognition Method and Device for Air-to-Ground Communication Based on Feature Fusion" (Publication No.: CN115240651A). These two solutions, in terms of air-to-ground communication speaker recognition technology, mainly suffer from the following four shortcomings at the implementation level:

[0003] First, cascaded systems suffer from error propagation and high processing latency. The CN115240651A employs a strict cascaded architecture: S1, noise reduction → S2, voice activity detection (VAD) → S3, automatic speech recognition (ASR) → S4, multimodal role recognition. This design has a double drawback. First, in a system composed of cascaded modules, errors are propagated cascaded. For example, any errors generated by the ASR module in step S3 (such as call signs, waypoints, typos, etc.) will be irreversibly passed to the multimodal role recognition module in step S4. This results in the latter receiving contaminated text input, significantly increasing the difficulty of determining the speaker's role and consequently reducing the overall system recognition accuracy. Second, the operations of these cascaded modules are serial, which may lead to high cumulative system latency, making it difficult to meet the real-time requirements of "online computing."

[0004] Second, in multi-task learning architectures, the relationships between tasks are implicit, and the system output relies excessively on acoustic features. CN115206293A employs a multi-task learning architecture where the ASR module and speaker recognition module share the same acoustic encoder. Although features from auxiliary tasks are designed to feed back into the main ASR task, these relationships are implicit (through feature sharing), and the entire system's judgment still heavily depends on acoustic features—leading to difficulties in distinguishing speaker roles based solely on acoustic features when key acoustic features are ambiguous (e.g., the pilot's repeated intonation is similar to an air traffic controller's instructions).

[0005] Third, the speaker recognition systems involved in the published patents all lack advanced semantic understanding and contextual reasoning capabilities. Whether it is the "modal attention mechanism" in CN115240651A or the Transformer encoder in CN115206293A, they are essentially performing feature extraction and fusion at the sentence-level (utterance-level). Therefore, the systems lack a deep understanding of extremely long temporal contexts (such as historical dialogues, previous turns, etc.) and structured domain knowledge (such as ICAO call rules and standard operating procedures (SOPs)).

[0006] Fourth, all the speaker recognition systems involved in the published patents lack cognitive verification and closed-loop correction steps. Both schemes are forward-looking—the model's output is the final result. They do not design a closed-loop "verification-correction" mechanism. "Cognitive verification" here refers to a verification process that transcends basic pattern recognition and is similar to advanced human reasoning. Specifically, traditional ASR and speaker role recognition models solve the perceptual problem of "what is heard"; while "cognitive verification" solves the cognitive problem of "whether the heard content is logical." A major common flaw in these two published patents is that they are both forward-looking "perceptual" systems—the system receives audio, uses complex statistical models (such as CNN, Transformer, attention mechanisms, etc.) to provide what it considers to be the answer with the highest acoustic probability, and then feeds this answer back to the user as the system's final output. The problem is that these systems lack a "reflection" step, that is, they lack a mechanism to examine the acoustically optimal solution to verify whether it is "reasonable" from the perspective of real-world logic, rules, and dialogue context. For example, an air traffic controller instructs a pilot to "...fly to waypoint DUMET," but due to accents, radio noise interference, etc., the receiver hears "DUMET" more like "DUMMY." The ASR models designed in these patents assume that, acoustically, "DUMMY" is more likely than "DUMET," and therefore consider the former to be the final result of ASR.

[0007] The cognitive verification and closed-loop correction steps can address this issue as follows: First, the cognitive verification step accesses a domain knowledge base (such as a knowledge graph) to obtain the necessary waypoint data. Then, the cognitive reasoning engine performs reasoning: "The 'DUMMY' output by the ASR module is a valid English word, which may be acoustically reasonable. However, after searching the waypoint list in the knowledge base, it is found that 'DUMMY' is not a registered named waypoint. It is very similar acoustically to the registered 'DUMET' in the knowledge base. Therefore, the output of the ASR module may be incorrect and should be corrected using the entity 'DUMET' from the knowledge base." This case demonstrates that existing patents belong to a closed-world model, meaning they do not understand the concept of "waypoint" itself because their knowledge only includes acoustic and textual patterns obtained from training data, lacking the "cognitive verification" capability to reference external structured knowledge for fact-checking. Summary of the Invention

[0008] To address the core shortcomings of existing technologies, such as the lack of advanced semantic reasoning and post-processing correction capabilities, the present invention aims to propose an authentication system and method based on a large language model (LLM) to solve the following technical problems:

[0009] To overcome the limitations of existing technologies involving preliminary instruction recognition modules (outputting ASR candidate text and speaker role candidate labels);

[0010] Eliminate error propagation and high latency issues at the architectural level;

[0011] To perform secondary correction on the initial output text of the ASR system;

[0012] The role labels provided by the upstream preliminary identification module are then corrected.

[0013] Establish a complete closed-loop system from acoustic perception to semantic correction;

[0014] Enable dynamic registration and continuous identity verification of voiceprint templates based on voice content.

[0015] To achieve the above objectives, in a first aspect, embodiments of the present invention provide an authentication system based on a large language model, comprising:

[0016] The streaming joint recognition module layer is used to receive the original air traffic control speech stream in real time and perform streaming joint perception processing on the original air traffic control speech stream to obtain a preliminary recognition result; the preliminary recognition result is an interleaved sequence including preliminary role labels and ASR text;

[0017] Domain knowledge base layer, a structured database used to provide visible retrieval;

[0018] The LLM semantic correction and reasoning module layer is used to receive the preliminary recognition results output by the streaming joint recognition module layer, combine the large language model and the structured database provided by the domain knowledge base layer to complete cognitive verification, and output the final recognition result; the final recognition result is the corrected role label and ASR text with higher confidence.

[0019] As one specific implementation of this application, the streaming joint identification module layer includes:

[0020] The SE front-end uses a lightweight convolutional architecture to receive raw air traffic control audio streams in real time and perform noise reduction processing on them.

[0021] The streaming encoder employs a block-based causal Conformer architecture to decompose, enhance, and extract contextual acoustic features from the denoised original air traffic control speech stream to obtain encoded data.

[0022] An E2E joint decoder is used to decode the encoded data in a streaming manner, and outputs an interleaved sequence including preliminary role tags and ASR text;

[0023] The voiceprint template registration management unit, in collaboration with the E2E joint decoder, is used to complete the creation of voiceprint templates and streaming voiceprint verification.

[0024] As one specific implementation of this application, the voiceprint template registration management unit is used for:

[0025] If a specific identity keyword is identified in the interleaved sequence, the speech segment containing the keyword is automatically extracted, and the speech segment is processed using a high-performance speaker embedding model to obtain a voiceprint embedding vector.

[0026] Obtain the logical identity associated with the keyword, bind the voiceprint embedding vector to the logical identity to form a voiceprint template, and store the voiceprint template in the voiceprint template buffer;

[0027] Based on the voiceprint template, streaming voiceprint verification is performed on subsequent speech data to obtain a high-confidence voiceprint verification signal, and the voiceprint verification information is passed to the LLM semantic correction and inference module layer; the subsequent speech data is the speech segment that is subsequently input and has been initially labeled with role tags by the E2E joint decoder.

[0028] As one specific implementation of this application, the structured database stores global waypoints, routes, airline call sign lists, ICAO standard terms and abbreviations, airport and sector information; the domain knowledge base layer includes:

[0029] The data collection unit is used to collect raw data, including official civil aviation standard documents, compilations of flight information, aviation databases, airline information and call signs.

[0030] The data processing unit is used to perform multimodal data extraction, knowledge graph construction, entity normalization and linking, and hybrid indexing and storage based on the raw data, so as to build a graph database and a vector database. The graph database is used to store entities and complex structured relationships to assist the LLM semantic correction and reasoning module layer in performing multi-hop logical reasoning. The vector database is used to store vector embeddings of all entities, terms and descriptive text.

[0031] The data processing unit is also used to build a retrieval API interface for the LLM semantic correction and inference module layer to call.

[0032] As one specific implementation of this application, the LLM semantic correction and reasoning module layer includes:

[0033] The context aggregation unit is used to receive the preliminary recognition results output by the streaming joint recognition module layer, and at the same time load the most recent N rounds of historical dialogue data cached by the system;

[0034] The knowledge retrieval and injection unit is used to retrieve a knowledge-enhanced list based on the preliminary identification results and the vector database.

[0035] The dynamic prompt construction unit is used to obtain the system preset instructions, and assemble the system preset instructions, preliminary recognition results, recent N rounds of historical dialogue data and the associations stored in the graph database into LLM prompt words according to a predefined template;

[0036] The LLM inference engine is used to perform inference correction based on the LLM prompts and output the final recognition result.

[0037] Secondly, embodiments of the present invention also provide an authentication method based on a large language model, applicable to the system described in the first aspect above, comprising:

[0038] A domain knowledge base layer is constructed; the domain knowledge base layer stores a graph database and a vector database;

[0039] Construct a voiceprint template;

[0040] Real-time reception of raw air traffic control voice stream;

[0041] The original air traffic control speech stream is processed by a streaming joint recognition module layer to obtain a preliminary recognition result; the preliminary recognition result is an interleaved sequence including preliminary role labels and ASR text;

[0042] Based on the voiceprint template and preliminary recognition results, streaming voiceprint verification is performed to obtain a voiceprint verification signal;

[0043] The LLM semantic correction and inference module layer is used to complete cognitive verification by combining the preliminary recognition results, voiceprint verification signals, graph database and vector database, and output the final recognition result; the final recognition result is the corrected role label and ASR text with higher confidence.

[0044] As a specific implementation of this application, the construction of the domain knowledge base layer is as follows:

[0045] Collect raw data; the raw data includes official civil aviation standard documents, compilations of flight information, aviation databases, airline information and call signs;

[0046] Based on the original data, multimodal data extraction, knowledge graph construction, entity normalization and linking, and hybrid indexing and storage are implemented to build a graph database and a vector database. The graph database is used to store entities and complex structured relationships to assist the LLM semantic correction and reasoning module layer in performing multi-hop logical reasoning. The vector database is used to store vector embeddings of all entities, terms, and descriptive text.

[0047] Build a retrieval API interface for the LLM semantic correction and inference module layer to call.

[0048] As one specific implementation of this application, the construction of the voiceprint template is as follows:

[0049] Acquire the empty tube data stream of the sample;

[0050] The original air traffic control speech stream is processed by a streaming joint recognition module layer to obtain sample recognition results; the sample recognition results are an interleaved sequence including preliminary role labels and ASR text.

[0051] If a specific identity keyword is identified in the interleaved sequence, the speech segment containing the keyword is automatically extracted, and the speech segment is processed using a high-performance speaker embedding model to obtain a voiceprint embedding vector.

[0052] Obtain the logical identity associated with the keyword, bind the voiceprint embedding vector to the logical identity to form a voiceprint template, and store the voiceprint template in the voiceprint template buffer.

[0053] As a specific implementation of this application, the streaming joint identification module layer includes an SE front-end, a streaming encoder, and an E2E joint decoder; the preliminary identification result is as follows:

[0054] The SE front-end is used to receive the raw air traffic control voice stream in real time and perform noise reduction processing on it;

[0055] The streaming encoder is used to decompose, enhance, and extract contextual acoustic features from the denoised original air traffic control speech stream to obtain encoded data.

[0056] The E2E joint decoder is used to decode the encoded data in a streaming manner, and a preliminary recognition result is output.

[0057] As a specific implementation of this application, the final recognition result is output as follows:

[0058] Receive the preliminary recognition result output by the streaming joint recognition module layer, and load the most recent N rounds of historical dialogue data cached by the system;

[0059] Based on the preliminary identification results and the vector database, a knowledge-enhanced list is obtained.

[0060] Obtain system preset instructions, and assemble the system preset instructions, preliminary recognition results, recent N rounds of historical dialogue data, and the relationships stored in the graph database into LLM prompt words according to a predefined template;

[0061] Based on the LLM prompts, inference correction is performed, and the final recognition result is output.

[0062] Implementing the embodiments of the present invention has significant advantages in terms of system architecture robustness, semantic cognition depth, and result correction reliability, as detailed below:

[0063] 1. This invention, through its innovative streaming joint recognition architecture, fundamentally overcomes the inherent error propagation and high latency problems of traditional cascaded systems. Existing technologies (such as CN115240651A) generally employ a serial pipeline of "noise reduction-VAD-ASR-role recognition". This pipeline architecture not only has high computational latency, but also irreversibly amplifies small decision deviations in preceding modules. In contrast, this invention introduces a Conformer-based streaming end-to-end joint model at the streaming joint recognition module layer, abandoning independent VAD and role classification modules, and directly modeling the joint probability of speech to "role + text" interleaved sequences. This strongly coupled design allows the model to utilize both acoustic features and linguistic cues during the decoding process, achieving millisecond-level streaming response and effectively avoiding the "domino effect" of downstream role recognition failure due to ASR transcription errors, significantly improving system robustness in complex air traffic control noise environments.

[0064] 2. This invention overcomes the limitations of existing technologies that only operate at the "perception level," introducing for the first time a closed-loop correction mechanism with "cognitive verification" capabilities into air traffic control speech recognition. Addressing the shortcomings of existing technologies, such as lack of advanced semantic understanding and inability to perform fact-checking, this embodiment constructs a collaborative architecture between a domain knowledge base and an LLM semantic reasoning layer. By building global waypoints, airline call signs, and ICAO standard terminology into a structured knowledge graph and vector index, the system can compare the preliminary results output by the acoustic model with authoritative domain knowledge in real time. This Retrieval Enhanced Generation (RAG) mechanism enables the system to identify and correct acoustically similar but logically flawed errors (such as mishearing "DUMET" as "DUMMY"), thus achieving a qualitative leap from "sound recognition" to "understanding and error correction."

[0065] 3. This embodiment of the invention utilizes the thought chain (CoT) reasoning capability of the large language model, greatly enhancing the system's understanding of long-term context and complex interaction logic. Unlike existing technologies that only perform feature fusion at the sentence level, the LLM semantic reasoning layer in this embodiment can aggregate historical dialogue context and leverage the powerful intent inference capability of LLM to perform secondary correction on role labels based on the logical relationship between instructions and repetitions (e.g., after an air traffic controller issues an instruction, the next sentence is likely to be repeated by the pilot). This multi-dimensional verification system, which combines acoustic features, dialogue logic, and domain knowledge, ensures that the system maintains extremely high recognition accuracy and business compliance even when facing non-standard calls, accent interference, and high-density communication scenarios.

[0066] Furthermore, this embodiment innovatively introduces a dynamic registration and verification mechanism for voiceprint templates based on voice content, achieving a deep integration of biometric recognition and semantic recognition. Compared to the shortcomings of existing voiceprint recognition systems that require independent registration and have static, rigid templates, this solution seamlessly embeds voiceprint registration into the normal work process, achieving a non-intrusive experience. Simultaneously, by combining the strong biometric feature of voiceprint with LLM semantic reasoning, a two-factor authentication system of "known content (flight number) + all voiceprints" is constructed, effectively identifying and preventing security threats such as identity theft. This represents a higher level of security capability not addressed in existing patented technologies. Attached Figure Description

[0067] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0068] Figure 1 This is a flowchart illustrating the overall architecture of the identity verification system based on a large language model provided in this embodiment of the invention.

[0069] Figure 2 yes Figure 1 The flowchart shown is the implementation flow of the streaming joint identification module layer.

[0070] Figure 3 This is a flowchart of the training process for the streaming joint recognition module layer;

[0071] Figure 4 This is a flowchart of the overall architecture of the domain knowledge base layer;

[0072] Figure 5 This is a flowchart of the overall implementation of the LLM semantic correction and inference module layer;

[0073] Figure 6 This is a flowchart of the authentication method based on a large language model provided in an embodiment of the present invention. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0076] The inventive concept of this invention is to address the core deficiency of existing technologies in lacking advanced semantic reasoning and post-processing correction capabilities by proposing a land-air dialogue recognition, correction, and reasoning scheme based on a large language model. The core innovation lies in the introduction of a "semantic correction and reasoning module" downstream of the traditional recognition model. This module is a large language model that has been fine-tuned with knowledge from the air traffic control command domain.

[0077] Meanwhile, in order to overcome the limitations of the preliminary instruction recognition module (outputting ASR candidate text and speaker role candidate labels) involved in existing patents, and to further improve the overall system performance, this embodiment also includes the following innovations:

[0078] 1. Design a streaming, end-to-end, multi-output preliminary recognition model that jointly performs speech enhancement (SE), ASR, and role recognition to eliminate error propagation and high latency issues at the architectural level.

[0079] 2. Leveraging the powerful language prior knowledge and understanding of structured domain knowledge (such as waypoints, call signs, secondary codes, etc.) of LLM, secondary correction is performed on the initial output text of the ASR system.

[0080] 3. Utilize the powerful contextual reasoning capabilities and role language pattern knowledge of LLM (e.g., the difference in language structure patterns between imperative and repetitive statements) to perform secondary correction on the role labels given by the upstream preliminary identification module.

[0081] 4. Establish a complete closed-loop system from acoustic perception to semantic correction.

[0082] 5. Achieving dynamic registration and continuous identity verification of voiceprint templates based on speech content. To address the issues of existing technologies where voiceprint recognition relies on independent registration processes and static templates, this patent further proposes utilizing inherent identity information (such as flight numbers) in air-to-ground communication to dynamically associate logical identities during the speech content recognition stage, and simultaneously capturing and updating voiceprint templates based on the speech segment. Subsequently, through real-time voiceprint matching, continuous and seamless verification of the speaker's identity is achieved, forming a two-factor authentication closed loop of "content recognition confirming logical identity, and voiceprint verification confirming biometric identity," further enhancing the system's security in areas such as identity theft prevention.

[0083] Please refer to Figure 1 This is an authentication system based on a large language model provided in an embodiment of the present invention, comprising:

[0084] The streaming joint recognition module layer is used to receive the original air traffic control speech stream in real time and perform streaming joint perception processing on the original air traffic control speech stream to obtain a preliminary recognition result; the preliminary recognition result is an interleaved sequence including preliminary role labels and ASR text;

[0085] Domain knowledge base layer, a structured database used to provide visible retrieval;

[0086] The LLM semantic correction and reasoning module layer is used to receive the preliminary recognition results output by the streaming joint recognition module layer, combine the large language model and the structured database provided by the domain knowledge base layer to complete cognitive verification, and output the final recognition result; the final recognition result is the corrected role label and ASR text with higher confidence.

[0087] I. Streaming Joint Identification Module Layer (L1)

[0088] This architecture layer is responsible for processing the raw speech stream with low signal-to-noise ratio, with the aim of low latency and simultaneous output of preliminary ASR text and speaker role labels.

[0089] This embodiment innovatively proposes a streaming joint speech enhancement and multi-output Conformer architecture. This architecture can completely replace the cascaded scheme of CN115240651A and the parallel multi-head scheme of CN115206293A. In this embodiment, the streaming joint recognition module layer includes:

[0090] The SE front-end uses a lightweight convolutional architecture to receive raw air traffic control audio streams in real time and perform noise reduction processing on them.

[0091] The streaming encoder employs a block-based causal Conformer architecture to decompose, enhance, and extract contextual acoustic features from the denoised original air traffic control speech stream to obtain encoded data.

[0092] An E2E joint decoder is used to decode the encoded data in a streaming manner, and outputs an interleaved sequence including preliminary role tags and ASR text;

[0093] The voiceprint template registration management unit, in collaboration with the E2E joint decoder, is used to complete the creation of voiceprint templates and streaming voiceprint verification.

[0094] More specifically, the architecture diagram for this layer can be found in [link to diagram]. Figure 2 The specific internal implementation scheme is as follows:

[0095] 1. A joint speech enhancement (SE) front-end (L1.1) serves as the model's input layer. It employs a lightweight convolutional U-Net architecture (such as Conv-TasNet or DPT-Net) and is not treated as a standalone processing step, but rather as part of the model, jointly trained with downstream tasks. This ensures that the noise knowledge learned by the model is specific to the empty-channel dialogue scenario corresponding to the training data, aiming to specifically utilize it to improve ASR and role recognition accuracy.

[0096] 2. The streaming Conformer encoder (L1.2) is the core encoder of this architecture layer. It adopts a causal Conformer architecture based on chunk-based processing. This architecture (such as E-Branchformer) can achieve low-latency streaming processing by limiting the receptive field of self-attention to a limited future context, thereby meeting the high real-time business requirements of air traffic control scenarios.

[0097] 3. An end-to-end (E2E) joint decoder (L1.3) serves as the output layer of this architecture. It employs a unified output objective (e.g., using an RNN-T transducer or a CTC / Attention joint decoder). This decoder is trained to directly output an interleaved sequence containing role labels and ASR text, such as "<role_ATC> X-plane YYYY at altitude 8500<role_PILOT> "At an altitude of 8500, X-axis YYYY". The advantage of this E2E scheme is that it enables strong coupling between ASR and role recognition tasks at the decision level, and the model learns the joint probability of the two, fundamentally solving the error propagation problem of CN115240651A and the implicit association problem of CN115206293A.

[0098] 4. The voiceprint template registration and management unit (L1.4) is an enhancement to the identity authentication capability of this invention. As a collaborative module with the L1.3 decoder, it is responsible for dynamically managing voiceprint templates based on speech content. Specifically, this voiceprint template registration and management unit performs the following steps:

[0099] (1) Content-triggered template creation: When a specific identification keyword (e.g., the flight number "CES1234" reported by the pilot, or the controller's seat number "BeijingApproach") is identified in the interleaved sequence output by the L1.3 decoder, the unit will be triggered. The system automatically extracts the speech segment containing the keyword (usually a complete sentence within a certain time window before and after the keyword) and uses a high-performance speaker embedding model (such as ECAPA-TDNN) to extract the speaker embedding vector for the speech segment.

[0100] (2) Logical identity-voiceprint template binding: The extracted voiceprint embedding vector is bound to the logical identity associated with keywords (such as "CES1234 flight captain" or "Beijing approach controller") and stored in a dedicated voiceprint template buffer as a dynamically created voiceprint template.

[0101] (3) Streaming voiceprint verification: For subsequent incoming speech segments that have been initially labeled with role tags by L1.3, this unit will extract their voiceprint features in real time and calculate the similarity with all active voiceprint templates in the buffer. If the similarity with a certain template exceeds a preset threshold, a high-confidence voiceprint verification signal will be generated and the signal will be passed to the L3 layer.

[0102] Furthermore, the model training scheme for the L1 layer design involves the following two stages, with step-by-step illustrations shown below. Figure 3 :

[0103] The first stage involves noise robustness pre-training. This step uses large-scale unlabeled speech data (such as LibriSpeech, GigaSpeech, etc. databases) and real air traffic control noise data (such as cockpit background noise, radio howling, etc.) to dynamically mix them, and then pre-trains the model based on dual loss objectives: (1) speech reconstruction loss of joint speech enhancement front-end (such as SI-SNR loss); (2) self-supervised loss of Conformer encoder.

[0104] The second stage involves end-to-end (E2E) joint fine-tuning. Using labeled ATC data (such as the labeled data mentioned in CN115206293A), the entire model (SE front-end, encoder, decoder) is fine-tuned end-to-end according to the interleaved sequence objective designed in L1.3.

[0105] II. Domain Knowledge Base Layer (L2)

[0106] The knowledge base designed in the current architecture layer will serve as an "external" knowledge source for the LLM in the next architecture layer (L3), providing factual basis for it to perform inference steps (such as syntactic structure analysis, semantic understanding, ASR text correction, etc.).

[0107] This architectural layer is essentially a searchable structured database storing content including, but not limited to: global waypoints, routes, airline call sign lists, ICAO standard terminology and abbreviations, airport and sector information, etc. Its aim is to transform dispersed, multimodal aviation domain expertise into searchable and understandable structured data within the LLM semantic correction and reasoning module layer (L3). Understandably, this domain knowledge base layer includes:

[0108] The data collection unit is used to collect raw data, including official civil aviation standard documents, compilations of flight information, aviation databases, airline information and call signs.

[0109] The data processing unit is used to perform multimodal data extraction, knowledge graph construction, entity normalization and linking, and hybrid indexing and storage based on the raw data, so as to build a graph database and a vector database. The graph database is used to store entities and complex structured relationships to assist the LLM semantic correction and reasoning module layer in performing multi-hop logical reasoning. The vector database is used to store vector embeddings of all entities, terms and descriptive text.

[0110] The data processing unit is also used to build a retrieval API interface for the LLM semantic correction and inference module layer to call.

[0111] See the overall implementation flowchart of this architecture layer. Figure 4 Specifically, it includes:

[0112] (1) Corpus and Data Collection. The original data sources for the knowledge base need to be highly authoritative and professional, so they are mainly collected through the following channels:

[0113] ① Official standard documents: Crawl and parse official publications, annexes and manuals issued by the International Civil Aviation Organization (ICAO), especially the sections on communication procedures and standard terminology.

[0114] ② Aeronautical Information Programs (AIPs): Periodically obtain AIPs published by the civil aviation authorities of various countries (especially those in the target airspace). This is the most reliable source for obtaining waypoint, route, airport information, sector allocation, and VHF frequency allocation.

[0115] ③ Structured aviation data: Access or import professional aviation databases (such as ARINC 424 standard format data, Jeppesen database) to obtain accurate, machine-readable navigation data.

[0116] ④ Airline and Call Sign: Collect airline codes, aircraft type codes, and corresponding radio call signs published by IATA and ICAO.

[0117] (2) Transform the collected raw data into an "external knowledge source" that can be retrieved by LLM. This step involves the following key technical processes:

[0118] ① Multimodal data extraction. This step aims to establish a robust data extraction pipeline: for massive PDF documents such as AIPs, a dedicated model combining OCR and layout analysis is used to extract multimodal content such as tables, maps, and text; for structured data (such as ARINC 424 standard format data), a dedicated parser can be written for field extraction.

[0119] ② Knowledge Graph Construction. Using a knowledge graph instead of a simple key-value database to store knowledge ensures that the next level (L3) LLM can not only "find" entities, but also "understand" the relationships between entities. The first step in constructing a knowledge graph is to design its ontology (schema), that is, to define core entities (such as waypoints, airports, routes, etc.) and relationships (such as "waypoint A is in sector B", "C is the call sign of D", "E is the abbreviation of F", etc.). The second step is to use natural language processing (NLP) technology to automatically extract triples (subject-verb-object) from unstructured text (such as ICAO documents) and populate them into the knowledge graph.

[0120] ③ Entity Normalization and Linking. This step aims to solve the problems of "synonymous but different names" and "same name but different meanings" among knowledge graph entity objects by establishing an "authoritative entity database." For example, linking "Beijing Capital International Airport," "ZBAA," and "PEK" to the same normalized entity ensures that when the LLM performs knowledge retrieval, it can hit the same airport entity regardless of which alias is initially identified by the ASR at the L1 layer.

[0121] ④ Implementation of Hybrid Indexing and Storage. This step aims to build a hybrid storage scheme combining graph databases (Graph DB) and vector databases (Vector DB) to support both precise and fuzzy retrieval. Graph databases (such as Neo4j) are used to store entities and complex structured relationships, assisting L3 layer LLMs in multi-hop logical reasoning (e.g., "Waypoint A belongs to sector B, sector B is under the jurisdiction of center C"). Vector databases (such as FAISS) are used to store vector embeddings of all entities, terms, and descriptive text—this is the core of implementing RAG (Retrieval Enhancement Generation).

[0122] ⑤ Construction of the Retrieval Enhancement Generation Interface Layer. This step aims to build a unified retrieval API interface for L3 layer LLM to call. Specifically, after the L3 layer module receives the preliminary ASR identification results output by the L1 layer, this interface layer automatically performs the following operations: 1) Extract key entities (such as waypoints, call signs, etc.) from the L1 layer output results; 2) Simultaneously query the graph database and vector database; 3) Format the most relevant knowledge fragments retrieved and automatically inject them into the prompt words of the L3 layer LLM as the factual basis for its correction.

[0123] III. LLM Semantic Correction and Inference Module Layer (L3)

[0124] This architecture layer serves as the "cognitive center" of the system. Its core task is to receive the acoustic perception results output from the L1 layer, combine them with the domain knowledge and dialogue context provided by the L2 layer, and use the reasoning ability of the large language model to perform "cognitive verification" in order to output the final high-confidence recognition result.

[0125] This embodiment innovatively employs a technical architecture combining Retrieval Enhanced Generation (RAG) and Chain-of-Thought (CoT) at this layer. In this embodiment, the LLM semantic correction and reasoning module layer includes:

[0126] The context aggregation unit is used to receive the preliminary recognition results output by the streaming joint recognition module layer, and at the same time load the most recent N rounds of historical dialogue data cached by the system;

[0127] The knowledge retrieval and injection unit is used to retrieve a knowledge-enhanced list based on the preliminary identification results and the vector database.

[0128] The dynamic prompt construction unit is used to obtain the system preset instructions, and assemble the system preset instructions, preliminary recognition results, recent N rounds of historical dialogue data and the associations stored in the graph database into LLM prompt words according to a predefined template;

[0129] The LLM inference engine is used to perform inference correction based on the LLM prompts and output the final recognition result.

[0130] More specifically, the architecture diagram for this layer can be found in [link to diagram]. Figure 5 The specific internal implementation scheme is as follows:

[0131] (1) Context aggregation unit, which is responsible for maintaining the short-term memory of the system. It receives the "preliminary recognition result package" output by the L1 layer (containing ASR candidate text, role labels and confidence scores), and loads the most recent N rounds of historical dialogues cached by the system (including the previous speaker roles and instruction content), aiming to provide necessary contextual information for subsequent reasoning.

[0132] (2) Knowledge Retrieval and Injection Unit: This unit serves as a bridge connecting the L3 and L2 architecture layers. It calculates the vector embeddings of key entities (such as suspected waypoints, frequencies, call signs, etc.) in the L1 layer output text and performs an approximate nearest neighbor search (ANN) in the vector database of the L2 layer. If the retrieved entity is acoustically similar to the L1 recognition result but semantically different (e.g., L1 identifies it as "DUMMY", while L2 retrieves it as "DUMET"), this unit will extract the retrieved "knowledge enhancement list" together with the association relationships (such as sector affiliation relationships) extracted from the L2 graph database to prepare for the injection of prompt words.

[0133] (3) Dynamic prompting construction unit, which is responsible for converting structured data into natural language instructions that LLM can understand. It assembles the system's preset instructions, the dialogue history provided by the context aggregation unit, the preliminary recognition results of the L1 layer, and the domain factual knowledge provided by the knowledge retrieval unit into complete LLM prompt words according to a predefined template.

[0134] (4) The LLM reasoning and correction engine, which consists of a large language model fine-tuned in the air traffic control domain. It receives dynamic cues and integrates voiceprint verification signals from the L1.4 unit, and executes the following logical reasoning steps through the thought chain (CoT) mechanism:

[0135] ① Role and Identity Consistency Verification: This involves comprehensively evaluating the role label output by L1, the dialogue turn logic, and the voiceprint verification results. For example, if a sentence is initially identified by L1 as coming from a "pilot" and contains the instruction "descend altitude," this does not match the role's behavioral pattern; simultaneously, the voiceprint of this speech successfully matches the "air traffic controller" template in the buffer. In this case, LLM will integrate these contradictory pieces of information, prioritizing the voiceprint verification results and semantic logic to correct the role label.

[0136] ② Entity consistency verification: This involves comparing the ASR text retrieved in L1 with the knowledge facts retrieved in L2, and prioritizing the adoption of valid entities existing in the knowledge base to correct acoustic errors.

[0137] ③ Intent compliance verification: This involves analyzing whether the instruction intent in the text conforms to the standard call terminology specifications.

[0138] Ultimately, the engine integrates acoustic perception (L1), domain knowledge (L2), and biometrics (voiceprint) to output corrected ASR text and role tags with higher confidence. It can also output successfully verified identity information for high-level security audits.

[0139] As can be seen from the above description, implementing the embodiments of the present invention has significant advantages in terms of system architecture robustness, semantic cognition depth, and result correction reliability, as detailed below:

[0140] 1. This invention, through its innovative streaming joint recognition architecture, fundamentally overcomes the inherent error propagation and high latency problems of traditional cascaded systems. Existing technologies (such as CN115240651A) generally employ a serial pipeline of "noise reduction-VAD-ASR-role recognition". This pipeline architecture not only has high computational latency, but also irreversibly amplifies small decision deviations in preceding modules. In contrast, this invention introduces a Conformer-based streaming end-to-end joint model at the streaming joint recognition module layer, abandoning independent VAD and role classification modules, and directly modeling the joint probability of speech to "role + text" interleaved sequences. This strongly coupled design allows the model to utilize both acoustic features and linguistic cues during the decoding process, achieving millisecond-level streaming response and effectively avoiding the "domino effect" of downstream role recognition failure due to ASR transcription errors, significantly improving system robustness in complex air traffic control noise environments.

[0141] 2. This invention overcomes the limitations of existing technologies that only operate at the "perception level," introducing for the first time a closed-loop correction mechanism with "cognitive verification" capabilities into air traffic control speech recognition. Addressing the shortcomings of existing patents, which lack advanced semantic understanding and are unable to perform fact-checking, this embodiment constructs a collaborative architecture between a domain knowledge base and an LLM semantic reasoning layer. By building global waypoints, airline call signs, and ICAO standard terminology into a structured knowledge graph and vector index, the system can compare the preliminary results output by the acoustic model with authoritative domain knowledge in real time. This Retrieval Enhanced Generation (RAG) mechanism enables the system to identify and correct acoustically similar but logically flawed errors (such as mishearing "DUMET" as "DUMMY"), thus achieving a qualitative leap from "sound recognition" to "understanding and error correction."

[0142] 3. This embodiment of the invention utilizes the thought chain (CoT) reasoning capability of the large language model, greatly enhancing the system's understanding of long-term context and complex interaction logic. Unlike existing technologies that only perform feature fusion at the sentence level, the LLM semantic reasoning layer in this embodiment can aggregate historical dialogue context and leverage the powerful intent inference capability of LLM to perform secondary correction on role labels based on the logical relationship between instructions and repetitions (e.g., after an air traffic controller issues an instruction, the next sentence is likely to be repeated by the pilot). This multi-dimensional verification system, which combines acoustic features, dialogue logic, and domain knowledge, ensures that the system maintains extremely high recognition accuracy and business compliance even when facing non-standard calls, accent interference, and high-density communication scenarios.

[0143] Furthermore, this embodiment innovatively introduces a dynamic registration and verification mechanism for voiceprint templates based on voice content, achieving a deep integration of biometric recognition and semantic recognition. Compared to the shortcomings of existing voiceprint recognition systems that require independent registration and have static, rigid templates, this solution seamlessly embeds voiceprint registration into the normal work process, achieving a non-intrusive experience. Simultaneously, by combining the strong biometric feature of voiceprint with LLM semantic reasoning, a two-factor authentication system of "known content (flight number) + all voiceprints" is constructed, effectively identifying and preventing security threats such as identity theft. This represents a higher level of security capability not addressed in existing patented technologies.

[0144] Based on the same inventive concept, embodiments of the present invention also provide an authentication method based on a large language model, applicable to the system described in the first aspect above. For example... Figure 6 As shown, the method may include the following steps:

[0145] S1, construct the domain knowledge base layer and voiceprint template.

[0146] In this embodiment, constructing the domain knowledge base layer specifically includes:

[0147] Collect raw data; the raw data includes official civil aviation standard documents, compilations of flight information, aviation databases, airline information and call signs;

[0148] Based on the original data, multimodal data extraction, knowledge graph construction, entity normalization and linking, and hybrid indexing and storage are implemented to build a graph database and a vector database. The graph database is used to store entities and complex structured relationships to assist the LLM semantic correction and reasoning module layer in performing multi-hop logical reasoning. The vector database is used to store vector embeddings of all entities, terms, and descriptive text.

[0149] Build a retrieval API interface for the LLM semantic correction and inference module layer to call.

[0150] Furthermore, the construction of the voiceprint template specifically involves:

[0151] Acquire the empty tube data stream of the sample;

[0152] The original air traffic control speech stream is processed by a streaming joint recognition module layer to obtain sample recognition results; the sample recognition results are an interleaved sequence including preliminary role labels and ASR text.

[0153] If a specific identity keyword is identified in the interleaved sequence, the speech segment containing the keyword is automatically extracted, and the speech segment is processed using a high-performance speaker embedding model to obtain a voiceprint embedding vector.

[0154] Obtain the logical identity associated with the keyword, bind the voiceprint embedding vector to the logical identity to form a voiceprint template, and store the voiceprint template in the voiceprint template buffer.

[0155] S2 receives raw air traffic control voice streams in real time.

[0156] S3, the original air traffic control speech stream is processed by the streaming joint recognition module layer to obtain preliminary recognition results.

[0157] In this embodiment, the streaming joint identification module layer includes an SE front-end, a streaming encoder, and an E2E joint decoder; the preliminary identification result is as follows:

[0158] The SE front-end is used to receive the raw air traffic control voice stream in real time and perform noise reduction processing on it;

[0159] The streaming encoder is used to decompose, enhance, and extract contextual acoustic features from the denoised original air traffic control speech stream to obtain encoded data.

[0160] The E2E joint decoder is used to decode the encoded data in a streaming manner, and the preliminary recognition results are output, including the interleaved sequence of preliminary role labels and ASR text.

[0161] S4. Based on the voiceprint template and preliminary recognition results, perform streaming voiceprint verification to obtain the voiceprint verification signal.

[0162] In practice, the real-time air conditioning voice stream obtained during this correction process is subjected to streaming joint recognition to obtain voice segments labeled with role tags. After extracting their voiceprint features, they are matched with the voiceprint template in the buffer to obtain a high-confidence voiceprint verification signal.

[0163] S5 employs an LLM semantic correction and reasoning module layer, combining the preliminary recognition results, voiceprint verification signals, graph database, and vector database to complete cognitive verification and output the final recognition result.

[0164] In practice, step S5 includes:

[0165] Receive the preliminary recognition result output by the streaming joint recognition module layer, and load the most recent N rounds of historical dialogue data cached by the system;

[0166] Based on the preliminary identification results and the vector database, a knowledge-enhanced list is obtained.

[0167] Obtain system preset instructions, and assemble the system preset instructions, preliminary recognition results, recent N rounds of historical dialogue data, and the relationships stored in the graph database into LLM prompt words according to a predefined template;

[0168] Based on the LLM prompts, inference correction is performed, and the final recognition result is output.

[0169] It should be noted that for a more detailed description of the process of this method embodiment, please refer to the aforementioned system embodiment section, which will not be repeated here.

[0170] To better understand the system and method provided in the embodiments of the present invention, the following uses a real ground-to-air communication scenario as an example to fully describe the entire process of the system of the present invention from the original voice input to the final result output.

[0171] Scenario setting: Beijing Area Control Center, the controller (seat "Beijing Area 01") is communicating with China Eastern Airlines flight CES1234.

[0172] Raw voice input (approximately 6 seconds):

[0173] Air traffic controller's voice: "CES1234, direct flight to DUMET." (Due to VHF radio noise interference, the "T" at the end of "DUMET" is attenuated during transmission, making it sound similar to "DUMMY" on the receiver.)

[0174] 1. Step S1: Streaming Joint Sensing (L1 Layer)

[0175] The L1 layer receives the speech segment in real time (sampling rate 16kHz, mono). The joint speech enhancement front-end first processes the noisy speech: the Conv-TasNet encoder decomposes the speech into 256-dimensional time-frequency features, the TCN separation module separates the clean speech features, and the decoder reconstructs the enhanced speech signal. The enhanced speech is then fed into a streaming Conformer encoder (12-layer E-Branchformer, block size 40 frames) to extract contextual acoustic features block by block. The RNN-T joint decoder decodes token by token in a streaming manner, outputting preliminary recognition results.

[0176] <role_ATC> CES1234 direct flight to DUMMY

[0177] The accompanying output confidence scores are: "Character" confidence score 0.92, "CES1234" confidence score 0.95, "Direct Flight" confidence score 0.90, and "DUMMY" confidence score 0.68.

[0178] 2. Step S2: Dynamic registration of voiceprint template (L1.4 unit)

[0179] The L1.3 decoder detects the flight number "CES1234" in the interleaved sequence output, triggering the L1.4 unit. The system extracts the speech from the audio buffer (approximately 4.5 seconds, from 0.5 seconds before the start of "CES1234" to 0.5 seconds after the end of the sentence), inputs it into the ECAPA-TDNN model to extract a 1920-dimensional voiceprint embedding vector, binds it to the logical identity "CES1234 crew", and stores it in the voiceprint template buffer with a TTL of 30 minutes.

[0180] At the same time, the system detects the voice of the current speaker (controller) and matches it with the existing voiceprint template of "Beijing Area 01 Controller" in the buffer. The similarity reaches 91%, and a voiceprint verification signal is generated: "Current speaker matches 'Beijing Area 01 Controller', confidence level 0.91".

[0181] 3. Step S3: Knowledge retrieval and voiceprint signal enhancement (L2 layer + L3 layer retrieval unit)

[0182] The knowledge retrieval unit at layer L3 extracts the key entity "DUMMY" from the output of layer L1 and performs a retrieval in the knowledge base at layer L2:

[0183] Neo4j exact query for "DUMMY" returns null.

[0184] FAISS vector search for “DUMMY” returns the top 3 results: “DUMET” (similarity 0.94), “DUMEY” (similarity 0.79), and “DUMBA” (similarity 0.71).

[0185] Neo4j further queries the attributes of "DUMET": coordinates N30°45′E120°30′, located in "Beijing Area 01 Sector", which is consistent with the current controlled sector.

[0186] The knowledge retrieval unit formats the results into a knowledge-enhanced list:

[0187] Knowledge base hint: 'DUMMY' is not a registered waypoint. Acoustic similar entities: 'DUMET' (similarity 0.94, located in Beijing region sector 01, valid waypoint), 'DUMEY' (similarity 0.79, located in Shanghai region).

[0188] Meanwhile, the voiceprint verification signal ("Current speaker matches 'Beijing Area 01 Controller', confidence level 0.91") is transmitted to Layer 3.

[0189] 4. Step S4: Cognitive Reasoning and Multimodal Correction (L3 Layer)

[0190] The L3 layer's context aggregation unit provides the dialogue history (the previous round was CES1234 crew requesting a direct flight), the knowledge retrieval unit provides a knowledge enhancement list, and the L1.4 unit provides the voiceprint verification signal. The dynamic prompting construction unit assembles all information into LLM prompt words.

[0191] The LLM inference engine (Qwen2.5-7B-Instruct, fine-tuned with LoRA) receives the prompt words and then performs thought chain inference:

[0192] Analyzing the ASR output: The role is air traffic controller, and the text is "CES1234 direct flight to DUMMY". The role and dialogue logic are consistent (the previous round was a request from the crew, and this round is a response from the controller), so no correction is needed.

[0193] Checking entity "DUMMY": This waypoint does not exist in the knowledge base. The acoustically similar entity "DUMET" exists in the knowledge base and is located in the current controlled sector (Sector 01, Beijing Area), which matches the context.

[0194] Voiceprint verification: The current speaker matches the "Beijing Area 01 Controller" template by 91%, confirming the speaker's identity as a controller.

[0195] Conclusion: ASR misidentified "DUMET" as "DUMMY" and should be corrected.

[0196] 5. Step S5: Generate the final result

[0197] The LLM inference engine outputs the correction results in JSON format:

[0198] {

[0199] "corrected_role": "ATC",

[0200] "corrected_text": "CES1234 direct flight to DUMET",

[0201] "reasoning": "The ASR output 'DUMMY' is not a valid waypoint, while 'DUMET' in the knowledge base is a valid waypoint located in the current sector. Based on voiceprint verification (matching the controller template) and the dialogue context, this is corrected."

[0202] "speaker_identity": "Beijing Area 01 Controller",

[0203] "confidence": 0.94

[0204] }

[0205] The system feeds back the result to the controller's interface (displaying the corrected instruction text as subtitles), and simultaneously transmits it to downstream automation systems (such as flight planning systems, conflict detection systems, etc.) through standard interfaces for subsequent business use.

[0206] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0207] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0208] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0209] In addition, each module will only collect and store user information with the user's full authorization and in compliance with relevant laws and regulations, and will protect the security and privacy of user data, strictly prohibiting unauthorized access; data processing will be carried out within the scope of the law and will not exceed the purpose and scope authorized by the user; at the same time, users have the right to access, correct, delete, restrict processing, and refuse their personal data; and will strictly comply with applicable laws and regulations and conduct compliance reviews.

[0210] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An authentication system based on a large language model, characterized in that, include: The streaming joint recognition module layer is used to receive the original air traffic control speech stream in real time and perform streaming joint perception processing on the original air traffic control speech stream to obtain a preliminary recognition result; the preliminary recognition result is an interleaved sequence including preliminary role labels and ASR text; Domain knowledge base layer, a structured database used to provide visible retrieval; The LLM semantic correction and reasoning module layer is used to receive the preliminary recognition results output by the streaming joint recognition module layer, combine the large language model and the structured database provided by the domain knowledge base layer to complete cognitive verification, and output the final recognition result; the final recognition result is the corrected role label and ASR text with higher confidence.

2. The system as described in claim 1, characterized in that, The streaming joint identification module layer includes: The SE front-end uses a lightweight convolutional architecture to receive raw air traffic control audio streams in real time and perform noise reduction processing on them. The streaming encoder employs a block-based causal Conformer architecture to decompose, enhance, and extract contextual acoustic features from the denoised original air traffic control speech stream to obtain encoded data. An E2E joint decoder is used to decode the encoded data in a streaming manner, and outputs an interleaved sequence including preliminary role tags and ASR text; The voiceprint template registration management unit, in collaboration with the E2E joint decoder, is used to complete the creation of voiceprint templates and streaming voiceprint verification.

3. The system as described in claim 2, characterized in that, The voiceprint template registration management unit is used for: If a specific identity keyword is identified in the interleaved sequence, the speech segment containing the keyword is automatically extracted, and the speech segment is processed using a high-performance speaker embedding model to obtain a voiceprint embedding vector. Obtain the logical identity associated with the keyword, bind the voiceprint embedding vector to the logical identity to form a voiceprint template, and store the voiceprint template in the voiceprint template buffer; Based on the voiceprint template, streaming voiceprint verification is performed on subsequent speech data to obtain a high-confidence voiceprint verification signal, and the voiceprint verification information is passed to the LLM semantic correction and inference module layer; the subsequent speech data is the speech segment that is subsequently input and has been initially labeled with role tags by the E2E joint decoder.

4. The system as described in claim 1, characterized in that, The structured database stores global waypoints, routes, airline call sign lists, ICAO standard terminology and abbreviations, airport and sector information; the domain knowledge base layer includes: The data collection unit is used to collect raw data, including official civil aviation standard documents, compilations of flight information, aviation databases, airline information and call signs. The data processing unit is used to perform multimodal data extraction, knowledge graph construction, entity normalization and linking, and hybrid indexing and storage based on the raw data, so as to build a graph database and a vector database. The graph database is used to store entities and complex structured relationships to assist the LLM semantic correction and reasoning module layer in performing multi-hop logical reasoning. The vector database is used to store vector embeddings of all entities, terms and descriptive text. The data processing unit is also used to build a retrieval API interface for the LLM semantic correction and inference module layer to call.

5. The system as described in claim 4, characterized in that, The LLM semantic correction and reasoning module layer includes: The context aggregation unit is used to receive the preliminary recognition results output by the streaming joint recognition module layer, and at the same time load the most recent N rounds of historical dialogue data cached by the system; The knowledge retrieval and injection unit is used to retrieve a knowledge enhancement list based on the preliminary identification results and the vector database. The dynamic prompt construction unit is used to obtain the system preset instructions, and assemble the system preset instructions, preliminary recognition results, recent N rounds of historical dialogue data and the associations stored in the graph database into LLM prompt words according to a predefined template; The LLM inference engine is used to perform inference correction based on the LLM prompts and output the final recognition result.

6. An authentication method based on a large language model, characterized in that, The method is applicable to the system according to any one of claims 1-5, comprising: A domain knowledge base layer is constructed; the domain knowledge base layer stores a graph database and a vector database; Construct a voiceprint template; Real-time reception of raw air traffic control voice stream; The original air traffic control speech stream is processed by a streaming joint recognition module layer to obtain a preliminary recognition result; the preliminary recognition result is an interleaved sequence including preliminary role labels and ASR text; Based on the voiceprint template and preliminary recognition results, streaming voiceprint verification is performed to obtain a voiceprint verification signal; The LLM semantic correction and inference module layer is used to complete cognitive verification by combining the preliminary recognition results, voiceprint verification signals, graph database and vector database, and output the final recognition result; the final recognition result is the corrected role label and ASR text with higher confidence.

7. The method as described in claim 6, characterized in that, The specific steps for constructing the domain knowledge base layer are as follows: Collect raw data; the raw data includes official civil aviation standard documents, compilations of flight information, aviation databases, airline information and call signs; Based on the original data, multimodal data extraction, knowledge graph construction, entity normalization and linking, and hybrid indexing and storage are implemented to build a graph database and a vector database. The graph database is used to store entities and complex structured relationships to assist the LLM semantic correction and reasoning module layer in performing multi-hop logical reasoning. The vector database is used to store vector embeddings of all entities, terms, and descriptive text. Build a retrieval API interface for the LLM semantic correction and inference module layer to call.

8. The method as described in claim 6, characterized in that, The specific steps for constructing a voiceprint template are as follows: Acquire the empty tube data stream of the sample; The original air traffic control speech stream is processed by a streaming joint recognition module layer to obtain sample recognition results; the sample recognition results are an interleaved sequence including preliminary role labels and ASR text. If a specific identity keyword is identified in the interleaved sequence, the speech segment containing the keyword is automatically extracted, and the speech segment is processed using a high-performance speaker embedding model to obtain a voiceprint embedding vector. Obtain the logical identity associated with the keyword, bind the voiceprint embedding vector to the logical identity to form a voiceprint template, and store the voiceprint template in the voiceprint template buffer.

9. The method as described in claim 6, characterized in that, The streaming joint identification module layer includes an SE front-end, a streaming encoder, and an E2E joint decoder; the preliminary identification results are as follows: The SE front-end is used to receive the raw air traffic control voice stream in real time and perform noise reduction processing on it; The streaming encoder is used to decompose, enhance, and extract contextual acoustic features from the denoised original air traffic control speech stream to obtain encoded data. The E2E joint decoder is used to decode the encoded data in a streaming manner, and a preliminary recognition result is output.

10. The method as described in claim 6, characterized in that, The final recognition result is output as follows: Receive the preliminary recognition result output by the streaming joint recognition module layer, and load the most recent N rounds of historical dialogue data cached by the system; Based on the preliminary identification results and the vector database, a knowledge-enhanced list is obtained. Obtain system preset instructions, and assemble the system preset instructions, preliminary recognition results, recent N rounds of historical dialogue data, and the relationships stored in the graph database into LLM prompt words according to a predefined template; Based on the LLM prompts, inference correction is performed, and the final recognition result is output.

Images