Intelligent question and answer method and system based on air traffic control industry language model and improved RAG

Abstract

The embodiment of the application discloses an intelligent question and answer method and system based on an air traffic control industry language model and an improved RAG; the method comprises the following steps: obtaining a user question, inputting the user question into an improved RAG algorithm for processing to obtain original text of a knowledge base; inputting the user question and the original text of the knowledge base into a pre-constructed and trained air traffic control industry language model for fusion reasoning to generate a question and answer result. Through the introduction of an air traffic control field corpus and a knowledge graph, the model is fine-tuned in the field, and professional term understanding is strengthened; meanwhile, a rule constraint and a traceable mechanism are added in the retrieval and generation links to ensure the reliability and safety of the generated result. The improved RAG system not only retains the understanding and generation advantages of the large model, but also meets the requirements of air traffic control for real-time performance, accuracy and interpretability, realizes the balance between flexibility and engineering controllability, and significantly improves the intelligent level and practical value of information processing in the air traffic control scene.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of air traffic management and artificial intelligence, specifically to an intelligent question-answering method and system based on an air traffic control industry language model and an improved RAG. Background Technology

[0002] Existing technology 1 provides a system based on keyword or keyword pinyin matching and template generation, which is widely used in air traffic control industry information management platforms. The basic principle of this system is to match and parse air traffic control commands input via voice recognition or text through a predefined command template library and rule templates (such as matching initial letter abbreviations in pinyin). This method is simple to implement and has a fast response time, but it relies on manually defined rules, making it difficult to adapt to complex and varied command formats. Furthermore, it cannot handle unfamiliar collocations, synonyms, or colloquial expressions (simply put, the matching is quite strict and cannot accommodate synonyms or misspellings). In other words, while traditional keyword matching systems are fast, they lack generalization ability and struggle to handle the diverse expressions in air traffic control commands.

[0003] Existing technology 2 provides a retrieval-augmented generation (RAG) question-answering system based on a general large language model. This system uses a general-domain RAG, employing semi-open-source or open-source frameworks such as Dify and AutoGPT, and combines a large language model (LLM) to build the RAG process. While this approach performs well in open-domain question answering, it faces several challenges in air traffic control scenarios: Firstly, the general language model lacks a professional understanding of aviation terminology, operating procedures, and communication protocols. For example, ATM might be interpreted as "Automated Teller Machine," whereas in air traffic control, ATM typically refers to Air Traffic Management. Secondly, air traffic control tasks place extremely high demands on system response latency and decision interpretability, while RAG systems based on general large models often suffer from long inference times due to their reliance on large-scale parameter models. In other words, while general RAGs possess language generation capabilities, they lack specialized semantic understanding in the air traffic control domain, easily leading to ambiguity and response delays.

[0004] In summary, existing air traffic control information processing systems still have shortcomings in terms of semantic understanding depth, response efficiency, and interpretability. Summary of the Invention

[0005] To address the technical deficiencies of existing technologies, the purpose of this invention is to provide an intelligent question-answering method and system based on an air traffic control industry language model and an improved RAG, in order to achieve the following objectives:

[0006] 1. Enhance adaptability to complex command expressions: By defining and training, a professional language model with knowledge of the air traffic control domain is introduced to enhance the system's ability to adapt to and understand diverse commands such as synonyms, misspellings, and colloquial expressions, significantly improving the flexibility and robustness of RAG.

[0007] 2. Reduce manual maintenance and update costs: By leveraging the semantic understanding capabilities of the language model, the system avoids heavy reliance on static templates and manual rules, automatically adapts to different expression forms, reduces manual intervention, and improves the system's scalability and maintenance efficiency.

[0008] 3. Enhance the professionalism and accuracy of semantic understanding: By integrating air traffic control terminology database, procedural knowledge and communication protocol specifications, the system can achieve higher professional semantic consistency in the process of command parsing and response generation, reduce the risk of semantic deviation, and improve the controllability and security of output content.

[0009] 4. Meets the requirements of air traffic control scenarios for real-time performance and low latency: Through structural optimization and lightweight model design (the NavLight proposed in this invention has only 85M parameters, far lower than the large models on the market that often have hundreds of billions of parameters), the inference cost is effectively reduced while maintaining the ability to understand, and the response speed is accelerated, so as to better adapt to the real-time decision-making needs of air traffic management tasks.

[0010] 5. Improve and optimize specific steps of RAG to enhance the accuracy of knowledge matching and the relevance of generated results, thereby further strengthening the overall performance of the system.

[0011] To achieve the above objectives, in a first aspect, embodiments of the present invention provide an intelligent question-answering method based on an air traffic control industry language model and an improved RAG, comprising:

[0012] Obtain the user question, input the user question into the improved RAG algorithm for processing, and obtain the original text of the knowledge base;

[0013] The user question and the original text of the knowledge base are input into a pre-built and trained air traffic control industry language model for fusion reasoning to generate question-and-answer results;

[0014] The improved RAG algorithm includes a combination strategy that integrates the recursive segmentation mechanism of the air traffic control industry, the air traffic control title enhancement mechanism, and the context-aware compression retrieval combined with cross-coding reordering; the air traffic control industry language model includes LayerNorm normalization, GELU activation function, and rotation position embedding.

[0015] As a specific implementation of this application, the user question is input into the improved RAG algorithm for processing to obtain the original knowledge base text, specifically:

[0016] Knowledge documents related to the air traffic control industry are obtained, and the knowledge documents are preprocessed by combining a recursive segmentation mechanism and an air traffic control title enhancement mechanism.

[0017] The preprocessed knowledge document is vectorized to obtain embedded vectors that the model can understand, and the embedded vectors are stored in the local retrieval system;

[0018] Receive user questions input in natural language form and vectorize the user questions into query embedding vectors that the model can understand;

[0019] A combined strategy of context-aware compressed retrieval and cross-coding reordering is adopted. Based on the query embedding vector, the query is matched in the local retrieval system to obtain the original text of the knowledge base.

[0020] As a specific implementation of this application, the process of integrating the recursive segmentation mechanism of the air traffic control industry is as follows:

[0021] The air traffic control document is obtained, and the document is initially segmented using a predefined air traffic control industry delimiter to obtain multiple text blocks. The air traffic control industry delimiter includes a message field header, a flight plan segment marker, a meteorological message marker, a NOTAM marker, and a radar conflict event marker.

[0022] Iterate through all text blocks, merge the text blocks that are too short, and obtain text blocks that meet the length requirements and have relatively complete semantics;

[0023] For excessively long text blocks, a custom universal delimiter is used to recursively subdivide them, resulting in text blocks that meet the length requirements and have relatively complete semantics.

[0024] As one specific implementation of this application, the process of the air traffic control title enhancement mechanism is as follows:

[0025] For the segmented text blocks that meet the length requirements and are relatively semantically complete, multiple scoring reference factors are used to score them in order to determine the title. The multiple scoring reference factors include whether it contains commonly used keywords in air traffic control industry titles, whether it contains numbers, whether there are line breaks before and after it, whether the paragraph indentation is 0, and whether it contains punctuation marks.

[0026] The determined title is appended as a prefix to the corresponding subsequent body paragraphs.

[0027] As a specific implementation of this application, the combined strategy of context-aware compressed retrieval and cross-coding reordering is as follows:

[0028] Based on the initial query vector q, a preliminary set of relevant documents D is obtained from the vector database;

[0029] The sub-paragraphs within each document di in the set of related documents D are scored using a local semantic similarity function;

[0030] Sub-paragraphs with scores greater than the threshold are retained to form a candidate document set D';

[0031] The candidate document set D' is subjected to deep semantic matching and re-ranking using a re-ranking model, and the most relevant top-k results are finally returned.

[0032] As a preferred implementation of this application, the method further includes constructing and training an air traffic control industry language model, specifically:

[0033] Define the structural parameters of the language model for the air traffic control industry; the structural parameters include the number of parameters, vocabulary size, number of Tranformer layers, model size, hidden layer dimension, number of attention heads, and feedforward neural network dimension;

[0034] LayerNorm normalization, GELU activation function, and rotational position embedding are introduced;

[0035] Acquire text content related to the field of air traffic management and combine it with knowledge distillation to assist in model pre-training;

[0036] Acquire dialogue data and use the dialogue data to perform single-round dialogue fine-tuning on the air traffic control industry language model after the model pre-training is completed;

[0037] Obtain a corpus containing consecutive dialogue rounds, and use the corpus to fine-tune the language model for the air traffic control industry through multi-turn dialogue.

[0038] Obtain dialogue instances and use these instances to perform direct preference optimization on the air traffic control industry language model; the dialogue instances include exchanges between experts and real communication records between pilots and ground control centers.

[0039] Secondly, embodiments of the present invention also provide an intelligent question-answering system based on an air traffic control industry language model and an improved RAG, including a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method described in the first aspect.

[0040] Thirdly, embodiments of the present invention also provide another intelligent question-answering system based on an air traffic control industry language model and an improved RAG, including:

[0041] The retrieval and query unit is used to obtain user questions, input the user questions into the improved RAG algorithm for processing, and obtain the original text of the knowledge base;

[0042] The fusion reasoning unit is used to input the user question and the original text of the knowledge base into a pre-built and trained air traffic control industry language model for fusion reasoning and to generate question-and-answer results.

[0043] The improved RAG algorithm includes a combination strategy that integrates the recursive segmentation mechanism of the air traffic control industry, the air traffic control title enhancement mechanism, and the context-aware compression retrieval combined with cross-coding reordering; the air traffic control industry language model includes LayerNorm normalization, GELU activation function, and rotation position embedding.

[0044] Furthermore, as a preferred implementation of this application, the intelligent question-answering system further includes a model training unit for constructing and training an air traffic control industry language model, specifically:

[0045] Define the structural parameters of the language model for the air traffic control industry; the structural parameters include the number of parameters, vocabulary size, number of Tranformer layers, model size, hidden layer dimension, number of attention heads, and feedforward neural network dimension;

[0046] LayerNorm normalization, GELU activation function, and rotational position embedding are introduced;

[0047] Acquire text content related to the field of air traffic management and combine it with knowledge distillation to assist in model pre-training;

[0048] Acquire dialogue data and use the dialogue data to perform single-round dialogue fine-tuning on the air traffic control industry language model after the model pre-training is completed;

[0049] Obtain a corpus containing consecutive dialogue rounds, and use the corpus to fine-tune the language model for the air traffic control industry through multi-turn dialogue.

[0050] Obtain dialogue instances and use these instances to perform direct preference optimization on the air traffic control industry language model; the dialogue instances include exchanges between experts and real communication records between pilots and ground control centers.

[0051] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the method described in the first aspect.

[0052] The advantages of implementing the embodiments of the present invention are as follows:

[0053] 1. Enhanced adaptability to complex command semantics: This invention introduces a language model that integrates professional knowledge from the air traffic control field, and embeds industry terminology, communication standards, and other knowledge during the pre-training and fine-tuning stages. This enables the model to have contextual semantic understanding capabilities, effectively parse diverse expression forms, and significantly improve the robustness and adaptability of command recognition.

[0054] 2. Enhance system generalization and compatibility: Adopting a dynamic knowledge retrieval mechanism and semantic context understanding strategy based on the RAG framework, the retrieval module automatically calls knowledge documents related to the current question, and the generation module outputs the answer. It can autonomously adapt to expression differences without relying on manual template updates, which greatly improves the automation level and maintenance efficiency of the system.

[0055] 3. More professional semantic understanding, with industry-consistent and controllable generated content: This invention embeds air traffic control terminology, operating procedures, and communication protocols into the model corpus and knowledge base, enabling the system to possess highly consistent professional semantic understanding capabilities. Simultaneously, domain knowledge constraints and response control mechanisms are introduced in the generation process to enhance the controllability, security, and professional credibility of the system's output results.

[0056] 4. Possesses low-latency response capability to meet real-time scenario requirements: Proposes a lightweight industry language model NavLight (with only 85M parameters), which maintains semantic understanding capability while significantly reducing inference costs and shortening response time, meeting the real-time requirements of rapid decision-making in air traffic management tasks.

[0057] 5. Improved RAG process integrating air traffic control characteristics to enhance knowledge matching accuracy and generation quality: This invention optimizes the RAG process in multiple ways, including recursive segmentation integrating air traffic control industry features, enhanced air traffic control titles, and a combination of context compression retrieval and cross-coding reordering. Multi-module collaborative optimization of the relevance and accuracy of generated results effectively improves the overall performance of the system.

[0058] 6. The system has good controllability, security and flexible configuration capabilities.

[0059] The system allows users to flexibly configure parameters such as the number of original text fragments retrieved (Top-k) and the context memory length of the language model, enhancing the model's adaptability to different scenarios. Simultaneously, this invention provides comprehensive knowledge base management functions, supporting the creation, modification, deletion, access control, and annotation management of knowledge. It also implements access control based on permissions during the retrieval process, ensuring the security and accuracy of data usage. These functions far surpass the current technical limitations of traditional RAG systems, which suffer from uncontrollable knowledge, unmanageable permissions, and inflexible configuration. Attached Figure Description

[0060] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0061] Figure 1 This is a flowchart of the intelligent question-answering method based on the air traffic control industry language model and improved RAG provided in the embodiments of the present invention;

[0062] Figure 2 yes Figure 1 Another flowchart of the method shown;

[0063] Figure 3 This is an architecture diagram of the NavLight model in the implementation of this invention;

[0064] Figure 4 This is a flowchart of the NavLight model training process;

[0065] Figure 5 This is an improved RAG architecture diagram of the present invention in conjunction with the air traffic control industry;

[0066] Figure 6 This is a schematic diagram of the main interface for deploying this invention;

[0067] Figure 7 This is a diagram illustrating knowledge base management.

[0068] Figure 8 This is a structural diagram of the intelligent question-answering system based on the air traffic control industry language model and improved RAG provided in this embodiment of the invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0071] The inventive concept of this invention is to construct a RAG method and system that integrates industry-specific professional knowledge, possesses semantic adaptability, rapid response capabilities, and excellent response quality. This invention's embodiments introduce a dedicated corpus and knowledge graph for air traffic control, fine-tuning the model to enhance its understanding of technical terms. Simultaneously, rule constraints and traceability mechanisms are added to the retrieval and generation stages to ensure the reliability and security of the generated results. The improved RAG system retains the advantages of understanding and generating large models while meeting the air traffic control requirements for real-time performance, accuracy, and interpretability, achieving a balance between flexibility and engineering controllability, and significantly improving the intelligence level and practical value of information processing in air traffic control scenarios.

[0072] Please refer to Figure 1 and Figure 2 The intelligent question-answering method based on the air traffic control industry language model and improved RAG provided in this embodiment of the invention may include the following steps:

[0073] S1, constructing and training the language model for the air traffic control industry and designing an improved RAG algorithm.

[0074] This invention focuses on innovation in language modeling and RAG capability optimization for the air traffic control industry, and elaborates on the following two aspects: First, it proposes and trains a lightweight language model, NavLight, specifically designed for the air traffic control industry scenario; second, based on the air traffic control industry, it proposes and implements an improved RAG process optimization and system architecture, covering key optimization technology modules for air traffic control industry retrieval.

[0075] (I) Construction and Training of NavLight, a Language Model for the Air Traffic Control Industry

[0076] This invention, based on the conventional design principles of Large Models (LLM), optimizes the Decoder-Only architecture of Transformer by simplifying parameters and structure to design the NavLight model, whose architecture is as follows. Figure 3 As shown. This model has fewer than 100M parameters, reducing computational overhead while maintaining model performance, thus achieving more efficient deployment. The specific structural parameters of the NavLight model are as follows:

[0077] Table 1

[0078]

[0079] In terms of technical configuration, NavLight adopts mainstream technical components such as LayerNorm normalization, GELU activation function, and Rotary Positional Embedding (RoPE), achieving a good balance between parameter count, inference efficiency, and practical application requirements while improving model performance. The innovation of the NavLight model proposed in this invention is mainly reflected in:

[0080] 1. NavLight uses LayerNorm normalization to normalize the input of each Transformer sublayer to more effectively address the issue of internal covariate shift and accelerate the training process. LayerNorm is more effective at handling internal covariate shift and helps accelerate training and improve model performance. The formula is:

[0081]

[0082] γ and β are the scaling parameter (scale) and the offset parameter (shift), with initial values ​​of 1 and 0, respectively. This represents the mean and variance of all elements in the eigenvector.

[0083] 2. NavLight uses the GELU (Gaussian Error Linear Unit) activation function instead of the traditional ReLU, which better captures feature distribution and enhances the model's expressiveness while maintaining non-linearity. Its form is as follows:

[0084]

[0085] in This refers to the Gaussian normal distribution function of x, expressed as:

[0086]

[0087] in, These represent the mean, standard deviation, and variance of a normal distribution.

[0088] 3. Replace absolute position embedding with Rotation Position Embedding (RoPE). RoPE encodes absolute position information into the Transformer by using rotation operations in Euclidean rotation space.

[0089] The rotation matrix is ​​defined as:

[0090]

[0091] Where θ is the rotation angle, and cosθ and sinθ are the trigonometric function terms that make up the rotation matrix, used to encode position information.

[0092] When expanding to higher-dimensional spaces, RoPE divides the space into d / 2 subspaces (as shown in Table 1, d is the hidden dimension of the Transformer, d_model=768), and creates a rotation matrix of size n for each word at its position i. The feature matrix is ​​rotated by an angle proportional to its position index to achieve relative positional encoding.

[0093]

[0094] The training process of the NavLight model follows a phased and progressively refined strategy, aiming to ensure that the model not only masters the professional knowledge of the air traffic control industry, but also has efficient dialogue understanding and generation capabilities.

[0095] After defining the model, the next step is to inject knowledge into the model, which is called training. The training flowchart is as follows: Figure 4 As shown, the main steps include the following:

[0096] 1. Pre-training

[0097] The purpose of pre-training is to build a foundation in language and industry knowledge understanding. Understandably, the model's task at this stage is to predict the next token. The dataset primarily comes from text content related to the field of aviation traffic management, including but not limited to industry standard documents, operation manuals, training materials, and other professional knowledge bases. Furthermore, to enhance the system's versatility and fault tolerance, Wikipedia, news reports, and other various documents are added as supplementary materials. These texts are prepared in plain text format and fed to the model.

[0098] In this embodiment, the number of epochs in the pre-training phase is set to 50 to ensure that the model can fully absorb and understand a large amount of knowledge. The learning rate during training adopts a dynamic learning rate strategy, adjusting between 1e-4 and 1e-5 to adapt to changes in NavLight's performance during training. The input sequence length is 4096 tokens.

[0099] Furthermore, during the pre-training process, to improve model capabilities and training convergence speed, this invention incorporates knowledge distillation to assist training. Specifically, through comparative experiments, this invention selects the DeepSeek-R1 model as the teacher network and NavLight as the student network. This method allows the student network to effectively absorb knowledge from the teacher network, improving its own performance. The loss function used during training in this invention is the student loss. and distillation loss The combination of:

[0100]

[0101] Where a and b are loss weight coefficients, which are 0.4 and 0.6 respectively in this invention.

[0102] Distillation loss uses mean squared loss (MSE) to measure the difference between the teacher network's output vector ZT (the soft target after softmax processing) and the student network's output vector ZS (that is, the vector after softmax and before the tokenizer decoder in the detailed model structure diagram, i.e., the soft prediction).

[0103]

[0104] The student loss is the KL divergence between the true labeled value Y (i.e., the hard target) and the model output of the student network (i.e., the hard prediction Yt).

[0105] =KL(Y‖Yt)

[0106] The training process of the NavLight model follows a phased and progressively refined strategy, which aims to ensure that the model not only masters professional knowledge in the field of air traffic management, but also has efficient dialogue understanding and generation capabilities.

[0107] 2. Fine-tuning of single-round dialogue

[0108] The essence of the pre-training task in the previous step lies in continuation writing, but this method is more suitable for text generation and may not be able to answer user questions well. To improve the model's performance in dialogue tasks, especially its ability to answer user questions, this invention introduces single-turn dialogue fine-tuning after pre-training, aiming to align the NvaLight model with the dialogue structure.

[0109] This phase of the dataset focuses on dialogue data, such as short question-and-answer exchanges between pilots and air traffic controllers; it also needs to cover dialogue data from other broad domains to improve the model's ability to handle non-professional conversations and unusual situations. Each question-and-answer pair is carefully crafted to ensure the model learns accurate and appropriate responses.

[0110] During this fine-tuning phase, the learning rate is set between 1e-5 and 1e-6 to achieve more precise parameter adjustments, and the weight decay is 0.1.

[0111] 3. Fine-tuning through multiple rounds of dialogue

[0112] The goal of multi-turn dialogue fine-tuning is to enable NavLight to generate coherent responses while taking into account dialogue history, thereby improving its understanding and generation of multi-turn dialogues.

[0113] Therefore, this phase used a corpus containing consecutive dialogue rounds, such as actual aviation communication records or simulated dialogues, daily conversations, etc., which reflect real communication scenarios.

[0114] In this way, the model can learn how to maintain topic consistency in multi-turn conversations while responding appropriately based on contextual information.

[0115] 4. The final stage is the Direct Preference Optimization (DPO) method.

[0116] The goal of this phase is to further customize the NavLight model output style, improve the generation quality and align with industry preferences, making it more responsive to the specific needs of aviation industry users.

[0117] This step focuses specifically on key dialogue instances involving flight planning, flight status updates, and air traffic management decisions. Data sources include both expert exchanges and real-world communication logs between pilots and ground control. All dialogue instances are structured so that the model can learn to recognize user preferences, thereby improving the naturalness and fluency of the dialogue. This type of data is typically stored in the form of dialogue trees or sequences to help the model respond optimally in complex situations.

[0118] It is worth noting that this invention incorporates a suitable amount of general dialogue-optimized datasets at each training stage to enhance the model's dialogue processing capabilities and user experience without compromising its focus on the core domain. Furthermore, as the application environment changes and technology advances, the dataset should be continuously expanded, and the quality of the content and human annotations will directly determine the upper limit of the model's performance. Therefore, continuous iteration and optimization of the model are crucial for adapting to the diverse needs of aviation operations, ensuring that NavLight consistently provides efficient and secure decision support.

[0119] (II) RAG Process Optimization and System Implementation

[0120] RAG (Retrieval-Augmented Generation) is an architecture that integrates information retrieval and generative language models. Its core idea is to retrieve relevant information from the knowledge base as context before the generative model answers the user's question, and then input it into the generative model for reasoning and answering, thereby dynamically introducing private knowledge without retraining the model.

[0121] By introducing RAG technology, this system can combine its internal private knowledge base with the latest data sources, laws, and policies to conduct in-depth analysis of knowledge in the air traffic control industry and generate more accurate answers. This not only significantly improves the system's responsiveness and the scientific nature of its decision-making, but also provides aviation managers with more reliable support when facing complex and ever-changing aviation policies.

[0122] The improved RAG architecture of this invention, tailored to the air traffic control industry, is shown in the diagram below. Figure 5 As shown, the main steps include the following:

[0123] First, the knowledge document is preprocessed. This invention uses recursive segmentation that integrates the characteristics of the air traffic control industry, combined with air traffic control title enhancement, to improve contextual coherence and preserve semantic integrity.

[0124] The second step is vectorization, which involves building a knowledge index. Simply put, this step transforms unstructured document information into embedded vectors that the model can understand and stores them in a local retrieval system (FAISS used in this invention).

[0125] Next is the user query phase, where the system receives user questions in natural language and then vectorizes them into embedding vectors that the model can understand.

[0126] Subsequently, the retrieval module quickly searches the knowledge base based on the semantics of the question, matching and extracting relevant original text. To improve the competitiveness and practicality of this invention in air traffic management scenarios, the following steps are optimized:

[0127] 1. Integrating recursive character text splitting in the air traffic control industry.

[0128] Traditional text segmentation methods often segment data based on fixed character lengths. While simple and efficient, this approach can easily disrupt the logical structure of critical data in air traffic control corpora (including flight plans (FPL), air traffic control clearance instructions (ATC), weather notices (METAR / TAF), radar conflict warning logs, and route change instructions). For example, if information such as "flight number—route—altitude layer—speed limit—estimated time" is broken down, it can lead to a decrease in the accuracy of automated semantic understanding, flight conflict detection, and flight plan parsing.

[0129] To address this, this invention introduces a recursive segmentation mechanism that integrates the air traffic control industry. Its core advantage is the intelligent segmentation of text while maintaining semantic integrity and logical continuity. This method performs multi-level recursive segmentation using predefined air traffic control industry delimiters and general delimiters. The entire segmentation process is as follows: first, initial segmentation is performed using industry delimiters; then, the text block length is checked; excessively short text blocks are merged; excessively long text blocks are recursively subdivided according to delimiter priority; finally, text blocks that meet the length requirements and are semantically relatively complete are generated.

[0130] Examples of air traffic control industry separators include:

[0131] Message header fields: such as FPL:, RMK:, TFC:

[0132] Flight plan segment markers: such as flight departure information, route segment information

[0133] Weather report symbols: such as METAR:, TAF:

[0134] Notification of Navigation (NOTAM / ATIS): such as NOTAM:, ATIS:

[0135] Radar collision event markers: such as ALERT:, CONFLICT:

[0136] These markers are typically present at the beginning of text, serving as boundaries for paragraphs or blocks of information, and thus can be used as document separators. However, they are generally unsuitable as sentence-level identifiers because when they appear within a sentence, they often represent only a portion of the content and do not indicate paragraph boundaries.

[0137] Common delimiters include newline character \n, period, comma, etc. During recursive splitting, the system prioritizes industry-specific delimiters, followed by common delimiters for secondary or tertiary splitting.

[0138] Specifically, the segmentation process includes:

[0139] The first step is to use the air traffic control industry delimiters mentioned above to perform a preliminary segmentation of, for example, an air traffic control document, resulting in multiple text blocks of varying lengths.

[0140] More specifically, the entire air traffic control document is traversed using air traffic control industry delimiters to find positions that can be used to split the document, and preliminary segmentation is performed based on these positions.

[0141] The second step involves iterating through the multiple text blocks after the initial segmentation, merging those that are too short to obtain text blocks that meet the length requirements and have relatively complete semantics. The size of the text blocks can be customized as needed. Generally, text blocks exceeding 1024 tokens need to be further segmented. Blocks below 100 tokens need to be merged with adjacent text blocks to ensure semantic integrity.

[0142] The third step involves recursively splitting large text blocks according to the priority of common delimiters, ultimately generating text blocks that meet the length requirements and have relatively complete semantics.

[0143] It should be noted that the general delimiters are customizable according to requirements and can be set to different priorities. For example, second-level delimiters include headings, chapters, and first-level headings; third-level delimiters can include more granular delimiters such as general punctuation and special characters. The first step of primary segmentation can be understood as using the highest priority air traffic control industry delimiter. After that, the length of the text block after primary segmentation is checked. If it exceeds the length threshold, second-level and third-level delimiters are used to further recursively subdivide it until the text block length meets the requirements or no lower-level delimiters are available.

[0144] This segmentation method can be recursively applied to text, ensuring the continuity and integrity of text blocks while performing more refined and flexible text segmentation. Therefore, these segmented text fragments are usually the most semantically related, providing a better foundation for subsequent retrieval answers.

[0145] 2. Enhanced air traffic control title.

[0146] Air traffic control documents often contain a mixture of information, such as instruction text, flight plans, route modifications, and weather alerts. Furthermore, titles and text are frequently blurred and intertwined, leading to decreased semantic matching accuracy in automated processing. To address this, this invention proposes a title enhancement mechanism that explicitly embeds title information into corresponding paragraphs to improve the accuracy of retrieval, classification, and information extraction. The process is as follows:

[0147] The first step is to identify the title.

[0148] This invention proposes an air traffic control title recognition method based on a comprehensive scoring mechanism. By scoring multiple features of a text block, it determines whether it is a title. The scoring reference factors include, but are not limited to:

[0149] Does it contain commonly used keywords in air traffic control titles, such as control, handover, separation, coordination, conflict, ALERT, emergency, alarm, etc.?

[0150] Does it contain numbers?

[0151] Are there newline characters at both the beginning and the end?

[0152] Is the paragraph indentation set to 0?

[0153] Should it be centered? Should bold font be used?

[0154] Is the font size larger than the surrounding content?

[0155] Is the segment length no more than 10 characters;

[0156] Does it have bullet points or numbering (such as "1.", "I.", etc.);

[0157] Does it contain punctuation marks;

[0158] Set corresponding weights according to the above feature items, and calculate the comprehensive score of the text block. If the score exceeds the preset threshold, it is determined as a title.

[0159] In the second step, the identified title is appended as a prefix to the subsequent corresponding body paragraph.

[0160] 3. Retrieval Strategy Optimization

[0161] To further improve the accuracy and robustness of the present invention in the retrieval step, the present invention experimentally compares the performance of multiple retrieval strategies, and finally the system adopts a combined strategy of contextual compression retrieval combined with cross-encoder reranking to implement the retrieval strategy. This solution performs well in multiple evaluation dimensions. Although its computational overhead is slightly higher than that of lightweight solutions, it shows stronger adaptability and practicability in practical applications, so it is selected as the main retrieval module.

[0162] This combined strategy not only effectively filters irrelevant content and enhances semantic relevance, but also has strong contextual understanding capabilities, and is especially suitable for scenarios with high accuracy requirements such as complex question answering and multi-turn conversations. Specifically, the combination realizes efficient semantic retrieval through two stages:

[0163] First stage: Contextual Compression Retriever

[0164] First, the user input passes through an embedding model to obtain an embedding vector q. This module first recalls a preliminary set of related documents D={d1,d2,...,dn} from the vector database based on the initial query q. Then, a local semantic similarity function S(q,di) is used to score the sub-paragraphs inside each document di, and the parts with scores higher than the threshold θ are retained, so as to form a streamlined document set D'={d1',d2',...,dm'}, where m≤n.

[0165] The formula is expressed as follows:

[0166]

[0167] Here, s represents a sub-segment in document di, θ is a set semantic relevance threshold, and the similarity function S(q,s) measures the semantic relevance between user query q and document sub-segment s. This paper adopts a vector space-based distance metric, encoding the query and the segment into vector representations q and s respectively, and using L1 distance (Manhattan distance) as the similarity scoring criterion.

[0168]

[0169] Phase 2: CrossEncoderReranker

[0170] After obtaining the compressed candidate document set D', the re-ranking model is further used to perform deep semantic matching and re-ranking on these documents. By jointly modeling the interaction relationship between the query and the document, a more reliable relevance score R(q,di′) is output, thereby globally ranking the candidate documents and finally returning the most relevant Top-k results.

[0171] The sorting function can be expressed as:

[0172]

[0173] This process significantly improves the relevance and accuracy of search results, and is especially suitable for high-precision semantic retrieval scenarios such as complex question-and-answer and multi-turn dialogues.

[0174] After obtaining the original text through RAG, the NavLight language model proposed earlier is used to fuse and infer the retrieved original text content, generating accurate and highly interpretable answers. This entire process effectively synergizes external knowledge with model capabilities, significantly enhancing the system's question-answering accuracy and practicality.

[0175] S2: Obtain the user's question, input it into the improved RAG algorithm for processing, and obtain the original text of the knowledge base.

[0176] In specific implementation, based on the aforementioned description of the improved RAG algorithm, step S2 includes:

[0177] Knowledge documents related to the air traffic control industry are obtained, and the knowledge documents are preprocessed by combining a recursive segmentation mechanism and an air traffic control title enhancement mechanism.

[0178] The preprocessed knowledge document is vectorized to obtain embedded vectors that the model can understand, and the embedded vectors are stored in the local retrieval system;

[0179] Receive user questions input in natural language form and vectorize the user questions into query embedding vectors that the model can understand;

[0180] A combined strategy of context-aware compressed retrieval and cross-coding reordering is adopted. Based on the query embedding vector, the query is matched in the local retrieval system to obtain the original text of the knowledge base.

[0181] S3 inputs user questions and original text from the knowledge base into a pre-built and trained air traffic control industry language model for fusion reasoning and generates question-and-answer results.

[0182] Based on the foregoing method and process description, in practical applications, this invention provides the following... Figure 6 The deployment main interface diagram shown is similar to... Figure 7 The diagram illustrates the knowledge base management deployment. As can be seen from the diagram, this invention not only outputs the final generated answer but also simultaneously provides the original search text and its source information, significantly improving the traceability and credibility of the results.

[0183] The advantages of implementing the intelligent question-answering method based on the air traffic control industry language model and improved RAG provided in this embodiment of the invention are as follows:

[0184] 1. Enhanced adaptability to complex command semantics: This invention introduces a language model that integrates professional knowledge from the air traffic control field, and embeds industry terminology, communication standards, and other knowledge during the pre-training and fine-tuning stages. This enables the model to have contextual semantic understanding capabilities, effectively parse diverse expression forms, and significantly improve the robustness and adaptability of command recognition.

[0185] 2. Enhance system generalization and compatibility: Adopting a dynamic knowledge retrieval mechanism and semantic context understanding strategy based on the RAG framework, the retrieval module automatically calls knowledge documents related to the current question, and the generation module outputs the answer. It can autonomously adapt to expression differences without relying on manual template updates, which greatly improves the automation level and maintenance efficiency of the system.

[0186] 3. More professional semantic understanding, with industry-consistent and controllable generated content: This invention embeds air traffic control terminology, operating procedures, and communication protocols into the model corpus and knowledge base, enabling the system to possess highly consistent professional semantic understanding capabilities. Simultaneously, domain knowledge constraints and response control mechanisms are introduced in the generation process to enhance the controllability, security, and professional credibility of the system's output results.

[0187] 4. Possesses low-latency response capability to meet real-time scenario requirements: Proposes a lightweight industry language model NavLight (with only 85M parameters), which maintains semantic understanding capability while significantly reducing inference costs and shortening response time, meeting the real-time requirements of rapid decision-making in air traffic management tasks.

[0188] 5. Improved RAG process integrating air traffic control characteristics to enhance knowledge matching accuracy and generation quality: This invention optimizes the RAG process in multiple ways, including recursive segmentation integrating air traffic control industry features, enhanced air traffic control titles, and a combination of context compression retrieval and cross-coding reordering. Multi-module collaborative optimization of the relevance and accuracy of generated results effectively improves the overall performance of the system.

[0189] 6. The system has good controllability, security and flexible configuration capabilities.

[0190] The system allows users to flexibly configure parameters such as the number of original text fragments retrieved (Top-k) and the context memory length of the language model, enhancing the model's adaptability to different scenarios. Simultaneously, this invention provides comprehensive knowledge base management functions, supporting the creation, modification, deletion, access control, and annotation management of knowledge. It also implements access control based on permissions during the retrieval process, ensuring the security and accuracy of data usage. These functions far surpass the current technical limitations of traditional RAG systems, which suffer from uncontrollable knowledge, unmanageable permissions, and inflexible configuration.

[0191] Based on the same inventive concept, embodiments of the present invention provide an intelligent question-answering system based on an air traffic control industry language model and an improved RAG, comprising:

[0192] The model training unit is used to construct and train the air traffic control industry language model; the air traffic control industry language model includes LayerNorm normalization, GELU activation function and rotational position embedding.

[0193] The retrieval and query unit is used to obtain user questions, input the user questions into the improved RAG algorithm for processing, and obtain the original text of the knowledge base; wherein, the improved RAG algorithm includes a combination strategy of integrating the recursive segmentation mechanism of the air traffic control industry, the air traffic control title enhancement mechanism, and the context-aware compression retrieval combined with cross-coding reordering.

[0194] The fusion reasoning unit is used to input the user question and the original text of the knowledge base into a pre-built and trained air traffic control industry language model for fusion reasoning and to generate question-and-answer results.

[0195] In specific implementation, the model training unit is used for:

[0196] Define the structural parameters of the language model for the air traffic control industry; the structural parameters include the number of parameters, vocabulary size, number of Tranformer layers, model size, hidden layer dimension, number of attention heads, and feedforward neural network dimension;

[0197] LayerNorm normalization, GELU activation function, and rotational position embedding are introduced;

[0198] Acquire text content related to the field of air traffic management and combine it with knowledge distillation to assist in model pre-training;

[0199] Acquire dialogue data and use the dialogue data to perform single-round dialogue fine-tuning on the air traffic control industry language model after the model pre-training is completed;

[0200] Obtain a corpus containing consecutive dialogue rounds, and use the corpus to fine-tune the language model for the air traffic control industry through multi-turn dialogue.

[0201] Obtain dialogue instances and use these instances to perform direct preference optimization on the air traffic control industry language model; the dialogue instances include exchanges between experts and real communication records between pilots and ground control centers.

[0202] In practice, the user question is input into the improved RAG algorithm for processing to obtain the original knowledge base text, specifically:

[0203] Knowledge documents related to the air traffic control industry are obtained, and the knowledge documents are preprocessed by combining a recursive segmentation mechanism and an air traffic control title enhancement mechanism.

[0204] The preprocessed knowledge document is vectorized to obtain embedded vectors that the model can understand, and the embedded vectors are stored in the local retrieval system;

[0205] Receive user questions input in natural language form and vectorize the user questions into query embedding vectors that the model can understand;

[0206] A combined strategy of context-aware compressed retrieval and cross-coding reordering is adopted. Based on the query embedding vector, the query is matched in the local retrieval system to obtain the original text of the knowledge base.

[0207] In practice, the process of integrating the recursive segmentation mechanism of the air traffic control industry is as follows:

[0208] The air traffic control document is obtained and recursively segmented into multiple levels using a predefined list of air traffic control industry delimiters and general delimiters. The air traffic control industry delimiters include message field headers, flight plan segment markers, meteorological message markers, NOTAM markers, and radar conflict event markers.

[0209] The text is traversed using the air traffic control industry delimiter list to find the position that can be used to split the document, and the text is initially segmented based on the position to obtain multiple text blocks;

[0210] Iterate through all text blocks, merge smaller text blocks, and subdivide larger text blocks using a custom second-level delimiter.

[0211] In practice, the air traffic control title enhancement mechanism works as follows:

[0212] The segmented text blocks are scored using multiple scoring reference factors to determine the title. These factors include whether the text blocks contain commonly used keywords in air traffic control industry titles, whether they contain numbers, whether there are line breaks before and after the text, whether the paragraph indentation is 0, and whether they contain punctuation marks.

[0213] The determined title is appended as a prefix to the corresponding subsequent body paragraphs.

[0214] In practice, the combined strategy of context-aware compressed retrieval and cross-coding reordering is as follows:

[0215] Based on the initial query vector q, a preliminary set of relevant documents D is obtained from the vector database;

[0216] The sub-paragraphs within each document di in the set of related documents D are scored using a local semantic similarity function;

[0217] Sub-paragraphs with scores greater than the threshold are retained to form a candidate document set D';

[0218] The candidate document set D' is subjected to deep semantic matching and re-ranking using a re-ranking model, and the most relevant top-k results are finally returned.

[0219] It should be noted that the specific workflow of this embodiment is described in the foregoing method embodiment section, and will not be repeated here.

[0220] Optionally, another embodiment of the present invention also provides an intelligent question-answering system based on an air traffic control industry language model and an improved RAG, such as... Figure 8 As shown, the system may include one or more processors 101, one or more input devices 102, one or more output devices 103, and a memory 104. The processors 101, input devices 102, output devices 103, and memory 104 are interconnected via a bus 105. The memory 104 stores a computer program, which includes program instructions. The processor 101 is configured to invoke the program instructions to execute the method described in the above-described method embodiment.

[0221] It should be understood that, in this embodiment of the invention, the processor 101 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0222] Input device 102 may include a keyboard, etc., and output device 103 may include a display (LCD, etc.), a speaker, etc.

[0223] The memory 104 may include read-only memory and random access memory, and provides instructions and data to the processor 101. A portion of the memory 104 may also include non-volatile random access memory. For example, the memory 104 may also store device type information.

[0224] In specific implementations, the processor 101, input device 102, and output device 103 described in the embodiments of the present invention can execute the implementation methods described in the embodiments of the intelligent question-answering method based on the air traffic control industry language model and improved RAG provided in the embodiments of the present invention, which will not be repeated here.

[0225] Accordingly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, implement the above-described intelligent question-answering method based on the air traffic control industry language model and improved RAG.

[0226] The computer-readable storage medium can be an internal storage unit of the system described in any of the foregoing embodiments, such as the system's hard disk or memory. The computer-readable storage medium can also be an external storage device of the system, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the system. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0227] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0228] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0229] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0230] Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units. When using each module, user information is collected and stored only with the user's full authorization and in compliance with relevant laws and regulations, protecting the security and privacy of user data, and strictly prohibiting unauthorized access; data processing will be conducted within the scope stipulated by law and will not exceed the purpose and scope authorized by the user; at the same time, users have the rights to access, correct, delete, restrict processing, and refuse their personal data; and must strictly comply with applicable laws and regulations and conduct compliance reviews.

[0231] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0232] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be defined by the scope of the claims.

Claims

1. An intelligent question-answering method based on an air traffic control industry language model and an improved RAG, characterized in that, include: Obtain the user question, input the user question into the improved RAG algorithm for processing, and obtain the original text of the knowledge base; The user question and the original text of the knowledge base are input into a pre-built and trained air traffic control industry language model for fusion reasoning to generate question-and-answer results; The improved RAG algorithm includes a combination strategy that integrates the recursive segmentation mechanism of the air traffic control industry, the air traffic control title enhancement mechanism, and the context-aware compression retrieval combined with cross-coding reordering; the air traffic control industry language model includes LayerNorm normalization, GELU activation function, and rotation position embedding.

2. The intelligent question-answering method as described in claim 1, characterized in that, The user question is input into the improved RAG algorithm for processing to obtain the original knowledge base text, specifically: Knowledge documents related to the air traffic control industry are obtained, and the knowledge documents are preprocessed by combining a recursive segmentation mechanism and an air traffic control title enhancement mechanism. The preprocessed knowledge document is vectorized to obtain embedded vectors that the model can understand, and the embedded vectors are stored in the local retrieval system; Receive user questions input in natural language form and vectorize the user questions into query embedding vectors that the model can understand; A combined strategy of context-aware compressed retrieval and cross-coding reordering is adopted. Based on the query embedding vector, the query is matched in the local retrieval system to obtain the original text of the knowledge base.

3. The intelligent question-answering method as described in claim 2, characterized in that, The specific process for integrating the recursive segmentation mechanism of the air traffic control industry is as follows: The air traffic control document is obtained, and the document is initially segmented using a predefined air traffic control industry delimiter to obtain multiple text blocks. The air traffic control industry delimiter includes a message field header, a flight plan segment marker, a meteorological message marker, a NOTAM marker, and a radar conflict event marker. Iterate through all text blocks, merge the text blocks that are too short, and obtain text blocks that meet the length requirements and have relatively complete semantics; For excessively long text blocks, a custom universal delimiter is used to recursively subdivide them, resulting in text blocks that meet the length requirements and have relatively complete semantics.

4. The intelligent question-answering method as described in claim 2, characterized in that, The specific process of the air traffic control title enhancement mechanism is as follows: For the segmented text blocks that meet the length requirements and are relatively semantically complete, multiple scoring reference factors are used to score them in order to determine the title. The multiple scoring reference factors include whether it contains commonly used keywords in air traffic control industry titles, whether it contains numbers, whether there are line breaks before and after it, whether the paragraph indentation is 0, and whether it contains punctuation marks. The determined title is appended as a prefix to the corresponding subsequent body paragraphs.

5. The intelligent question-answering method as described in claim 2, characterized in that, The specific strategy of combining context-aware compressed retrieval with cross-coding re-ranking is as follows: Based on the initial query vector q, a preliminary set of relevant documents D is obtained from the vector database; The sub-paragraphs within each document di in the set of related documents D are scored using a local semantic similarity function; Sub-paragraphs with scores greater than the threshold are retained to form a candidate document set D'; The candidate document set D' is subjected to deep semantic matching and re-ranking using a re-ranking model, and the most relevant top-k results are finally returned.

6. The intelligent question-answering method as described in claim 1, characterized in that, The method also includes constructing and training an air traffic control industry language model, specifically: Define the structural parameters of the language model for the air traffic control industry; the structural parameters include the number of parameters, vocabulary size, number of Tranformer layers, model size, hidden layer dimension, number of attention heads, and feedforward neural network dimension; LayerNorm normalization, GELU activation function, and rotational position embedding are introduced; Acquire text content related to the field of air traffic management and combine it with knowledge distillation to assist in model pre-training; Acquire dialogue data and use the dialogue data to perform single-round dialogue fine-tuning on the air traffic control industry language model after the model pre-training is completed; Obtain a corpus containing consecutive dialogue rounds, and use the corpus to fine-tune the language model for the air traffic control industry through multi-turn dialogue. Obtain dialogue instances and use these instances to perform direct preference optimization on the air traffic control industry language model; the dialogue instances include exchanges between experts and real communication records between pilots and ground control centers.

7. An intelligent question-answering system based on an air traffic control industry language model and an improved RAG, characterized in that, The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to perform the method as described in any one of claims 1-6.

8. An intelligent question-answering system based on an air traffic control industry language model and an improved RAG, characterized in that, include: The retrieval and query unit is used to obtain user questions, input the user questions into the improved RAG algorithm for processing, and obtain the original text of the knowledge base; The fusion reasoning unit is used to input the user question and the original text of the knowledge base into a pre-built and trained air traffic control industry language model for fusion reasoning and to generate question-and-answer results. The improved RAG algorithm includes a combination strategy that integrates the recursive segmentation mechanism of the air traffic control industry, the air traffic control title enhancement mechanism, and the context-aware compression retrieval combined with cross-coding reordering; the air traffic control industry language model includes LayerNorm normalization, GELU activation function, and rotation position embedding.

9. The intelligent question-answering system as described in claim 8, characterized in that, The intelligent question-answering system also includes a model training unit for constructing and training a language model for the air traffic control industry, specifically: Define the structural parameters of the language model for the air traffic control industry; the structural parameters include the number of parameters, vocabulary size, number of Tranformer layers, model size, hidden layer dimension, number of attention heads, and feedforward neural network dimension; LayerNorm normalization, GELU activation function, and rotational position embedding are introduced; Acquire text content related to the field of air traffic management and combine it with knowledge distillation to assist in model pre-training; Acquire dialogue data and use the dialogue data to perform single-round dialogue fine-tuning on the air traffic control industry language model after the model pre-training is completed; Obtain a corpus containing consecutive dialogue rounds, and use the corpus to fine-tune the language model for the air traffic control industry through multi-turn dialogue. Obtain dialogue instances and use these instances to perform direct preference optimization on the air traffic control industry language model; the dialogue instances include exchanges between experts and real communication records between pilots and ground control centers.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-6.

Images