Air-ground conversation intelligent monitoring method based on multiple rounds of context completion

Abstract

The invention discloses an air-ground communication intelligent monitoring method based on multi-round context completion, which comprises the following steps of: 1, receiving an air-ground communication segment from an interphone system, and performing voice recognition; 2, performing entity extraction on the transcriptional text to form a first instruction structural body; step 3, extracting voiceprint features of the air-ground call segment, and judging the identity of a call party in combination with an entity extraction result and an air-ground call rule to form a second instruction structure body; 4, updating a historical dialogue list of the specific flight number according to the second instruction structure body; 5, when it is detected that the second instruction structural body of the new call fragment is lack of key information, historical call fragments of the new call fragment are retrieved based on the flight number, the key information lacked in the current fragment is complemented, and a third instruction structural body is generated; and step 6, carrying out real-time security verification on the third instruction structural body and the structured data of the air traffic control automation system, and outputting a verification result and an alarm signal to realize intelligent monitoring.

Description

Technical Field

[0001] This invention relates to the field of air traffic control communication technology, and in particular to an intelligent monitoring method for air-to-ground communication based on multi-round context completion. Background Technology

[0002] Air-to-ground communication, as the core medium for transmitting instructions between air traffic controllers and pilots, is directly related to aviation operational safety in its integrity and accuracy. Currently, the air traffic control field widely uses automated monitoring systems to analyze air-to-ground communication in real time, but existing technologies have the following significant drawbacks:

[0003] 1. Issue of missing information in a single round of conversation:

[0004] Due to factors such as radio interference and differences in communication habits, key information is often missing from single communication segments (e.g., controller instructions lack altitude values, or pilots omit flight numbers when repeating instructions). Traditional rule-based or single-round NLP analysis methods cannot effectively handle such scenarios, leading to misjudgments or missed reports by the monitoring system. For example, when a pilot only replies "received" without repeating the instruction content, the system loses its monitoring capability because it cannot connect to historical instructions.

[0005] 2. Disconnection of fragmented instructions:

[0006] Existing technologies rely on independent analysis of each call segment, lacking the ability to dynamically model the context of multi-turn dialogues. Especially in complex scenarios such as when controllers simultaneously direct multiple flights and alternately issue control instructions, key instruction elements (such as altitude instructions and heading adjustments) are scattered across different segments, causing a break in the information chain during the safety verification process.

[0007] 3. Insufficient reliability of caller identification:

[0008] Mainstream voiceprint recognition technologies have a high false recognition rate in noisy aviation environments, while identity determination relying solely on text rules (such as identifying airlines by flight number prefixes) is prone to errors when the instructions are complex. This leads to incorrect association of call segments with flight information, which in turn causes subsequent security verification to fail. Summary of the Invention

[0009] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for intelligent monitoring of air-to-ground communication based on multi-round context completion, which addresses the shortcomings of the existing technology.

[0010] To address the aforementioned technical problems, this invention discloses a method for intelligent monitoring of air-to-ground communication based on multi-turn context completion, comprising the following steps:

[0011] Step 1: Receive ground-to-air communication segments from the intercom system, perform speech recognition to obtain transcribed text;

[0012] Step 2: Extract entities from the transcribed text to form the first instruction structure;

[0013] Step 3: Extract the voiceprint features of the air-to-ground communication segment, combine the entity extraction results with the air-to-ground communication rules, determine the identity of the communication party, and form the second instruction structure;

[0014] Step 4: Update the historical dialogue list for the specific flight number according to the second instruction structure;

[0015] Step 5: When it is detected that the second instruction structure of a new call segment is missing key information, retrieve its historical dialogue segments based on the flight number, analyze the context using a large language model and fill in the missing key information of the current segment to generate the third instruction structure;

[0016] Step 6: Perform real-time security verification between the third instruction structure and the structured data of the air traffic control automation system, and output the verification results and alarm signals to achieve intelligent monitoring.

[0017] Step 2, which involves entity extraction from the transcribed text to form the first instruction structure, specifically includes:

[0018] Step 2-1: Process the transcribed text using a BERT-CRF model trained on air traffic control corpus, and output BOI tag sequences. Each character in the tag sequence belongs to an attribute, which includes flight number, control unit name, instruction action, action value, and others.

[0019] Step 2-2: Based on the identified entities and their types, construct the first instruction structure according to the Civil Aviation Air-Ground Communication Standards, with the following format: {transcribed text: ..., flight number: ..., control unit name: ..., instruction action: ..., action value: ...}.

[0020] Step 3 involves extracting the voiceprint features from the air-to-ground communication segment, combining the entity extraction results with the air-to-ground communication rules, determining the identity of the caller, and forming a second instruction structure, specifically including:

[0021] Step 3-1: Extract the Mel frequency cepstral coefficient feature vector of the air-to-ground communication segment using the voiceprint recognition engine as the voiceprint feature of the air-to-ground communication segment, and perform probability density estimation with the Gaussian mixture model of each controller's voiceprint in the pre-established controller voiceprint feature library.

[0022] Step 3-2: When the maximum probability density of the controller's voiceprint feature database and the input voiceprint exceeds the first threshold, it is determined that the input voiceprint comes from the controller corresponding to the maximum probability density; when the maximum probability density of the voiceprint is higher than the second threshold but lower than the first threshold, the first instruction structure is checked using the air-to-ground communication rule feature for auxiliary determination; when the air-to-ground communication segment with a voiceprint matching degree lower than the second threshold has a flight number that is not empty in the first instruction structure, it is considered that the segment comes from the captain of that flight number; if no flight number is detected, it is considered that the segment comes from the captain of the flight number associated with the previous air-to-ground communication segment.

[0023] Step 3-3: Update the first instruction structure based on the identity determination result of the caller. In particular, if the flight number in the first instruction structure is empty, add this item according to the determination result to form the second instruction structure, with the format {caller identity: ..., transcribed text: ..., flight number: ..., control unit name: ..., instruction action: ..., action value: ...}.

[0024] The auxiliary determination described in step 3-2 is as follows:

[0025] (a) If the first instruction structure is empty, it comes from the controller corresponding to the highest probability density, and the flight number that controller is commanding is the flight number associated with the previous air-to-ground communication segment.

[0026] (b) If the first instruction structure is not empty and the call text begins with the flight number, it comes from the controller corresponding to the highest probability density.

[0027] (c) If the first instruction structure is not empty and the call text does not start with the flight number, it is from the captain of that flight number.

[0028] The method for establishing the controller voiceprint feature database described in step 3-1 includes:

[0029] The system collects the air-to-ground communication voices of controllers at each position, segments the voices according to pauses, and extracts the Mel frequency cepstral coefficient feature vector of each segment. A Gaussian mixture model is established for each controller, and the probability distribution of their voiceprint feature vector is stored. The voiceprint feature database is updated regularly based on the controller's position scheduling information.

[0030] Step 4, which involves updating the historical dialogue list for a specific flight number based on the second instruction structure, specifically includes:

[0031] After step 3, the second instruction structure must contain the flight number. Locate the historical dialogue list corresponding to the flight number. If it is empty, create a new one and store the second instruction structure at the end of the historical dialogue list.

[0032] Step 5 describes the process of retrieving historical dialogue segments based on flight numbers when a new call segment's instruction structure is found to lack key information. This involves using a large language model to analyze the context and complete the missing key information for the current segment, generating a third instruction structure. Specifically, this is implemented as follows:

[0033] Step 5-1: Check the second instruction structure corresponding to the latest received call segment. The instruction action and action value are key information. If one of them does not meet the conditions, the instruction structure is considered to be missing key information. If all conditions are met, skip to step 5-7.

[0034] Step 5-2: If key information is missing, extract the flight number from the second instruction structure;

[0035] Step 5-3: Based on the flight number, retrieve the historical dialogue list of the flight from the stored flight dialogue list, which is the list of second instruction structures stored in chronological order;

[0036] Step 5-4: Extract the latest N historical second instruction structures (N is a preset positive integer, such as 2) from the transcribed text in the historical dialogue list in chronological order, and assemble the second instruction structures into a historical context sequence in chronological order;

[0037] Step 5-5: Construct large language model input prompts in Markdown format, with the following format:

[0038] #Context of Historical Ground-Air Communication

[0039] [A chronological sequence of historical second instruction structures]

[0040] #Current fragment to be completed

[0041] [Identity of the caller in the current segment]: [Transcribed text of the current segment]

[0042] #Task Requirements

[0043] Complete the missing information in the current segment based on historical context:

[0044] If the instruction action is missing, output 'Instruction action: [Complete value]'

[0045] If the action value is missing, output 'Action value: [Complete value]'.

[0046] If both are missing, output the two completed results on separate lines.

[0047] Steps 5-6: Feed the input prompts into the large language model, parse the model's output completion value, and update the instruction action and / or action value attributes of the current second instruction structure;

[0048] Steps 5-7: The current second instruction structure is the third instruction structure.

[0049] The conditions described in step 5-1 specifically include:

[0050] If this attribute is empty or its value does not comply with civil aviation regulations, the civil aviation regulations that need to be checked include:

[0051] (a) The instruction action attribute value must belong to a predefined set of control instruction actions;

[0052] (b) The numerical attribute value of the action must meet the format and range requirements corresponding to the instruction action attribute value, and the format and range requirements are preset according to the civil aviation control rules.

[0053] Step 6 specifically includes:

[0054] Step 6-1: Query the structured data related to the current flight number from the air traffic control automation system in real time. The structured data includes flight plan, real-time flight status, airspace restriction information and surrounding traffic situation.

[0055] Step 6-2: Based on the instruction actions and action values ​​in the third instruction structure, and combined with the queried structured data, apply a predefined set of security rules for verification;

[0056] Step 6-3: Generate verification results based on the verification results. If all rules pass, the verification result is "Safe"; if any rule fails, the verification result is "Risk" with information about the rule violation.

[0057] Step 6-4: When the verification result is "risk", an alarm signal is triggered. The alarm signal includes visual alarm, sound alarm and log recording, and is pushed to the monitoring interface of the controller's seat automation system in real time.

[0058] The security rule set mentioned in step 6-2 includes:

[0059] (a) Altitude rules: Check whether the altitude instructions in the action values ​​comply with the cruising altitude requirements in the flight plan and do not violate the altitude restrictions of the relevant airspace;

[0060] (b) Speed ​​rules: Check whether the speed command in the action value exceeds the aircraft performance limit or does not conform to the current airspace speed limit;

[0061] (c) Track Conflict Rule: Using the track prediction model of the air traffic control automation system, based on the command actions and action values ​​in the third command structure, the real-time flight status in the structured data and the surrounding traffic situation, predict the track of the flight in the next T minutes (T is a preset positive number, such as 10) after the execution of the command, and check whether there is a track conflict.

[0062] Beneficial effects:

[0063] 1. Solve monitoring blind spots caused by missing information in single rounds: Through a deep understanding of historical context using a large language model, missing instructions, actions, or values ​​are intelligently completed, breaking through the limitations of traditional single-round analysis.

[0064] 2. Construct a highly reliable call identity chain: Integrate a dual verification mechanism that combines voiceprint probability matching with air-to-ground call rules to effectively solve the binding error problem when multiple flights are in cross-call.

[0065] 3. Reduce controller workload: Automated information completion and risk warning mechanisms reduce the need for manual intervention. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0067] Figure 2 This is a schematic diagram of BERT-CRF entity extraction.

[0068] Figure 3 It is a flowchart for determining the identity of the caller by combining the entity extraction results with the rules for air-to-ground communication. Detailed Implementation

[0069] This invention relates to a method for intelligently monitoring, completing information integrity, and verifying real-time security of air-to-ground communication between controllers and pilots by utilizing speech recognition, voiceprint recognition, natural language processing (NLP) entity extraction, large language model (LLM) context understanding and completion technologies, combined with data from air traffic control automation systems. The aim is to solve the monitoring blind spot problem caused by incomplete information in single-round communication and improve the safety and reliability of air traffic control operations.

[0070] This invention proposes a flight call chain construction mechanism based on dual verification of voiceprint and entity, combined with multi-turn context completion technology of a large language model, to achieve intelligent monitoring of air-to-ground calls. A method for intelligent monitoring of air-to-ground calls based on multi-turn context completion is provided, the process of which is as follows: Figure 1 As shown in the figure. The technical solution of the present invention will be further described in detail below with reference to the embodiments.

[0071] A ground-to-air communication intelligent monitoring method based on multi-turn context completion includes the following steps:

[0072] Step 1: Receive ground-to-air communication segments from the intercom system, perform speech recognition to obtain transcribed text;

[0073] Step 2: Extract entities from the transcribed text to form the first instruction structure;

[0074] Step 3: Extract the voiceprint features of the air-to-ground communication segment, combine the entity extraction results with the air-to-ground communication rules, determine the identity of the communication party, and form the second instruction structure;

[0075] Step 4: Update the historical dialogue list for the specific flight number according to the second instruction structure;

[0076] Step 5: When it is detected that the second instruction structure of a new call segment is missing key information, retrieve its historical dialogue segments based on the flight number, analyze the context using a large language model and fill in the missing key information of the current segment to generate the third instruction structure;

[0077] Step 6: Perform real-time security verification between the third instruction structure and the structured data of the air traffic control automation system, and output the verification results and alarm signals to achieve intelligent monitoring.

[0078] Step 2, which involves entity extraction from the transcribed text to form the first instruction structure, specifically includes:

[0079] Step 2-1: Process the transcribed text using a BERT-CRF model trained on air traffic control corpus, and output BOI tag sequences. Each character in the tag sequence belongs to an attribute, which includes flight number, control unit name, instruction action, action value, and others.

[0080] Step 2-2: Based on the identified entities and their types, construct the first instruction structure according to the Civil Aviation Air-Ground Communication Standards, with the following format: {transcribed text: ..., flight number: ..., control unit name: ..., instruction action: ..., action value: ...}.

[0081] Step 3 involves extracting the voiceprint features from the air-to-ground communication segment, combining the entity extraction results with the air-to-ground communication rules, determining the identity of the caller, and forming a second instruction structure, specifically including:

[0082] Step 3-1: Extract the Mel frequency cepstral coefficient feature vector of the air-to-ground communication segment using the voiceprint recognition engine as the voiceprint feature of the air-to-ground communication segment, and perform probability density estimation with the Gaussian mixture model of each controller's voiceprint in the pre-established controller voiceprint feature library.

[0083] Step 3-2: When the maximum probability density of the controller's voiceprint feature database and the input voiceprint exceeds the first threshold, it is determined that the input voiceprint comes from the controller corresponding to the maximum probability density; when the maximum probability density of the voiceprint is higher than the second threshold but lower than the first threshold, the first instruction structure is checked using the air-to-ground communication rule feature for auxiliary determination; when the air-to-ground communication segment with a voiceprint matching degree lower than the second threshold has a flight number that is not empty in the first instruction structure, it is considered that the segment comes from the captain of that flight number; if no flight number is detected, it is considered that the segment comes from the captain of the flight number associated with the previous air-to-ground communication segment.

[0084] Step 3-3: Update the first instruction structure based on the identity determination result of the caller. In particular, if the flight number in the first instruction structure is empty, add this item according to the determination result to form the second instruction structure, with the format {caller identity: ..., transcribed text: ..., flight number: ..., control unit name: ..., instruction action: ..., action value: ...}.

[0085] The auxiliary determination described in step 3-2 is as follows:

[0086] (a) If the first instruction structure is empty, it comes from the controller corresponding to the highest probability density, and the flight number that controller is commanding is the flight number associated with the previous air-to-ground communication segment.

[0087] (b) If the first instruction structure is not empty and the call text begins with the flight number, it comes from the controller corresponding to the highest probability density.

[0088] (c) If the first instruction structure is not empty and the call text does not start with the flight number, it is from the captain of that flight number.

[0089] The method for establishing the controller voiceprint feature database described in step 3-1 includes:

[0090] The system collects the air-to-ground communication voices of controllers at each position, segments the voices according to pauses, and extracts the Mel frequency cepstral coefficient feature vector of each segment. A Gaussian mixture model is established for each controller, and the probability distribution of their voiceprint feature vector is stored. The voiceprint feature database is updated regularly based on the controller's position scheduling information.

[0091] Step 4, which involves updating the historical dialogue list for a specific flight number based on the second instruction structure, specifically includes:

[0092] After step 3, the second instruction structure must contain the flight number. Locate the historical dialogue list corresponding to the flight number. If it is empty, create a new one and store the second instruction structure at the end of the historical dialogue list.

[0093] Step 5 describes the process of retrieving historical dialogue segments based on flight numbers when a new call segment's instruction structure is found to lack key information. This involves using a large language model to analyze the context and complete the missing key information for the current segment, generating a third instruction structure. Specifically, this is implemented as follows:

[0094] Step 5-1: Check the second instruction structure corresponding to the latest received call segment. The instruction action and action value are key information. If one of them does not meet the conditions, the instruction structure is considered to be missing key information. If all conditions are met, skip to step 5-7.

[0095] Step 5-2: If key information is missing, extract the flight number from the second instruction structure;

[0096] Step 5-3: Based on the flight number, retrieve the historical dialogue list of the flight from the stored flight dialogue list, which is the list of second instruction structures stored in chronological order;

[0097] Step 5-4: Extract the latest N historical second instruction structures (N is a preset positive integer, such as 2) from the transcribed text in the historical dialogue list in chronological order, and assemble the second instruction structures into a historical context sequence in chronological order;

[0098] Step 5-5: Construct large language model input prompts in Markdown format, with the following format:

[0099] #Context of Historical Ground-Air Communication

[0100] [A chronological sequence of historical second instruction structures]

[0101] #Current fragment to be completed

[0102] [Identity of the caller in the current segment]: [Transcribed text of the current segment]

[0103] #Task Requirements

[0104] Complete the missing information in the current segment based on historical context:

[0105] If the instruction action is missing, output 'Instruction action: [Complete value]'

[0106] If the action value is missing, output 'Action value: [Complete value]'.

[0107] If both are missing, output the two completed results on separate lines.

[0108] Steps 5-6: Feed the input prompts into the large language model, parse the model's output completion value, and update the instruction action and / or action value attributes of the current second instruction structure;

[0109] Steps 5-7: The current second instruction structure is the third instruction structure.

[0110] The conditions described in step 5-1 specifically include:

[0111] If this attribute is empty or its value does not comply with civil aviation regulations, the civil aviation regulations that need to be checked include:

[0112] (a) The instruction action attribute value must belong to a predefined set of control instruction actions;

[0113] (b) The numerical attribute value of the action must meet the format and range requirements corresponding to the instruction action attribute value, and the format and range requirements are preset according to the civil aviation control rules.

[0114] Step 6 specifically includes:

[0115] Step 6-1: Query the structured data related to the current flight number from the air traffic control automation system in real time. The structured data includes flight plan, real-time flight status, airspace restriction information and surrounding traffic situation.

[0116] Step 6-2: Based on the instruction actions and action values ​​in the third instruction structure, and combined with the queried structured data, apply a predefined set of security rules for verification;

[0117] Step 6-3: Generate verification results based on the verification results. If all rules pass, the verification result is "Safe"; if any rule fails, the verification result is "Risk" with information about the rule violation.

[0118] Step 6-4: When the verification result is "risk", an alarm signal is triggered. The alarm signal includes visual alarm, sound alarm and log recording, and is pushed to the monitoring interface of the controller's seat automation system in real time.

[0119] The security rule set mentioned in step 6-2 includes:

[0120] (a) Altitude rules: Check whether the altitude instructions in the action values ​​comply with the cruising altitude requirements in the flight plan and do not violate the altitude restrictions of the relevant airspace;

[0121] (b) Speed ​​rules: Check whether the speed command in the action value exceeds the aircraft performance limit or does not conform to the current airspace speed limit;

[0122] (c) Track Conflict Rule: Using the track prediction model of the air traffic control automation system, based on the command actions and action values ​​in the third command structure, the real-time flight status in the structured data and the surrounding traffic situation, predict the track of the flight in the next T minutes (T is a preset positive number, such as 10) after the execution of the command, and check whether there is a track conflict.

[0123] Example 1:

[0124] Taking the intelligent monitoring system for air traffic control instructions in a certain air traffic control automation system as an example, the system input is the air-to-ground communication segments input according to the reception time, and each segment is divided according to the communication interval between the controller and the captain.

[0125] Step 1: Receive ground-to-air communication segments from the intercom system and perform speech recognition;

[0126] In this embodiment, three voice messages were received in sequence: “Shanghai District Administration Dongfang 3984 applied to be transferred to four liang”, “Dongfang 3984 agreed to transfer to four liang and maintain”, and “Received Dongfang 3984”. Among them, “Received Dongfang 3984” was the last voice message received.

[0127] Step 2: Extract entities from the transcribed text to form the first instruction structure;

[0128] Step 2, which describes entity extraction from the transcribed text, includes the following steps:

[0129] Step 2-1: Process the transcribed text using a BERT-CRF model trained on air traffic control corpus (reference paper: Souza F, Nogueira R, Lotufo R. Portuguese named entity recognition using BERT-CRF[J]. arXiv preprint arXiv:1909.10649,2019.), outputting BOI marker sequences, such as... Figure 2 As shown, mark which of the five attributes each character belongs to: flight number, control unit name, instruction action, action value, and others.

[0130] Step 2-2: Based on the identified entities and their types, construct the first instruction structure according to the Civil Aviation Air-Ground Communication Standards, with the following format: {transcribed text: ..., flight number: ..., control unit name: ..., instruction action: ..., action value: ...}. In this embodiment, the first instruction structure is {transcribed text: "Received from East 3984", flight number: "CES3984", control unit name: "", instruction action: "", action value: ""}

[0131] Step 3: Extract the voiceprint features from the air-to-ground communication segment, combine the entity extraction results with the air-to-ground communication rules, determine the identity of the caller, and form a second instruction structure, such as... Figure 3 As shown;

[0132] In this embodiment, because the maximum probability density is less than 0.5 and the flight number of the first instruction structure is not empty, the second instruction structure is formed as follows: {Calling party identity: "Captain", Transcribed text: "Received from East 3984", Flight number: "CES3984", Control unit name: "", Instruction action: "", Action value: ""}

[0133] Step 3 specifically includes:

[0134] Step 3-1: Extract the Mel frequency cepstral coefficient feature vector of the air-to-ground communication segment using the voiceprint recognition engine as the voiceprint feature of the air-to-ground communication segment, and perform probability density estimation with the Gaussian mixture model of each controller's voiceprint in the pre-established controller voiceprint feature library.

[0135] The method for establishing the controller voiceprint feature database described in step 3-1 includes:

[0136] The system collects the air-to-ground communication voices of controllers at each position, segments the voices according to pauses, and extracts the Mel frequency cepstral coefficient feature vector of each segment. A Gaussian mixture model is established for each controller, and the probability distribution of their voiceprint feature vector is stored. The voiceprint feature database is updated regularly based on the controller's position scheduling information.

[0137] Step 3-2: When the maximum probability density of the controller's voiceprint feature database and the input voiceprint exceeds the first threshold (in this embodiment, the first threshold is 0.8), it is determined that the input voiceprint comes from the controller corresponding to the maximum probability density. When the maximum probability density of the voiceprint is higher than the second threshold but lower than the first threshold (in this embodiment, the second threshold is 0.5), the following air-to-ground communication rule feature check is used to assist in the determination of the first instruction structure:

[0138] (a) If the first instruction structure is empty, it comes from the controller corresponding to the highest probability density, and the flight number that controller is commanding is the flight number associated with the previous air-to-ground communication segment.

[0139] (b) If the first instruction structure is not empty and the call text begins with the flight number, it comes from the controller corresponding to the highest probability density.

[0140] (c) If the first instruction structure is not empty and the call text does not start with the flight number, it is from the captain of that flight number.

[0141] Step 3-3: For air-to-ground communication segments with a probability density maximum value of less than 0.5, if the flight number in the first instruction structure is not empty, it is assumed that the segment comes from the captain of the flight number. If no flight number is detected, it is assumed that the segment comes from the captain of the flight number associated with the previous air-to-ground communication segment.

[0142] Steps 3-4: Update the first instruction structure based on the identity determination result of the caller. In particular, if the flight number in the first instruction structure is empty, add this item according to the determination result to form the second instruction structure, with the format {caller identity: ..., transcribed text: ..., flight number: ..., control unit name: ..., instruction action: ..., action value: ...}.

[0143] Step 4: Update the historical dialogue list for the specific flight number according to the second instruction structure;

[0144] Step 4 specifically includes:

[0145] After step 3, the second instruction structure must contain the flight number. Locate the historical dialogue list corresponding to the flight number. If it is empty, create a new one and store the second instruction structure at the end of the historical dialogue list.

[0146] Step 5: When it is detected that the second instruction structure of a new call segment is missing key information, retrieve its historical dialogue segments based on the flight number, analyze the context using a large language model and fill in the missing key information of the current segment to generate the third instruction structure;

[0147] Step 5 specifically includes:

[0148] Step 5-1: Check the second instruction structure corresponding to the latest received call segment. The instruction action and action value are key information. If one of them does not meet the conditions, the instruction structure is considered to be missing key information. If all conditions are met, skip to step 5-7.

[0149] Step 5-2: If key information is missing, extract the flight number from the second instruction structure;

[0150] Step 5-3: Based on the flight number, retrieve the historical dialogue list of the flight from the stored flight dialogue list, which is the list of second instruction structures stored in chronological order;

[0151] Step 5-4: Extract the latest N historical second instruction structures (N is a preset positive integer, such as 2) from the transcribed text in the historical dialogue list in chronological order, and assemble the second instruction structures into a historical context sequence in chronological order;

[0152] Step 5-5: Construct large language model input prompts in Markdown format. In this embodiment, this means:

[0153] #Context of Historical Ground-Air Communication

[0154] {Calling party's identity: "Captain", Transcript: "Shanghai Area Control, Dongfang 3984, requesting to descend to Siliang", Flight number: "CES3984", Air traffic control unit: "Shanghai Area Control", Command action: "Change altitude", Action value: "4200 meters"}

[0155] {Calling party identity: "Air Traffic Controller", Transcript: "Eastern 3984 agrees to descend to 400 meters and maintain", Flight number: "CES3984", Air Traffic Control Unit: "", Command action: "Change altitude", Action value: "4200 meters"}

[0156] #Current fragment to be completed

[0157] Captain: Received from Dongfang 3984

[0158] #Task Requirements

[0159] Complete the missing information in the current segment based on historical context:

[0160] If the instruction action is missing, output 'Instruction action: [Complete value]'

[0161] If the action value is missing, output 'Action value: [Complete value]'.

[0162] If both are missing, output the two completed results on separate lines.

[0163] Steps 5-6: Feed the input prompts into the large language model, parse the model's output completion value, and update the instruction action and / or action value attributes of the current second instruction structure;

[0164] For this embodiment, the output result of the large model is:

[0165] Command action: "Change height"

[0166] Action stat: "4200 meters"

[0167] Therefore, the second instruction structure is updated to {caller identity: "Captain", transcribed text: "Received from Dongfang 3984", flight number: "CES3984", air traffic control unit name: "", instruction action: "Change altitude", action value: "4200 meters"}

[0168] Steps 5-7: The current second instruction structure is the third instruction structure, namely {caller identity: "captain", transcribed text: "received from East 3984", flight number: "CES3984", control unit name: "", instruction action: "change altitude", action value: "4200 meters"}.

[0169] Step 6: Perform real-time security verification between the third instruction structure and the structured data of the air traffic control automation system, and output the verification results and alarm signals.

[0170] In this embodiment, the verification result is "safe".

[0171] Step 6 specifically includes:

[0172] Step 6-1: Query the structured data related to the current flight number from the air traffic control automation system in real time. The structured data includes flight plan, real-time flight status, airspace restriction information and surrounding traffic situation.

[0173] Step 6-2: Based on the instruction actions and action values ​​in the third instruction structure, and combined with the retrieved structured data, apply a predefined set of security rules for verification. The security rule set includes:

[0174] (a) Altitude rules: Check whether the altitude command in the action value meets the cruise altitude requirements in the flight plan and does not violate the altitude restrictions of the relevant airspace. In this example, since it is in the landing phase, there is no need to check the cruise altitude requirement, and there are no altitude restrictions in Shanghai sector 05.

[0175] (b) Speed ​​rule: Check whether the speed command in the action value exceeds the aircraft performance limit or does not match the current airspace speed limit. For this example, since the speed has not been modified, it is not necessary to check this rule.

[0176] (c) Track conflict rule: Using the track prediction model of the air traffic control automation system, based on the command actions and action values ​​in the third command structure, the real-time flight status in the structured data and the surrounding traffic situation, predict the track of the flight in the next T minutes (T is a preset positive number, such as 10) after the execution of the command, and check whether there is a track conflict. In this example, no track conflict was found after the check.

[0177] Step 6-3: Generate verification results based on the verification results. If all rules pass, the verification result is "Safe"; if any rule fails, the verification result is "Risk" with information about the rule violation.

[0178] Step 6-4: When the verification result is "risk", an alarm signal is triggered. The alarm signal includes visual alarm, sound alarm and log recording, and is pushed to the monitoring interface of the controller's seat automation system in real time.

[0179] Example 2:

[0180] This invention provides a method for intelligent monitoring of air-to-ground communication based on multi-round context completion, enabling intelligent monitoring of air-to-ground communication.

[0181] Step 1: Receiving and Recognizing Ground-to-Air Communication Segments

[0182] The system receives real-time audio streams of air-to-ground communications between air traffic controllers and pilots via an intercom system interface. Each call is segmented into independent segments with silence intervals as boundaries. An end-to-end speech recognition model (such as Conformer-Transformer) is used to convert the audio into text, supporting mixed Chinese and English recognition.

[0183] Step 2: Entity Extraction and Generation of the First Instruction Structure

[0184] Use the pre-trained BERT-CRF model in the field of air traffic control, with the input being the transcribed text of Step 1. The model outputs character-level tags (BIO annotation), identifies five types of entities, and standardizes aviation terms (e.g., "ascend" is converted to "change altitude", "nine two" is converted to "9,200 meters"): flight number (e.g., "CES5102"), control unit (e.g., "Beijing Approach"), command action (e.g., "change altitude", "change speed"), action value (e.g., "9,000 meters", "250 knots"), others (e.g., response word "received"). According to the aggregation result of entity tags, construct the first instruction structure. For example:

[0185] {Transcribed text: "Air China five one zero two descend to seven eight",

[0186] Flight number: "CCA5102",

[0187] Name of control unit: "",

[0188] Command action: "change altitude",

[0189] Action value: "7,800 meters"}

[0190] Note: The unrecognized entity attributes are set to "", and the subsequent steps trigger the completion logic.

[0191] Step 3: Voiceprint identity binding and generation of the second instruction structure

[0192] Extract the 39-dimensional MFCC features of the call segment (frame length 25 ms, frame shift 10 ms), and compare them with the voiceprint library of air traffic controllers. The voiceprint library is constructed by Gaussian mixture model, and each air traffic controller corresponds to a 2048-component GMM, and the model parameters are updated regularly through the seat scheduling information. Calculate the probability density of the input voiceprint and the model in the library, and set the thresholds: the first threshold = 0.7 (high-confidence match), the second threshold = 0.3 (low-confidence requires rule assistance).

[0193] If LLR > 0.7, directly bind the corresponding air traffic controller; if 0.3 < LLR ≤ 0.7, check the first instruction structure:

[0194] (a) If the flight number is empty, default to binding the flight number controller of the nearest segment;

[0195] (b) Whether the first word of the segment is the flight number (e.g., starting with "CA5102" → judged as the air traffic controller, otherwise judged as the captain);

[0196] If LLR ≤ 0.3 and there is no flight number, default to binding the captain of the flight number of the nearest segment. <000​​

[0198] Step 4: Save the conversations to the flight-specific chat list by timestamp.

[0199] Step 5: Contextual Completion for Large Language Model

[0200] Triggering conditions: Completion is initiated when the instruction action or action value of the second instruction structure is "", or when civil aviation rules are violated (such as the cruise altitude not belonging to the standard flight altitude layer).

[0201] Retrieve the latest 3 historical segments of the current flight (second instruction structure list), for example: {caller identity: "controller", transcribed text: "Air China Wuyaodong 2nd descent to Guai 8", flight number: "CA5102", control unit name: "Beijing Area", instruction action: "change altitude", action value: "7800 meters"}; {caller identity: "controller", transcribed text: "Air China Wuyaodong 2nd descent to Guai 8", flight number: "CA5102", control unit name: "", instruction action: "change altitude"} {Altitude", Action Value: "7800 meters"}; {Conversational Party Identity: "Captain", Transcript: "Descend to 8th Airway 5102", Flight Number: "CA5102", Air Traffic Control Unit Name: "", Instruction Action: "Change Altitude", Action Value: "7800 meters"}; {Conversational Party Identity: "Air Traffic Controller", Transcript: "Contact Beijing Approach 119", Flight Number: "CA5102", Air Traffic Control Unit Name: "", Instruction Action: "Contact Frequency", Action Value: "119.7"}

[0202] Hints for constructing a large model:

[0203] #Context of Historical Ground-Air Communication

[0204] {Calling party identity: "Air Traffic Controller", Transcript: "Air China descends from Wuyaodong 2nd to Guai 8th", Flight number: "CA5102", Air Traffic Control Unit: "", Command action: "Change altitude", Action value: "7800 meters"}

[0205] {Calling party identity: "Captain", Transcript: "Descend to the 8th National Airlines 5102", Flight number: "CA5102", Air traffic control unit: "", Command action: "Change altitude", Action value: "7800 meters"}

[0206] {Calling party identity: "Air Traffic Controller", Transcript: "Contact Beijing Approach 119:00", Flight number: "CA5102", Control unit name: "", Instruction action: "Contact frequency", Action value: "119.7"}

[0207] #Current fragment to be completed

[0208] Captain: Received two reports from Air China's 510.

[0209] #Task Requirements

[0210] Complete missing information: If the command action is missing, output 'Command Action: [Complete Value]'; if the action value is missing, output 'Action Value: [Complete Value]'.

[0211] Call a large language model (such as DeepSeek R1) and parse the output results (e.g., command action: "contact frequency", value: "119.7");

[0212] Update the current structure to generate a third instruction structure (e.g., {caller identity: "captain", transcribed text: "Air China 5102", flight number: "CA5102", control unit name: "", instruction action: "contact frequency", action value: "119.7"}).

[0213] Step 5: Real-time security verification and alerts.

[0214] This invention provides an intelligent monitoring method for air-to-ground communication based on multi-turn context completion. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for intelligent monitoring of air-to-ground communication based on multi-turn context completion, characterized in that, Includes the following steps: Step 1: Receive ground-to-air communication segments from the intercom system, perform speech recognition to obtain transcribed text; Step 2: Extract entities from the transcribed text to form the first instruction structure; Step 3: Extract the voiceprint features of the air-to-ground communication segment, combine the entity extraction results with the air-to-ground communication rules, determine the identity of the communication party, and form the second instruction structure; Step 4: Update the historical dialogue list for the specific flight number according to the second instruction structure; Step 5: When it is detected that the second instruction structure of a new call segment is missing key information, retrieve its historical dialogue segments based on the flight number, analyze the context using a large language model and fill in the missing key information of the current segment to generate the third instruction structure; Step 6: Perform real-time security verification between the third instruction structure and the structured data of the air traffic control automation system, and output the verification results and alarm signals to achieve intelligent monitoring.

2. The intelligent monitoring method for air-to-ground communication based on multi-turn context completion according to claim 1, characterized in that, Step 2, which involves entity extraction from the transcribed text to form the first instruction structure, specifically includes: Step 2-1: Process the transcribed text using a BERT-CRF model trained on air traffic control corpus, and output BOI tag sequences. Each character in the tag sequence belongs to an attribute, which includes flight number, control unit name, instruction action, action value, and others. Step 2-2: Based on the identified entities and their types, construct the first instruction structure according to the Civil Aviation Air-Ground Communication Standards, with the following format: {transcribed text: ..., flight number: ..., control unit name: ..., instruction action: ..., action value: ...}.

3. The intelligent monitoring method for air-to-ground communication based on multi-turn context completion according to claim 1, characterized in that, Step 3 involves extracting the voiceprint features from the air-to-ground communication segment, combining the entity extraction results with the air-to-ground communication rules, determining the identity of the caller, and forming a second instruction structure, specifically including: Step 3-1: Extract the Mel frequency cepstral coefficient feature vector of the air-to-ground communication segment using the voiceprint recognition engine as the voiceprint feature of the air-to-ground communication segment, and perform probability density estimation with the Gaussian mixture model of each controller's voiceprint in the pre-established controller voiceprint feature library. Step 3-2: When the maximum probability density of the controller's voiceprint feature database and the input voiceprint exceeds the first threshold, it is determined that the input voiceprint comes from the controller corresponding to the maximum probability density; when the maximum probability density of the voiceprint is higher than the second threshold but lower than the first threshold, the first instruction structure is checked using the air-to-ground communication rule feature for auxiliary determination; when the air-to-ground communication segment with a voiceprint matching degree lower than the second threshold has a flight number that is not empty in the first instruction structure, it is considered that the segment comes from the captain of that flight number; if no flight number is detected, it is considered that the segment comes from the captain of the flight number associated with the previous air-to-ground communication segment. Step 3-3: Update the first instruction structure based on the identity determination result of the caller. In particular, if the flight number in the first instruction structure is empty, add this item according to the determination result to form the second instruction structure, with the format {caller identity: ..., transcribed text: ..., flight number: ..., control unit name: ..., instruction action: ..., action value: ...}.

4. The intelligent monitoring method for air-to-ground communication based on multi-turn context completion according to claim 3, characterized in that, The auxiliary determination described in step 3-2 is as follows: (a) If the first instruction structure is empty, it comes from the controller corresponding to the highest probability density, and the flight number that controller is commanding is the flight number associated with the previous air-to-ground communication segment. (b) If the first instruction structure is not empty and the call text begins with the flight number, it comes from the controller corresponding to the highest probability density. (c) If the first instruction structure is not empty and the call text does not start with the flight number, it is from the captain of that flight number.

5. The intelligent monitoring method for air-to-ground communication based on multi-turn context completion according to claim 4, characterized in that, The method for establishing the controller voiceprint feature database described in step 3-1 includes: The system collects the air-to-ground communication voices of controllers at each position, segments the voices according to pauses, and extracts the Mel frequency cepstral coefficient feature vector of each segment. A Gaussian mixture model is established for each controller, and the probability distribution of their voiceprint feature vector is stored. The voiceprint feature database is updated regularly based on the controller's position scheduling information.

6. The intelligent monitoring method for air-to-ground communication based on multi-turn context completion according to claim 1, characterized in that, The step 4, which involves updating the historical dialogue list of a specific flight number based on the second instruction structure, specifically includes: finding the historical dialogue list corresponding to the flight number; if it is empty, creating a new one; and storing the second instruction structure at the end of the historical dialogue list.

7. The intelligent monitoring method for air-to-ground communication based on multi-turn context completion according to claim 1, characterized in that, Step 5 describes the process of retrieving historical dialogue segments based on flight numbers when a new call segment's instruction structure is found to lack key information. This involves using a large language model to analyze the context and complete the missing key information for the current segment, generating a third instruction structure. Specifically, this is implemented as follows: Step 5-1: Check the second instruction structure corresponding to the latest received call segment. The instruction action and action value are key information. If one of them does not meet the conditions, the instruction structure is considered to be missing key information. If all conditions are met, skip to step 5-7. Step 5-2: If key information is missing, extract the flight number from the second instruction structure; Step 5-3: Based on the flight number, retrieve the historical dialogue list of the flight from the stored flight dialogue list, which is the list of second instruction structures stored in chronological order; Step 5-4: Extract the latest N historical second instruction structures from the transcribed text in the historical dialogue list in chronological order, and assemble the second instruction structures into a historical context sequence in chronological order; Step 5-5: Construct input prompts for a large language model in Markdown format; Steps 5-6: Feed the input prompts into the large language model, parse the model's output completion value, and update the instruction action and / or action value attributes of the current second instruction structure; Steps 5-7: The current second instruction structure is the third instruction structure.

8. The intelligent monitoring method for air-to-ground communication based on multi-turn context completion according to claim 7, characterized in that, The conditions described in step 5-1 specifically include: If this attribute is empty or its value does not comply with civil aviation regulations, the civil aviation regulations that need to be checked include: (a) The instruction action attribute value must belong to a predefined set of control instruction actions; (b) The numerical attribute value of the action must meet the format and range requirements corresponding to the instruction action attribute value, and the format and range requirements are preset according to the civil aviation control rules.

9. The intelligent monitoring method for air-to-ground communication based on multi-turn context completion according to claim 1, characterized in that, Step 6 specifically includes: Step 6-1: Query the structured data related to the current flight number from the air traffic control automation system in real time. The structured data includes flight plan, real-time flight status, airspace restriction information and surrounding traffic situation. Step 6-2: Based on the instruction actions and action values ​​in the third instruction structure, and combined with the queried structured data, apply a predefined set of security rules for verification; Step 6-3: Generate a verification result based on the verification results. If all rules pass, the verification result is "Safe"; if any rule fails, the verification result is "Risk" with information about the rule violation. Step 6-4: When the verification result is "risk", an alarm signal is triggered. The alarm signal includes visual alarm, sound alarm and log recording, and is pushed to the monitoring interface of the controller's seat automation system in real time.

10. A method for intelligent monitoring of air-to-ground communication based on multi-turn context completion according to claim 8, characterized in that, The security rule set mentioned in step 6-2 includes: (a) Altitude rules: Check whether the altitude instructions in the action values ​​comply with the cruising altitude requirements in the flight plan and do not violate the altitude restrictions of the relevant airspace; (b) Speed ​​rules: Check whether the speed command in the action value exceeds the aircraft performance limit or does not conform to the current airspace speed limit; (c) Track Conflict Rule: Using the track prediction model of the air traffic control automation system, based on the command actions and action values ​​in the third command structure, the real-time flight status in the structured data and the surrounding traffic situation, predict the track of the flight in the next T minutes after the execution of the command, and check whether there is a track conflict.

Images