Voice instruction intelligent analysis method and system applied to air traffic control scene

Abstract

This invention provides a method and system for intelligent parsing of voice commands in air traffic control scenarios, relating to the field of air traffic control technology. Targeting the characteristics of voice command interaction between controllers and crew members in air traffic control scenarios, it constructs a command interaction expectation model. This model includes command sequence association rules and a set of semantic constraints generated based on historical control call data. It captures in real time the speaker identity features, command component temporal features, and voice tone fluctuation features during the voice interaction process, inputting these features into the command interaction expectation model for dynamic matching and deviation identification, generating interaction deviation analysis results. Based on the analysis results, it triggers intent clarification and command confirmation processes to obtain a complete set of command components. After structured transformation processing, it generates command parsing results that conform to air traffic control data exchange standards and pushes them to the air traffic control automation system, effectively improving the accuracy and efficiency of air traffic control voice command parsing.

Description

Technical Field

[0001] This invention relates to the field of air traffic control technology, and more specifically, to a method and system for intelligent parsing of voice commands applied in air traffic control scenarios. Background Technology

[0002] In air traffic control, the voice command interaction between controllers and flight crews is a crucial link in ensuring flight safety and operational order. In traditional air traffic control scenarios, voice command interpretation primarily relies on manual methods, requiring controllers to understand and interpret the voice content based on their experience. However, with the continuous growth of air traffic volume and the significant increase in air traffic control communication traffic, manual interpretation faces numerous challenges. On the one hand, manual interpretation is susceptible to the subjective factors of controllers; different controllers may have different understandings of the commands, making it difficult to guarantee the accuracy and consistency of the interpretation. On the other hand, under high-intensity work environments, manual interpretation is inefficient, making it difficult to quickly process large volumes of voice commands, potentially delaying the transmission and execution of flight instructions, thereby affecting normal flight takeoffs and landings and flight safety.

[0003] Currently, although some automated speech recognition technologies have been applied in the air traffic control field, most of these technologies only focus on the accuracy of speech recognition and fail to fully consider the complexity and particularity of command interaction in air traffic control scenarios. For example, without combining historical control call data to mine command sequence association rules and semantic constraints, it is difficult to accurately determine whether voice commands conform to the expected interaction pattern; at the same time, the lack of comprehensive analysis of various interaction features during voice interaction makes it impossible to effectively identify deviation information in commands, and it is difficult to meet the high precision and high reliability requirements of intelligent voice command parsing in air traffic control scenarios. Summary of the Invention

[0004] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a method for intelligent parsing of voice commands applied in air traffic control scenarios, the method comprising:

[0005] A command interaction expectation model for air traffic control scenarios is constructed, which includes a set of command sequence association rules and semantic constraints generated based on historical air traffic control call data.

[0006] The system captures the voice interaction process between air traffic controllers and flight crew members in real time and extracts a set of interaction features during the voice interaction process. The set of interaction features includes speaker identity features, temporal features of instruction components, and voice tone fluctuation features.

[0007] The set of interactive features is input into the instruction interaction expectation model, and dynamic matching and deviation identification operations are performed to generate an interaction deviation analysis result containing the expected deviation type and deviation confidence level.

[0008] Based on the interaction deviation analysis results, the corresponding intent clarification and instruction confirmation process is triggered. The missing instruction element information is obtained by supplementing the interaction with the query, and a complete set of confirmed instruction components is generated.

[0009] The complete set of instruction components undergoes structured transformation to generate instruction parsing results conforming to air traffic control data exchange standards, and these results are then pushed to the air traffic control automation system. Furthermore, this invention also provides a voice instruction intelligent parsing system for air traffic control scenarios, comprising a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium stores programs, instructions, or code, and the processor executes the programs, instructions, or code stored in the machine-readable storage medium to implement the aforementioned method.

[0010] Based on the above, this embodiment of the invention constructs an instruction interaction expectation model that includes instruction sequence association rules and a set of semantic constraints. By fully utilizing historical air traffic control call data, it effectively improves the accuracy and reliability of instruction parsing. It overcomes the shortcomings of traditional manual parsing, which is easily affected by subjective factors, and the lack of consideration for the special characteristics of air traffic control scenarios in existing automation technologies. It captures multiple sets of interaction features in the voice interaction process in real time and inputs them into the instruction interaction expectation model for dynamic matching and deviation identification. It accurately generates interaction deviation analysis results that include expected deviation types and deviation confidence levels. Based on the interaction deviation analysis results, it triggers corresponding intent clarification and instruction confirmation processes. By supplementing the interaction with questions to obtain missing instruction element information, it ensures the accuracy and completeness of the generated complete instruction component set. Finally, it performs structured transformation processing on the complete instruction component set to generate instruction parsing results that conform to air traffic control data exchange standards and pushes them to the air traffic control automation system. This realizes automated and standardized parsing and transmission of voice instructions, improves air traffic control communication efficiency, and ensures the safety and orderliness of flight operations. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the execution flow of the intelligent voice command parsing method for air traffic control scenarios provided in this embodiment of the invention.

[0012] Figure 2 This is a schematic diagram of exemplary hardware and software components of a voice command intelligent parsing system applied to air traffic control scenarios, provided in an embodiment of the present invention. Detailed Implementation

[0013] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating a voice command intelligent parsing method for air traffic control scenarios provided by an embodiment of the present invention. The following is a detailed description of this voice command intelligent parsing method for air traffic control scenarios.

[0014] Step S110: Construct an expected command interaction model in an air traffic control scenario. The expected command interaction model includes a set of command sequence association rules and semantic constraints generated based on historical air traffic control call data.

[0015] In this embodiment, to construct the expected instruction interaction model, it is first necessary to collect a sufficient amount of representative historical air traffic control call data. This historical air traffic control call data covers the air traffic control interaction processes of different time periods, different control areas, and different types of flights, ensuring that the model can adapt to diverse air traffic control scenarios. During the collection process, the data needs to be strictly screened to remove data containing privacy-sensitive information. For privacy content that cannot be directly removed, data anonymization techniques are used, such as anonymizing and converting the involved personnel identity information and specific contact information to meet the requirements of data security and privacy protection.

[0016] The collected historical air traffic control call data includes voice call recordings and corresponding text transcriptions, all of which are pre-labeled with instruction types and interaction intentions. For example, in interactions involving flight altitude adjustments, instruction types may include "request altitude," "agree to altitude," and "refuse altitude," while the interaction intentions correspond to the purposes behind these instructions, such as "avoid air traffic conflicts" and "fly according to the planned flight path."

[0017] Next, instruction sequence association rules and a set of semantic constraints are generated based on these historical air traffic control call data. The instruction sequence association rules aim to uncover the inherent connections and patterns of occurrence between different instruction components, while the set of semantic constraints clarifies the combinations, conflicts, and priority relationships between instruction components based on air traffic control regulations and standard operating procedures. By integrating these two parts into the model, the instruction interaction expectation model can make reasonable predictions and judgments about the real-time voice instruction interaction process.

[0018] Step S111: Collect a set of historical air traffic control call data in the air traffic control scenario. The set of historical air traffic control call data includes voice call recordings with labeled instruction types and interaction intentions, as well as corresponding text transcription results.

[0019] When collecting historical air traffic control call data sets, it is necessary to establish clear data collection standards. First, the time range of the data should be determined, which usually covers air traffic control calls under different seasons, weather conditions, and traffic flow conditions to ensure the comprehensiveness of the data. For example, it should include both busy air traffic control calls during weekday peak hours and sparse air traffic control calls during off-peak hours on holidays; it should include both normal flight control on clear days and air traffic control adjustments under special weather conditions such as thunderstorms.

[0020] Voice call recordings must be clear and intelligible, avoiding excessive background noise interference. For recordings with poor sound quality, preprocessing, such as noise reduction, is necessary to improve the accuracy of subsequent text transcription. The transcribed text must completely correspond to the spoken content, including every instruction, response, and interjection used in the conversation. Simultaneously, the annotators need professional air traffic control knowledge to accurately identify and annotate the type of instruction and the intent of each interaction in the call.

[0021] For example, in a recorded conversation between an air traffic controller and a flight crew, the transcribed text would read, "Controller: CSN3102, maintain current heading, ascend to designated altitude. Crew: CSN3102 received, maintain current heading, ascend to designated altitude." The annotator would label the controller's instruction type as "request to adjust flight status" and the interaction intent as "ensure the flight flies according to the planned route and altitude"; the crew's instruction type would be labeled as "confirmation instruction," and the interaction intent as "responding to controller's instruction and indicating intention to execute."

[0022] The collected data needs to be categorized and stored to create a structured database. Each record in the database includes information such as the storage path of the voice call recording, the corresponding transcribed text, the command type label, and the interaction intent label, facilitating subsequent processing and analysis.

[0023] Step S112: Perform instruction component segmentation processing on the text transcription results in the historical control call data set to identify the instruction subject component, operation object component, constraint condition component, and execution requirement component in each call segment.

[0024] When segmenting instruction components in text transcription results, word segmentation and entity recognition methods from natural language processing techniques are required. First, the text is segmented, breaking down the continuous text sequence into individual words or phrases. Then, based on predefined instruction component categories, such as instruction body components, operation object components, constraint condition components, and execution requirement components, each word or phrase is identified and classified.

[0025] The main component of an instruction typically refers to the recipient of the instruction, which in air traffic control scenarios is primarily the controller or flight crew. In the text, it may appear as a call sign, such as "Controller" or "CSN3102 Crew." The target component is the objective of the instruction, such as "heading," "altitude," "speed," or "a specific airspace." The constraint component restricts or preconditions on the target component, such as "currently," "specified," or "within a certain time period." The execution requirement component specifies the requirements for executing the instruction, such as "maintain," "ascend to," or "adjust to."

[0026] For example, when segmenting the text "Controller: CSN3102, maintain current heading, ascend to the designated altitude.", "Controller" is identified as the subject of the instruction; "CSN3102", although the recipient of the instruction, can be considered as an associated component related to the operation object in this context, while "heading" and "altitude" are identified as operation object components; "current" and "designated" are identified as constraint components; and "maintain" and "ascend to" are identified as execution requirement components.

[0027] During the segmentation process, some ambiguous words or phrases may be encountered, requiring judgment based on the context. For example, the word "plan" in "execute according to plan" needs to be determined by considering the surrounding text to ascertain its specific constraints, such as time plan, flight route plan, or altitude plan. Furthermore, for some compound components, further breakdown is necessary to ensure that each component can be accurately categorized.

[0028] After segmentation, the instruction components in each call segment need to be marked, and the start and end positions of each component need to be recorded in the text index for subsequent analysis and processing.

[0029] Step S113: Analyze the co-occurrence relationship and temporal dependency relationship between different instruction components, and use the association rule mining algorithm to extract the combination pattern of instruction components that meets the preset support threshold, and generate instruction sequence association rules.

[0030] When analyzing the co-occurrence relationships between different instruction components, it is necessary to statistically analyze the frequency of simultaneous occurrence of different instruction components in historical air traffic control communication data. For example, the execution requirement component "ascend to" often co-occurs with the operational object component "altitude," and the execution requirement component "maintain" also frequently co-occurs with operational object components such as "heading" and "speed." Through the above statistics, we can gain a preliminary understanding of which instruction components have strong correlations.

[0031] The analysis of temporal dependencies focuses on the order in which different instruction components appear on the timeline. In a control call, the appearance of instruction components often follows a predetermined order, such as first the instruction provider issuing the instruction, then describing the object of operation, then stating the constraints, and finally giving the execution requirements. For example, "The controller (instruction provider) requires CSN3102 (associated object) to maintain (execution requirements) the current (constraints) heading (object of operation)." This order reflects the temporal dependencies between instruction components.

[0032] When using association rule mining algorithms, the first step is to transform the segmented instruction components into a form suitable for algorithm processing, such as mapping each instruction component to a unique identifier. Then, a preset support threshold is set; support represents the probability of a combination of instruction components appearing in all data. When the support of a combination of instruction components reaches or exceeds this threshold, the combination pattern is considered to have a certain degree of universality and representativeness, and it is extracted as part of the instruction sequence association rules.

[0033] For example, after analysis, it was found that the combination pattern "execution requirement component - ascend to", "operation object component - height", and "constraint condition component - specify" had a support level in historical control call data that reached a preset threshold. Then, this combination pattern will be extracted as an instruction sequence association rule, indicating that when the execution requirement "ascend to" appears, it is likely to be accompanied by the operation object "height" and the constraint condition "specify".

[0034] Step S1131: Encode the segmented instruction components, mapping each instruction component to a unique component identifier, and generating a set of instruction component sequences composed of component identifiers.

[0035] Encoding the segmented instruction components is to convert the textual instruction components into a symbolic form that the computer can process more efficiently. Each instruction component, regardless of its type—whether it is an instruction body, an operation object, a constraint, or an execution requirement—is assigned a unique component identifier. These identifiers can be combinations of letters and numbers, or other forms of symbols, but they must be unique.

[0036] For example, "controller" is mapped to "Z1", "CSN3102" to "J1", "ascend to" to "X1", "altitude" to "C1", "current" to "Y1", "maintain" to "X2", "heading" to "C2", and so on. Through the above mapping, a text containing instructions will be converted into a sequence of these symbols, such as "Z1J1X2Y1C2X1Y2C1" (corresponding to "controller CSN3102 maintains current heading and ascends to the specified altitude").

[0037] The generated set of instruction component sequences needs to correspond one-to-one with the original instruction component sequences so that the corresponding original instruction components can be traced back during subsequent analysis and rule generation. Simultaneously, a mapping table between component identifiers and original instruction components is established to facilitate reverse conversion when needed, restoring the identifiers to the original instruction components.

[0038] Step S1132: Calculate the support parameters of any two instruction components that appear simultaneously in the instruction component sequence set, and generate the component co-occurrence frequency matrix.

[0039] Calculating the support parameter for any two instruction components requires traversing the entire set of instruction component sequences. For each instruction component sequence, all occurrences of each instruction component are examined, and the frequency of co-occurrence for each pair is recorded. Then, the frequency of co-occurrence for each pair is divided by the total number of all instruction component sequences to obtain the support parameter for that pair. The support parameter reflects the probability that the two instruction components co-occur.

[0040] For example, if the number of times "X1 (ascend to)" and "C1 (height)" appear together in all instruction component sequences is A, and the total number of instruction component sequences is B, then the support parameter for "X1" and "C1" is A / B.

[0041] When generating the component co-occurrence frequency matrix, the rows and columns of the matrix correspond to different component identifiers, and the elements in the matrix are the support parameters of the corresponding two instruction components. For example, the element in the i-th row and j-th column of the matrix represents the support parameter of component identifier i and component identifier j appearing simultaneously. This matrix allows for a direct visual understanding of which instruction components have a closer co-occurrence relationship.

[0042] During the calculation process, it is necessary to distinguish whether two instruction components are adjacent in the sequence and the impact of their order of occurrence on co-occurrence. However, in the calculation of the support parameter, the main focus is on the fact that they occur at the same time, without considering their specific positions and orders.

[0043] Step S1133: Based on the component co-occurrence frequency matrix, the Apriori algorithm is used to mine frequent itemsets that satisfy the minimum support threshold. The frequent itemsets represent combinations of instruction components whose co-occurrence probability reaches a preset threshold.

[0044] When applying the Apriori algorithm based on the component co-occurrence frequency matrix, the minimum support threshold must first be determined. This threshold needs to be set based on the actual historical control call data and business requirements. A threshold that is too high may result in too few frequently occurring itemsets, failing to reflect the true relationships; a threshold that is too low may generate a large number of meaningless frequently occurring itemsets, increasing the difficulty of subsequent processing.

[0045] The basic idea of ​​the Apriori algorithm is to find all frequent itemsets that meet the minimum support threshold through a layer-by-layer search. First, it identifies all frequent itemsets, i.e., sets where the support of a single instruction component reaches the minimum support threshold. Then, it generates candidate two-itemsets based on these frequent itemsets, checking whether their support reaches the threshold; those candidate two-itemsets that meet the condition are identified as frequent two-itemsets. This process is repeated, gradually generating higher-order frequent itemsets, until no new frequent itemsets can be generated.

[0046] For example, a frequent item set might include "X1 (ascend to)," "C1 (altitude)," "X2 (maintain)," and "C2 (heading)." Candidate two-items sets generated from these frequent item sets, such as "X1 and C1" and "X2 and C2," are checked and found to have support levels reaching the minimum support threshold, thus becoming frequent two-items sets. Candidate three-items sets, such as "X1, C1, and Y2 (specify)," are then generated based on these frequent two-items sets, and their support is further checked to determine if they are frequent three-items sets.

[0047] These frequent itemsets represent combinations of instruction components that frequently appear simultaneously in historical control call data, and are an important basis for generating instruction sequence association rules.

[0048] Step S1134: Perform sequence pattern analysis on the frequent itemset to identify the order of appearance of different instruction components on the time axis and generate a time sequence pattern rule that includes the order of appearance of components and the interval distance.

[0049] When performing sequence pattern analysis on frequent itemsets, it is necessary to combine the original set of instruction component sequences and focus on the order in which the instruction components in the frequent itemset appear in the sequence. For each frequent itemset, iterate through all instruction component sequences containing that frequent itemset and record the order in which the instruction components appear.

[0050] For example, for the frequent itemset "X1 (ascend to), Y2 (specify), C1 (height)," in different instruction component sequences, "Y2" might appear first, followed by "X1," and finally "C1," or "X1" might appear first, followed by "Y2," and then "C1," and so on. By statistically analyzing the frequency of these different sequences, the main order of occurrence of the instruction components in the frequent itemset can be determined.

[0051] Interval distance refers to the distance between the positions of two instruction components in a sequence, i.e., the number of instruction components in between. For example, in the sequence "Z1J1Y2X1C1", there are no other instruction components between "Y2" and "X1", so the interval distance is 1; the interval distance between "X1" and "C1" is also 1. By recording the interval distance, the timing relationship between instruction components can be described in more detail.

[0052] Based on the identified order of occurrence and interval distance of the components, a temporal pattern rule is generated. Each temporal pattern rule contains the instruction components that make up the rule, their order of occurrence, and the interval distance between adjacent components. For example, "Y2 (specify) - X1 (ascend to) (interval distance 1) - C1 (height) (interval distance 1)" is a temporal pattern rule, indicating that in the instruction sequence, "specify" usually appears before "ascend to" with an interval distance of 1, and "ascend to" appears before "height" with an interval distance of 1.

[0053] Step S1135: Associate and store the time-series pattern rules with the corresponding support parameters and confidence parameters to generate an instruction sequence association rule library. Each rule in the instruction sequence association rule library includes a preceding component sequence, a following component sequence, and a rule strength index.

[0054] When storing time-series pattern rules in association with support and confidence parameters, the confidence parameter for each time-series pattern rule must first be calculated. The confidence parameter represents the probability that a subsequent component sequence will appear when the preceding component sequence appears. It is calculated by counting the number of instruction component sequences that contain both the preceding and following component sequences, and dividing that number by the number of instruction component sequences that contain the preceding component sequence.

[0055] For example, for the time series pattern rule "Pre-component sequence - Y2 (specified), X1 (rising to); Post-component sequence - C1 (height)", the number of instruction component sequences that simultaneously contain "Y2, X1" and "C1" is C, and the number of instruction component sequences that contain "Y2, X1" is D. Then the confidence parameter of this rule is C / D.

[0056] The rule strength index can take into account support parameters and confidence parameters, or other indicators that can reflect the importance and reliability of the rule. The specific calculation method can be determined according to actual needs.

[0057] When generating the instruction sequence association rule base, each instruction sequence association rule explicitly includes a preceding component sequence, a following component sequence, and a rule strength index. The preceding component sequence is the instruction component sequence that appears first in the rule, and the following component sequence is the instruction component sequence that appears after the preceding component sequence. This structured storage method enables rapid querying and matching of relevant rules during subsequent real-time parsing, allowing for prediction and judgment of real-time instruction sequences.

[0058] Step S114: Based on air traffic control regulations and standard operating procedures, construct a set of semantic constraints, which includes allowed pairings, prohibited conflicts, and priority ordering relationships among instruction components.

[0059] Air traffic control regulations and standard operating procedures (SOPs) form the basis for constructing a set of semantic constraints. These regulations and procedures clearly define and require the interaction of instructions in air traffic control. The construction process requires the participation of professional air traffic controllers and technicians to conduct in-depth interpretation and analysis of these regulations and procedures, extracting the constraint relationships related to the instruction components.

[0060] Permissible combinations refer to which instruction components can be reasonably combined together. For example, the execution requirement components such as "ascend to" and "descend to" can be combined with the operation object component "altitude," and the execution requirement component "hold" can be combined with operation object components such as "heading," "speed," and "altitude."

[0061] The prohibition of conflicting relationships refers to the fact that certain instruction components cannot appear simultaneously, otherwise it will lead to contradictory instructions or non-compliance with operational specifications. For example, the two execution requirement components "ascend to" and "descend to" cannot be paired with the same "height" operation object component at the same time, because a height cannot simultaneously require both ascent and descent.

[0062] Priority ranking primarily concerns which instruction component or sequence has a higher execution priority when different instruction components or sequences appear simultaneously. For example, in an emergency, an instruction sequence containing emergency-related components such as "emergency climb" or "emergency descent" has a higher priority than a normal flight adjustment instruction sequence.

[0063] These extracted permitted collocation relationships, prohibited conflict relationships, and priority ordering relationships are organized and structured to form a set of semantic constraints.

[0064] Step S1141: Parse air traffic control regulations and standard operating procedures documents, extract normative clauses related to instruction interaction, and convert each normative clause into a structured constraint description statement.

[0065] When analyzing air traffic control regulations and standard operating procedures (SOPs), it is necessary to read and understand the document content sentence by sentence, and then filter out the normative clauses related to instruction interaction. This process needs to be combined with the actual business scenarios of air traffic control to ensure that the extracted clauses accurately reflect the constraints in the instruction interaction process. For example, an air traffic control regulation clause stipulates that "when controllers issue altitude adjustment instructions to aircraft crews, they must specify the specific altitude value and the timing of the adjustment." This clause is directly related to instruction interaction and can be extracted for subsequent processing.

[0066] When converting normative clauses into structured constraint descriptions, it is necessary to clarify the subject, object, and constraint relationship of the statement. The subject is usually the issuer or receiver of the instruction, the object is the operational object or execution requirement involved in the instruction, and the constraint relationship is the restriction or requirement stipulated in the clause. For example, the clause extracted above can be converted into "When the instruction subject (controller) issues an execution requirement (adjustment) for the operational object (altitude) to the instruction receiver (crew), it must include constraint conditions (specific altitude value, timing of adjustment)."

[0067] During the conversion process, it is necessary to avoid vague expressions and ensure that the constraint description statements are clear, targeted, and actionable. For complex clauses, it may be necessary to break them down into multiple structured constraint description statements to clearly express the multiple constraint relationships they contain. At the same time, a consistency check should be performed on the converted statements to ensure that there are no contradictions or repetitions between the converted statements of different clauses.

[0068] Step S1142: Perform semantic analysis on the constraint description statement to identify the subject components, object components and constraint relationship types in the statement. The constraint relationship types include allowed collocation relationships, prohibited conflict relationships and priority ordering relationships.

[0069] When performing semantic analysis on constraint description statements, semantic role labeling technology from natural language processing is used to determine the semantic role of each component in the statement. The subject component is usually the object performing the constraint or the object being constrained; in air traffic control scenarios, this often refers to controllers, flight crew, or specific instruction types. The object component is the object to which the constraint relationship points; it may be a specific instruction or a combination of instruction components.

[0070] Identifying constraint relationship types is the core of semantic analysis. Statements with permissive collocation relationships typically contain words like "can," "allow," or "able," such as "'maintain' execution requirement can be collocated with 'heading' operation object component." Statements with prohibitive conflict relationships often contain words like "must not," "prohibited," or "cannot," such as "'ascend to' and 'descend to' execution requirements cannot simultaneously act on the same 'altitude' operation object component." Statements with priority ordering relationships may contain expressions like "prioritizes," "higher than," or "precedes," such as "the priority of the command sequence in an emergency is higher than the normal flight command sequence."

[0071] Semantic analysis decomposes each constraint description statement into a triplet structure consisting of a subject component, an object component, and a constraint relation type, laying the foundation for subsequent construction of specific constraint models. For example, the constraint description statement "'ascend to' execution requirement component is allowed to be paired with 'height' operation object component" has, after semantic analysis, a subject component of "'ascend to' execution requirement component", an object component of "'height' operation object component", and a constraint relation type of "allowed pairing relation".

[0072] Step S1143: For allowed collocations, construct an allowed collocation matrix, where the matrix elements indicate whether there is an allowed collocation relationship between two corresponding instruction components.

[0073] A permissible collocation matrix is ​​a two-dimensional matrix in which rows and columns correspond to different instruction components. Each element in the matrix has a specific value rule; for example, "yes" indicates that there is a permissible collocation relationship between the corresponding two instruction components, and "no" indicates that there is no permissible collocation relationship.

[0074] When constructing the matrix, it is necessary to determine the pairing of each pair of instruction components one by one based on the allowed pairing relationships identified in step S1142. For example, according to the allowed pairing relationship "'Maintain' execution requirement component can be paired with 'Heading' operation object component", in the allowed pairing matrix, the intersection element of the row containing "'Maintain' execution requirement component" and the column containing "'Heading' operation object component" is "Yes".

[0075] For instruction pairs that do not explicitly state whether a pairing is permissible, a judgment needs to be made based on common sense about air traffic control operations and actual pairings in historical air traffic control communication data. If a pairing frequently appears in historical air traffic control communication data and does not violate any regulations, it can be considered a permissible pairing; otherwise, it is tentatively determined that a permissible pairing does not exist, and adjustments will be made based on actual feedback during subsequent model optimization.

[0076] The construction of the permissible combination matrix needs to be updated regularly to adapt to changes in air traffic control regulations and operating procedures. When new regulations are introduced or old regulations are revised, the relevant constraint descriptions need to be re-analyzed, and the values ​​of the corresponding elements in the matrix need to be adjusted.

[0077] Step S1144: For prohibited conflict relationships, construct a prohibited conflict list, which contains multiple combinations of mutually conflicting instruction components.

[0078] The prohibited conflict list is constructed based on the prohibited conflict relationships identified in step S1142, and combinations of conflicting instruction components are recorded in list form. Each conflict combination consists of two or more instruction components, and the simultaneous appearance of these components will lead to contradictory instructions or non-compliance with operating procedures.

[0079] For example, based on the prohibited conflict relationship "'ascend to' and 'descend to' execution requirement components may not act on the same 'height' operation object component at the same time", the prohibited conflict list will contain the above combination of "('ascend to' execution requirement component, 'descend to' execution requirement component, 'height' operation object component)".

[0080] During the construction process, each conflict combination needs to be described in detail, explaining the cause of the conflict and its potential consequences, so that conflict situations can be quickly identified and handled during subsequent instruction parsing. Simultaneously, an index should be created for the prohibited conflict list to improve query and matching efficiency. When new prohibited conflict relationships are identified, the corresponding conflict combination should be added to the list promptly; when certain conflict relationships are no longer applicable, they should also be removed from the list in a timely manner.

[0081] Step S1145: Construct a priority directed graph for priority sorting relationship. The nodes in the priority directed graph represent instruction components, and the directed edges represent the priority order between different instruction components.

[0082] Constructing a priority-oriented directed graph requires clearly defining the priority levels among the various instruction components. Nodes represent different instruction components, such as "emergency climb," "normal ascent," and "heading adjustment." Directed edges point from lower-priority nodes to higher-priority nodes, indicating that the instruction component represented by the node pointed to by the arrow has a higher priority.

[0083] For example, if the "emergency climb" execution request has a higher priority than the "normal ascent" execution request, there will be a directed edge in the priority directed graph pointing from the "normal ascent" node to the "emergency climb" node. When both types of instructions appear simultaneously, the "emergency climb" related instructions can be executed first based on the priority relationship in the priority directed graph.

[0084] When constructing a priority directed graph, it is necessary to refer to the explicit provisions regarding command priorities in air traffic control regulations, as well as industry operating practices. For command components that are not explicitly specified in regulations but have different priorities in actual operation, professional personnel should be organized to evaluate and determine them. At the same time, the priority directed graph must maintain consistency to avoid logical errors such as circular pointers. For example, if node A points to node B, and node B points to node C, then node A cannot point to node C to form a cycle, ensuring the transitivity and uniqueness of priority order.

[0085] Step S1146: Integrate the allowed collocation matrix, the prohibited conflict list, and the priority directed graph to generate a set of semantic constraints.

[0086] The integration process organically combines the allowed collocation matrix, the prohibited conflict list, and the priority directed graph into a unified set of semantic constraints. First, these three parts are standardized to ensure they can be invoked and used within the same framework. For example, unified identifiers are assigned to the instruction components in the allowed collocation matrix, the combinations of instruction components in the prohibited conflict list, and the nodes in the priority directed graph, ensuring consistency with the component identifiers generated in step S1131.

[0087] Next, the relationships between the three are established. For example, when a pair of instruction components in the allowed collocation matrix has an allowed collocation relationship, it is necessary to check whether there is a conflicting combination containing this pair of components in the prohibited conflict list. If so, it needs to be verified and corrected to ensure that there is no contradiction between the two. At the same time, the instruction components in the priority directed graph need to correspond to the instruction components in the allowed collocation matrix and the prohibited conflict list, so that multiple constraints can be applied simultaneously for judgment during instruction parsing.

[0088] The integrated set of semantic constraints needs to undergo completeness and consistency checks. Completeness checks ensure that all constraints related to instruction interaction are included without omission; consistency checks ensure that there are no contradictions among the constraints within the set. Any issues discovered during the checks must be corrected promptly until the set of semantic constraints meets the requirements.

[0089] Step S115: Input the instruction sequence association rules and the set of semantic constraints into the model training framework, and construct an instruction interaction expectation model through machine learning algorithms. The instruction interaction expectation model can predict the subsequent possible instruction component sequences based on the interaction features of the current input.

[0090] The selection of a model training framework needs to consider the characteristics of air traffic control scenarios and data features. Typically, a framework suitable for sequence prediction and constraint inference is chosen. The instruction sequence association rules and the set of semantic constraints are used as inputs to the model. The instruction sequence association rules provide the model with the patterns of instruction components appearing in historical air traffic control call data, while the set of semantic constraints provides the model with the basis for judging whether the combination of instruction components is reasonable.

[0091] The choice of machine learning algorithm should be based on the specific model objective. For example, recurrent neural networks can be used to handle the temporal relationships of instruction components, while decision trees can be used to handle reasoning under semantic constraints. During training, the instruction component sequences from historical traffic control data are first used as training samples and input into the model. The model predicts subsequent instruction components based on instruction sequence association rules and verifies and corrects the prediction results by incorporating a set of semantic constraints.

[0092] For example, when the model predicts that the "'descend to' execution requirement component" may appear based on the instruction sequence association rule, it can check whether the component can be matched with the already appearing "height" operation object component through the allowed matching matrix in the semantic constraint condition set. If the matching is allowed, the prediction result is retained; if not, the prediction result is adjusted.

[0093] During training, model parameters need to be continuously adjusted to improve the accuracy of model predictions and the degree to which they conform to semantic constraints. Through multiple iterations of training, the model can accurately predict subsequent possible instruction component sequences given the current interaction features, and these sequences must meet the requirements of instruction sequence association rules and the set of semantic constraints.

[0094] Step S120: Capture the voice interaction process between the controller and the crew in real time, and extract the interaction feature set in the voice interaction process. The interaction feature set includes speaker identity features, instruction component temporal features, and voice tone fluctuation features.

[0095] Real-time capture of voice interactions relies on voice acquisition equipment deployed at the control site. This equipment must possess high sensitivity and anti-interference capabilities, enabling it to clearly capture the voice signals between controllers and flight crew. The acquired voice signals are then transmitted in real-time to the processing system for subsequent feature extraction.

[0096] Extracting the interaction feature set is to convert the speech signal into feature data that can be processed by the instruction interaction expectation model. Speaker identity features are used to distinguish between controllers and crew members, ensuring that the sender and receiver of instructions can be accurately identified; instruction component temporal features reflect the order and distribution of the components in the instruction over time; and speech intonation fluctuation features can help determine the speaker's emotions and the urgency of the instruction.

[0097] During the extraction process, it is necessary to ensure that the time synchronization between the features is guaranteed, so that they can accurately correspond to the same moment or time period of the voice interaction process.

[0098] Step S121: Acquire the voice interaction signal between the controller and the crew in real time through the voice acquisition device, and perform frame segmentation processing on the voice interaction signal to obtain a voice frame sequence with timestamp information.

[0099] Voice acquisition equipment typically includes microphone arrays and audio interfaces, deployed at control positions and crew communication equipment locations, capable of receiving voice signals from both sides in real time. The acquired voice interaction signal is a continuous analog signal, which needs to be converted into a digital signal for subsequent processing. During the conversion process, appropriate sampling rates and quantization bits must be set. The sampling rate should meet the frequency range requirements of the air traffic control voice signal, while the quantization bits must ensure the accuracy of the voice signal.

[0100] When performing frame segmentation on digital speech signals, a sliding window approach is used to divide the continuous signal into multiple overlapping speech frames. The window length and overlap ratio need to be determined based on the characteristics of the speech signal. For example, the window length can be set to include a complete speech syllable, while the overlap ratio can reduce information loss between frames. Each speech frame is assigned a timestamp accurate to the millisecond level, recording the position of the original speech signal corresponding to the speech frame on the timeline.

[0101] For example, during the voice interaction between the controller and the CSN3102 crew, the collected voice signal can be processed into a series of voice frames after being segmented into frames. Each voice frame has a timestamp such as "1620000000000" or "1620000002000", which corresponds to the voice segment at different times.

[0102] Step S122: Extract Mel frequency cepstral coefficient features for each speech frame in the speech frame sequence to obtain the Mel frequency cepstral coefficient vector of each speech frame.

[0103] Mel-frequency cepstral coefficient feature extraction is a common method in speech signal processing, effectively reflecting the spectral characteristics of speech. First, a Fourier transform is performed on each speech frame to convert the time-domain signal into a frequency-domain signal, obtaining the power spectrum of the speech frame. Then, the power spectrum is passed through a Mel filter bank, whose frequency response is uniformly distributed on the Mel scale, simulating the human ear's perception of different frequency sounds.

[0104] After passing through the Mel filter bank, the logarithm of the output of each filter is taken to obtain the log-Mel spectrum. Then, a discrete cosine transform is performed on the log-Mel spectrum, and the first few coefficients of the transform result are used to form the Mel frequency cepstral coefficient vector of the speech frame. This vector typically contains eigenvalues ​​in multiple dimensions, each corresponding to a Mel frequency cepstral coefficient, which can characterize the spectral envelope features of the speech frame.

[0105] For example, after processing a speech frame, the resulting Mel frequency cepstral coefficient vector may contain 12 eigenvalues. These eigenvalues ​​together describe the spectral characteristics of the speech frame and can be used for subsequent speaker recognition and speech content analysis.

[0106] Step S123: Use a Gaussian mixture model to perform cluster analysis on the Mel frequency cepstral coefficient vector, and aggregate speech frames with similar features into candidate speech segments of the same speaker.

[0107] Gaussian mixture models are probabilistic clustering algorithms that group samples with similar characteristics into a single category. When performing clustering analysis on Mel frequency cepstral coefficient vectors, the model parameters, including the number of Gaussian components, must first be determined. The number of Gaussian components can be initially set based on the range of speakers in historical traffic control data, and then optimized through model training.

[0108] The Mel-frequency cepstral coefficient vectors of all speech frames are input into a Gaussian mixture model. The model calculates the probability that each vector belongs to each Gaussian component. Based on the probability, the vectors are assigned to the categories represented by the corresponding Gaussian components, thus aggregating speech frames with similar features together to form candidate speech segments. Each candidate speech segment is considered to be likely from the same speaker.

[0109] For example, in the voice interaction between the controller and the CSN3102 crew, the Mel frequency cepstral coefficient vectors of the voice frames belonging to the controller have similar characteristics and can be aggregated into one candidate voice segment; while the voice frames belonging to the crew members will be aggregated into another candidate voice segment.

[0110] Step S124: Calculate the similarity between different candidate speech segments, merge adjacent candidate speech segments belonging to the same speaker, and obtain the initial speaker segmentation result.

[0111] When calculating the similarity between candidate speech segments, methods such as dynamic time warping or cosine similarity can be used. Dynamic time warping can handle speech segments of different lengths and calculates the similarity between them by finding the optimal time alignment; cosine similarity measures the similarity by calculating the cosine value of the set of Mel-frequency cepstral coefficient vectors of the two segments.

[0112] When the similarity between two adjacent candidate speech segments reaches a preset threshold, they are considered to belong to the same speaker and are merged into a longer speech segment. During the merging process, the timestamp information of the two segments is retained to form the start and end timestamps of the merged speech segment. In this way, speech segments of the same speaker that might have been segmented are re-merged to obtain the initial speaker segmentation result.

[0113] For example, if two adjacent candidate speech segments come from controllers but are split into two segments due to speech pauses, and their similarity is found to reach a threshold, they are merged to form a controller speech segment containing longer speech content.

[0114] Step S125: Post-process the initial speaker segmentation results by analyzing the duration and interval characteristics of the speech segments to eliminate speech segments with durations below a preset threshold and noise interference segments.

[0115] The post-processing procedure first sets a minimum duration threshold for speech segments, which is determined based on the shortest duration of normal speech in air traffic control voice interaction. Speech segments with a duration lower than this threshold in the initial speaker segmentation results are considered to be noise or invalid speech segments and are removed from the results.

[0116] Next, analyze the interval characteristics between speech segments. If the interval between two adjacent speech segments from the same speaker is short and the signal energy during the interval is low, it may be due to short pauses during speech. In this case, the two speech segments can be merged. For segments with high energy during the interval but whose characteristics differ significantly from those of surrounding speakers, they are identified as noise interference segments and eliminated.

[0117] For example, if an initial speaker segmentation result contains a speech segment with a duration of only 0.2 seconds, which is far below the preset minimum duration threshold, the segment will be identified as invalid and eliminated; if the interval between two controller speech segments is 0.5 seconds and the signal energy during the interval is extremely low, the two segments will be merged.

[0118] Step S126: Assign a unique speaker identifier to each speaker, record the start and end timestamps of each speaker's speech segment, and generate a speaker segmentation result containing the speaker identifier and time interval.

[0119] After post-processing the initial speaker segmentation results, the remaining speech segments are further segmented by speaker. For each different speaker, a unique speaker identifier is assigned, such as "S1" or "S2", where "S1" can represent an air traffic controller and "S2" can represent the CSN3102 crew.

[0120] Record the start and end timestamps of each speaker's speech segment. The start timestamp is the timestamp of the first speech frame in the segment, and the end timestamp is the timestamp of the last speech frame. Associate the speaker identifier with the corresponding time interval to form the speaker segmentation result. This result can clearly reflect which speaker the speech interaction signal comes from in different time intervals.

[0121] For example, the speaker segmentation results may contain records such as "S1: 1620000000000-1620000010000" and "S2: 1620000011000-1620000020000", which respectively indicate that the voice signal came from the controller and the CSN3102 crew within the corresponding time interval.

[0122] Step S127: Perform feature alignment processing on the speaker identity features, the temporal features of the instruction components, and the speech intonation fluctuation features to generate an interactive feature set containing time dimension markers.

[0123] The core of feature alignment processing is to ensure that speaker identity features, temporal features of instruction components, and speech intonation fluctuation features are consistent in the time dimension so that subsequent models can accurately associate the correspondence between different features.

[0124] In this embodiment, alignment is performed based on timestamps. The speaker identity feature includes the start and end timestamps of each speaker's speech segment. Each instruction component in the instruction sequence feature also carries a corresponding timestamp. The speech intonation fluctuation feature is extracted based on the timestamps of the speech frame sequence. During alignment, if a certain type of feature information is missing in a time segment, it can be marked to indicate that this type of feature is missing during that time period. After alignment, each time segment in the generated interaction feature set contains the corresponding speaker identity feature, instruction sequence feature, and speech intonation fluctuation feature.

[0125] Step S130: Input the set of interactive features into the instruction interaction expectation model, perform dynamic matching and deviation identification operations, and generate an interaction deviation analysis result containing the expected deviation type and deviation confidence level.

[0126] After the set of interactive features is input into the expected instruction interaction model, the model uses the constructed set of instruction sequence association rules and semantic constraints to dynamically analyze the real-time interactive features. The dynamic matching process continuously compares the temporal features of the real-time instruction components with the expected instruction sequence in the model, while deviation identification identifies the differences between the two.

[0127] In this embodiment, based on the scenario of flight altitude adjustment, after the set of interaction features is input into the model, the model first focuses on the currently appearing instruction components, such as the controller's "CSN3102, ascend to...". Combining the instruction sequence association rules, it anticipates that subsequent instruction components such as "specify altitude value" and "maintain this altitude" may appear. By continuously receiving new interaction features, the model continuously updates the matching results. Once it finds that the actual instruction component does not match the expectation, such as the absence of the expected altitude value, or the appearance of components such as "descend to" that conflict with "ascend to", it will be identified as a deviation, and the confidence level of the deviation will be calculated, ultimately forming the interaction deviation analysis result.

[0128] Step S131: Extract the temporal features of the currently appearing instruction components from the interaction feature set and input them into the sequence prediction module of the instruction interaction expectation model.

[0129] When extracting the temporal features of currently appearing instruction components from the interaction feature set, it is necessary to traverse the time segments of the interaction feature set, collect the instruction components contained in each time segment, arrange them in chronological order, and form a continuous sequence of instruction components.

[0130] For example, in a flight altitude adjustment scenario, the temporal sequence features of the currently appearing instruction components extracted from the interaction feature set might be "Controller – CSN3102 – Ascend to – Designated," with these instruction components arranged in chronological order of their appearance. This sequence is then input into the sequence prediction module of the instruction interaction expectation model. Based on these already appearing instruction components and the rules in the instruction sequence association rule base, this module predicts subsequent possible instruction components.

[0131] The input to the sequence prediction module is a structured sequence of instruction components. Each instruction component has a corresponding component type and timestamp information. The module parses this information to determine the current state of the sequence and prepares for subsequent predictions.

[0132] Step S132: The sequence prediction module predicts the types and probabilities of subsequent possible instruction components based on the instruction sequence association rules, and generates the expected instruction component sequence.

[0133] During the prediction process, the sequence prediction module can first search the instruction sequence association rule base to find the preceding component sequence that matches the currently appearing instruction component sequence. For example, if the currently appearing instruction component sequence is "Controller - CSN3102 - Ascend to - Designated", the module will search the rule base for rules whose preceding component is this sequence or a fragment thereof.

[0134] After finding a matching rule, the types of subsequent instruction components are determined based on the sequence of post-components in the rule. For example, if a rule's pre-component is "rise to - specify" and its post-component is "height value - hold", then it predicts that subsequent instruction components of the height value type and the hold type may appear. Simultaneously, based on the confidence parameter in the rule strength index, the probability of these instruction components appearing is calculated; the higher the probability, the greater the likelihood of that component appearing.

[0135] The generated expected instruction component sequence is an ordered list of component types, each with a corresponding probability of occurrence, such as "high value (probability P1) - keep (probability P2)", where P1 and P2 are determined according to the confidence level of the rule, and P1+P2+...=1 (the sum of the probabilities of all possible component types is 1).

[0136] Step S133: Compare the temporal features of the real-time instruction components in the interaction feature set with the expected instruction component sequence, and calculate the component type matching degree and temporal order similarity.

[0137] The temporal features of real-time instruction components are continuously updated during the interaction process. During comparison, the latest real-time instruction component sequence is compared with the expected instruction component sequence. The component type matching degree is calculated by counting the number of components in the real-time instruction component sequence that have the same type as those in the expected instruction component sequence, and dividing that number by the total number of components in the expected instruction component sequence.

[0138] For example, if the expected instruction component sequence is "height value - hold" and the real-time instruction component sequence is "height value - confirm", then the component type matching degree is 1 / 2 (only "height value" matches).

[0139] The calculation of temporal order similarity involves checking whether the order of appearance of real-time instruction components matches the order of the expected instruction component sequence. If the order of components in the real-time sequence is completely consistent with the expected sequence, the similarity is 1; if there is an order reversal, such as the expected sequence being "height value - keep" while the real-time sequence is "keep - height value", the similarity is reduced. The specific value is determined based on the number and position of the components with reversed order, and can be obtained by calculating the ratio of the length of the longest common subsequence of the two sequences to the length of the expected sequence.

[0140] Step S134: Based on the component type matching degree and temporal sequence similarity, identify the deviation between the actual instruction component and the expected instruction component, determine the deviation location and deviation type, and the deviation type includes component missing deviation, component redundancy deviation and temporal sequence disorder deviation.

[0141] When the component type matching degree is lower than a set threshold or the temporal sequence similarity is low, it indicates that there is a deviation between the actual instruction component and the expected instruction component. Determining the deviation location involves finding the specific position in the real-time instruction component sequence that does not match the expected sequence. This can be achieved by comparing the components in the two sequences one by one, and the index position of the mismatched component is the deviation location.

[0142] For example, if the expected sequence is [component A, component B, component C] and the real-time sequence is [component A, component D, component C], then the deviation position is at index 1 (component B and component D do not match).

[0143] The types of deviations are classified based on their specific manifestations: missing component deviation refers to the presence of a component in the expected sequence but its absence in the real-time sequence, such as the expected presence of component B but its absence in the real-time sequence; redundant component deviation refers to the presence of a component in the real-time sequence that is not in the expected sequence, such as the presence of component D in the real-time sequence but its absence in the expected sequence; and temporal disorder deviation refers to the inconsistency between the order of components in the real-time sequence and the expected sequence, such as the expected order of component B before component C, but the real-time order of component C before component B.

[0144] Step S1341: Convert the real-time instruction component temporal features and the expected instruction component sequence into component type sequences, where each element represents the instruction component type at the corresponding position.

[0145] During the conversion, the specific content of the instruction component is ignored, and only its type is retained. For example, the timing characteristics of the real-time instruction component are "controller - CSN3102 - ascend to - designated - 8000 meters", which is converted into the component type sequence [instruction body, associated object, execution requirement, constraint condition, operation object (altitude value)]; the expected instruction component sequence is "altitude value - hold", which is converted into the component type sequence [operation object (altitude value), execution requirement].

[0146] The above transformation makes the comparison process more focused on the type attributes of the components, reduces the interference of specific content differences on the matching results, and facilitates the rapid identification of type-level deviations.

[0147] Step S1342: Use the dynamic time warping algorithm to align the two component type sequences and find the optimal matching path.

[0148] Dynamic time warping (RTW) is used to handle the alignment of two sequences of different lengths. By stretching or compressing the time axis of one of the sequences, it finds the optimal matching path between the two sequences. In the alignment of component type sequences, the algorithm calculates the similarity between each component type in the two sequences; the higher the similarity, the more matched the two component types are.

[0149] For example, the real-time component type sequence is [Instruction Body, Associated Object, Execution Requirement, Constraint Condition, Operation Object (Height Value)], and the expected component type sequence is [Execution Requirement, Constraint Condition, Operation Object (Height Value), Execution Requirement (Maintain)]. The dynamic time warping algorithm calculates the similarity of component types at each position. For example, the "Execution Requirement" in the real-time sequence has a high similarity to the "Execution Requirement" in the expected sequence, and the "Constraint Condition" has a high similarity to the "Constraint Condition". By adjusting the alignment of the sequences, it finds the matching path that maximizes the overall similarity, such as aligning the [Execution Requirement, Constraint Condition, Operation Object (Height Value)] in the real-time sequence with the [Execution Requirement, Constraint Condition, Operation Object (Height Value)] in the expected sequence.

[0150] Step S1343: Based on the optimal matching path, identify components that exist in the real-time instruction component sequence but not in the expected instruction component sequence, and mark them as potential redundant components.

[0151] After the optimal matching path is determined, the real-time instruction component sequence is traversed, and each component is checked to see if a corresponding matching component can be found in the expected instruction component sequence. If a component has no matching item in the expected sequence and does not appear at any position in the expected sequence, it is marked as a potentially redundant component.

[0152] For example, in a flight altitude adjustment scenario, the real-time instruction sequence is "Controller – CSN3102 – Ascend to – Designated – 8000 meters – Check fuel level", while the expected instruction sequence is "Controller – CSN3102 – Ascend to – Designated – 8000 meters – Hold". Through optimal matching path comparison, the component "Check fuel level" has no corresponding match in the expected sequence and is therefore marked as a potentially redundant component.

[0153] When marking potentially redundant components, their position and component type in the real-time sequence can be recorded so that subsequent analysis can determine whether the component is indeed redundant information.

[0154] Step S1344: Identify components that exist in the expected instruction component sequence but not in the real-time instruction component sequence, and mark them as potential missing components.

[0155] Similarly, based on the optimal matching path, the expected instruction component sequence is traversed, and each component is checked to see if a corresponding matching component can be found in the real-time instruction component sequence. If a component in the expected sequence has no matching item in the real-time sequence, it is marked as a potentially missing component.

[0156] For example, the expected instruction sequence is "Controller - CSN3102 - Ascend to - Designate - 8000 meters - Hold", while the real-time instruction sequence is "Controller - CSN3102 - Ascend to - Designate - 8000 meters". Through comparison, the component "Hold" in the expected sequence does not appear in the real-time sequence and is therefore marked as a potentially missing component.

[0157] Recording the location and type of potentially missing components is crucial for subsequent clarification of intent and confirmation of instructions, as it clarifies what additional questions need to be asked.

[0158] Step S1345: Analyze the cases where the component types are inconsistent at corresponding positions in the aligned component type sequence and mark them as potential type error components.

[0159] At the alignment position corresponding to the optimal matching path, compare the component types of the real-time component type sequence and the expected component type sequence. If the component types at the same alignment position are different, they are marked as potentially type-incorrect components.

[0160] For example, after alignment, the component at a certain position in the real-time sequence is "decline to" (execution requirement), while the component at the corresponding position in the expected sequence is "rise to" (execution requirement). Both are of type execution requirement, but their specific content conflicts, which is a component type inconsistency and is marked as a potential type error component.

[0161] Potential type errors may lead to misunderstandings of instructions, and require close attention and verification in subsequent processes.

[0162] Step S1346: Determine the location of the deviation based on the distribution of potential redundant components, potential missing components, and potential type error components.

[0163] The location of the deviation is determined based on the distribution of the three potential deviation components in the sequence. For a potential redundant component, its location in the real-time sequence is a deviation location; for a potential missing component, its location in the expected sequence corresponds to the corresponding location in the real-time sequence (i.e., the location where the component should appear in the real-time sequence but does not) as the deviation location; for a potential type-error component, its alignment location is the deviation location.

[0164] For example, the potential redundant component "check oil quantity" is located at index 5 in the real-time sequence, the potential missing component "keep" is located at index 5 in the expected sequence, corresponding to index 5 in the real-time sequence, and the potential type error component "decrease to" is located at alignment position index 2, then the deviation positions are index 2 and 5.

[0165] Identifying the location of the deviation helps in subsequent targeted deviation handling and instruction correction.

[0166] Step S1347: Based on the component characteristics of the deviation location, the deviation type is divided into component missing deviation, component redundancy deviation and temporal sequence disorder deviation. Component missing deviation means that the expected component does not appear, component redundancy deviation means that the unexpected component appears, and temporal sequence disorder deviation means that the component order does not match the expectation.

[0167] The deviation is classified according to the component characteristics of the deviation location: if the deviation location corresponds to a potentially missing component, the deviation type is component missing deviation; if it corresponds to a potentially redundant component, it is component redundancy deviation; if there is a situation in the component type sequence where the component order does not match the expectation, such as component A is after component B in the real-time sequence, but component A is before component B in the expected sequence, and the above order difference is not in the above two deviation types, it is classified as temporal disorder deviation.

[0168] For example, the expected sequence is "specified - height value - hold", and the real-time sequence is "height value - specified - hold". Both component types exist and are not redundant, but the order is different. The above situation belongs to the time sequence disorder deviation.

[0169] Step S135: Calculate the deviation confidence level corresponding to each deviation type, wherein the deviation confidence level is determined based on the importance weight of the deviation component and the matching degree difference value.

[0170] The importance weight of the deviation component is determined based on its role in the instruction sequence. Key components, such as execution requirements and operation objects, have higher weights, while auxiliary components, such as interjections, have lower weights. The matching degree difference value is calculated based on the similarity difference between the real-time component and the expected component using a dynamic time warping algorithm. The greater the difference, the higher the matching degree difference value.

[0171] The deviation confidence level is calculated as follows: Deviation Confidence Level = Importance Weight × Match Difference Value. For example, if the importance weight of the potentially missing component "Maintain" (execution requirement) is 0.8, and its match difference value is 0.9 (due to a complete mismatch), then the confidence level for the deviation of this missing component is 0.8 × 0.9 = 0.72; if the importance weight of the potentially redundant component "Check Oil Quantity" is 0.3, and its match difference value is 0.8, then the confidence level for its redundancy deviation is 0.3 × 0.8 = 0.24.

[0172] The higher the confidence level of the bias, the greater the credibility and impact of the bias, and the more it needs to be addressed.

[0173] Step S136: Integrate the deviation location, deviation type, and deviation confidence information to generate interactive deviation analysis results.

[0174] The identified deviation locations, corresponding deviation types, and calculated deviation confidence levels are summarized and organized according to a set format to generate interactive deviation analysis results. These results are presented in a structured data format, such as a list or table (the structured content is described in text here). Each record contains the deviation location, deviation type, deviation confidence level, and related component information.

[0175] For example, the results of the interaction bias analysis might be: [Bias location: index 5, bias type: component missing bias, bias confidence: 0.72, relevant component: hold; bias location: index 2, bias type: latent type error component, bias confidence: 0.65, relevant component: decrease to (actual) and increase to (expected)].

[0176] Step S140: Based on the interaction deviation analysis results, trigger the corresponding intent clarification and instruction confirmation process, obtain the missing instruction element information through supplementary inquiry interaction, and generate a complete instruction component set after confirmation.

[0177] When the interactive deviation analysis results show deviations, especially component missing deviations and potential type error components, it is necessary to initiate the intent clarification and instruction confirmation process. In this embodiment, for the component missing deviation "maintain" in the flight altitude adjustment scenario, a corresponding clarification inquiry can be generated to confirm with the controller or crew whether the altitude needs to be maintained; for the potential type error component "descend to", it can be asked whether it is a slip of the tongue, and that it should actually be "ascend to".

[0178] After obtaining missing or incorrect instruction element information through supplementary queries, it is integrated into the original instruction component sequence. After multiple confirmations and corrections, a complete and accurate set of instruction components is finally generated.

[0179] Step S141: Analyze the interaction deviation analysis results and extract the deviation location and corresponding expected instruction component type for deviation types that are component missing deviations.

[0180] When parsing the results of the interaction deviation analysis, records with the deviation type "component missing deviation" are filtered out, and their deviation locations and corresponding expected instruction component types are extracted. For example, deviation location index 5 is extracted from the results, and the expected instruction component type is "execution requirement (maintain)".

[0181] Step S142: Query the preset clarification query template library according to the expected instruction component type, select the clarification query statement that matches the missing component type, convert the clarification query statement into a speech signal through speech synthesis technology, and play it to the corresponding speaker through a speech output device.

[0182] The pre-built clarification inquiry template library stores standardized inquiry statements for different types of instruction components. For example, for missing components of the "Execution Requirement (Maintain)" type, the template library might contain statements such as "Do we need to maintain the current heading?" or "Please confirm whether to maintain the current state?"; for missing components of the "Constraint (Time)" type, it might contain statements such as "At what specific time should this be executed?" or "Within what time period should the adjustment be completed?". These template statements are designed by professional air traffic controllers based on actual work scenarios to ensure that the language is concise, clear, and conforms to air traffic control communication standards.

[0183] When querying the clarification query template library, you can filter the most matching statement from the template library based on the type of missing expected instruction component, such as execution requirements, operation objects, constraints, etc. For example, if the missing component is "operation object (height value)," you can locate the template category related to height and select "Please provide the specific target height?" as the clarification query statement.

[0184] After selecting a clarifying question, speech synthesis technology converts the text into a speech signal. During speech synthesis, speech parameters that meet the requirements of air traffic control communication can be used, such as moderate speech rate, steady intonation, and clear pronunciation, to ensure that the corresponding speaker can accurately understand the question. For example, when converting "Please provide the specific target altitude?" into a speech signal, the pronunciation of each word is calibrated to avoid ambiguity or unclear pronunciation.

[0185] Finally, the voice signal is played to the corresponding speaker via voice output devices, such as the controller's workstation speaker or the crew's headset. Before playback, the speaker's identity can be verified to ensure accurate transmission of the inquiry signal. For example, if the missing information is related to a crew member, the voice signal will be directed to that crew member's communication device to avoid interfering with other communications.

[0186] Step S143: Collect the speaker's response speech signal, perform speech recognition and semantic analysis on the response speech signal, extract supplementary instruction element information, identify the component type of the supplementary instruction element information, and determine the instruction component type to which it belongs.

[0187] After receiving a clarification inquiry, the speaker can respond via voice device, and the system will capture these responses in real time using voice acquisition equipment. During the acquisition process, the timestamp of the response can be recorded simultaneously for later alignment with the temporal characteristics of the command components. For example, if the crew responds with "target altitude is 8000 meters," the corresponding voice signal will be captured in real time, and the acquisition time will be marked.

[0188] The speech recognition process for responding to the speech signal follows the same logic as in step S126, involving framing, feature extraction, acoustic model matching, and language model decoding to convert the speech signal into a text sequence. For example, the speech signal "target altitude is 8000 meters" is converted into the corresponding text content.

[0189] In the semantic parsing stage, the grammatical structure and semantic relationships of the text sequence can be analyzed to extract instructional elements. For example, the core information "8000 meters" can be extracted from "target altitude is 8000 meters" as a supplementary instructional element.

[0190] Component type identification involves determining the type of instruction component based on the extracted supplementary information. For example, "8000 meters" is a specific description of "operation object (height)," and is therefore identified as a sub-component of "operation object (height)," falling under the operation object component type. If the response is "execute within 5 minutes," then the extracted "within 5 minutes" will be identified as a "constraint condition (time)" type.

[0191] Step S144: Based on the deviation position information in the interaction deviation analysis results, determine the insertion position of the supplementary instruction element information in the instruction component time sequence feature, insert the supplementary instruction element information into the insertion position, and update the component sequence of the instruction component time sequence feature.

[0192] The deviation location information in the interactive deviation analysis results clarifies the position of the missing component within the temporal characteristics of the original instruction component. For example, if the deviation location information indicates that "operation object (height)" should be located after "execution requirement (rise to)", then the supplementary height information needs to be inserted at that position.

[0193] The insertion point can be precisely calculated based on the timestamp and the logical order of the component sequence. For example, if the original component sequence is [Instruction subject (controller), Execution requirement (ascend to), Constraint condition (immediately)], and the deviation position is after the execution requirement, then the supplementary "Operation object (altitude 8000 meters)" will be inserted between "Execution requirement (ascend to)" and "Constraint condition (immediately)" to form a new sequence [Instruction subject (controller), Execution requirement (ascend to), Operation object (altitude 8000 meters), Constraint condition (immediately)].

[0194] During insertion, the indexes of the original component sequence can be automatically adjusted to ensure the logical continuity of the order of all components. Simultaneously, a corresponding timestamp is added to the supplementary instruction element information. This timestamp is based on the acquisition time of the response speech signal and fine-tuned with reference to the temporal relationship between preceding and following components to ensure consistency of temporal characteristics.

[0195] Step S145: Check whether the temporal features of the instruction components after inserting supplementary information satisfy the allowed collocation relationships in the semantic constraint condition set. If the allowed collocation relationships are satisfied, the updated temporal features of the instruction components are retained. If the allowed collocation relationships are not satisfied, they are marked as potential conflicts, and a further clarification and confirmation process is triggered.

[0196] After inserting supplementary information, the allowed collocations in the semantic constraint set can be used for verification. For example, check whether "execution requirement (ascend to)" and the supplemented "operation object (altitude 8000 meters)" conform to the allowed collocation rules. According to air traffic control regulations, "ascend to" and "altitude" are allowed collocations, so the verification passes, and the updated component sequence is retained.

[0197] If the supplementary information is "operation object (speed 800 knots)", and the original sequence contains "execution requirement (ascend to)", then the combination of "ascend to" and "speed" does not conform to the allowed combination relationship and can be marked as a potential conflict.

[0198] For potential conflicts, a further clarification and confirmation process can be triggered, such as generating a new inquiry statement: "Is the speed information you mentioned related to the ascending instruction? Please confirm the specific requirements." By interacting with the speaker again, the conflict issue can be resolved.

[0199] Step S146: Recalibrate the timestamp information in the updated temporal characteristics of the instruction components so that the chronological order of the instruction components is consistent with the actual interaction process.

[0200] When recalibrating the timestamps, the time axis of the actual interaction process can be used as a reference to check the timestamps of each instruction component. For example, in the original sequence, the timestamp of "Execution requirement (ascend to)" is T1, the acquisition time of the supplemented "Operation object (altitude 8000 meters)" is T2 (T2 > T1), and the timestamp of the original "Constraint condition (immediately)" is T3 (T3 > T2). After calibration, the order of T1 < T2 < T3 will be maintained to ensure that the timestamps are consistent with the actual conversation order.

[0201] If a timestamp anomaly occurs, such as the timestamp of the supplementary component being earlier than the preceding component, it can be automatically adjusted, and the timestamp of the supplementary component can be corrected to a reasonable value between the preceding and succeeding components. For example, if the timestamp of the preceding component is 10:00:02 and the acquisition time of the supplementary component is 10:00:01 (due to equipment delay), it can be calibrated to 10:00:03 to ensure correct time logic.

[0202] Step S147: Repeat the deviation identification and clarification inquiry process until there is no component missing deviation or the deviation confidence level is lower than the preset threshold in the interaction deviation analysis result, and summarize all the instruction components in the final temporal characteristics of the instruction components to generate a confirmed complete instruction component set.

[0203] After the first clarification inquiry, the dynamic matching and deviation identification operations in step S130 can be performed again to check whether there is still a component missing deviation. For example, after supplementing the altitude information, it may be found that the "Constraint condition (time)" is still missing. At this time, the processes of steps S141 to S146 can be repeated to clarify the time constraint condition.

[0204] When the deviation identification result shows that all components are complete, or the confidence level of the remaining deviations is lower than the preset threshold (for example, the confidence level of a certain potentially missing component is 0.2, lower than the threshold of 0.3), the system stops the clarification process.

[0205] Finally, all components in the final instruction sequence characteristics, such as the instruction subject, execution requirements, operation object, and constraints, are summarized to form a complete instruction component set. For example, the summarized set may include: [Instruction subject (controller), execution requirements (ascend to), operation object (altitude 8000 meters), constraints (immediately), instruction subject (crew), execution requirements (confirmation), operation object (altitude 8000 meters)].

[0206] Step S150: Perform structured conversion processing on the complete set of instruction components to generate instruction parsing results that conform to the air traffic control data exchange standard, and push the instruction parsing results to the air traffic control automation system.

[0207] A complete set of instruction components is a series of unstructured instruction elements that need to be converted into structured data that conforms to air traffic control data exchange standards before it can be recognized and processed by air traffic control automation systems. The structured conversion process includes data mapping, format verification, and standardization to ensure that the parsed results meet the data exchange requirements between systems.

[0208] Step S151: Parse the complete set of instruction components and extract the specific contents of the instruction body component, operation object component, constraint condition component, and execution requirement component.

[0209] When parsing a complete set of instruction components, the type of each component can be identified one by one, and its specific content can be extracted. For example, the main components of the instruction can be extracted from the set as "Controller A" and "Crew B"; the components of the operation object are "Altitude 8000 meters" and "Heading 360 degrees"; the components of the constraint conditions are "Immediately" and "Within 10 minutes"; and the components of the execution requirements are "Ascend to", "Maintain", and "Confirm", etc.

[0210] During the extraction process, the content of each component can be standardized to remove redundant information. For example, the core information in "Unit B responded and confirmed ascending to an altitude of 8,000 meters" can be extracted into the execution requirement "confirmation" and the operation object "altitude 8,000 meters".

[0211] Step S152: Based on the data fields defined in the air traffic control data exchange standard, map the extracted instruction components to the corresponding fields to generate structured data records.

[0212] The air traffic control data exchange standard specifies the data field formats for various command information, such as "command source," "operation type," "target parameters," and "constraints." Based on these field definitions, the extracted component content can be mapped.

[0213] For example, "Instruction Subject (Controller A)" is mapped to the "Instruction Source" field; "Execution Requirement (Upgraded to)" is mapped to the "Operation Type" field; "Operation Object (Altitude 8000 meters)" is mapped to the "Target Parameter" field; and "Constraint Condition (Immediate)" is mapped to the "Time Constraint" field.

[0214] After mapping is completed, the generated structured data record may be: {“Instruction Source”:“Controller A”,“Operation Type”:“Ascent to”,“Target Parameter”:“Altitude 8000 meters”,“Time Constraint”:“Immediate”,“Recipient”:“Flight B”}.

[0215] Step S153: Perform format verification on the structured data record to check whether the field length, data type and value range meet the requirements of the air traffic control data exchange standard.

[0216] The format validation first checks the field length. For example, the "Instruction Source" field is required to be no more than 20 characters. If the actual content is "Controller A0123456789" (12 characters in total), it meets the requirements. If it exceeds 20 characters, it can be truncated or prompted to re-enter.

[0217] Data type checks ensure that the content type of each field is correct. For example, "target parameter (height)" should be a string type with a number and unit. If it is a pure number or other non-string form, it is judged as not conforming.

[0218] The value range check verifies whether the content is within the range allowed by the standard. For example, the "Operation Type" field only allows preset values ​​such as "Rise to", "Fall to", and "Hold". If "Accelerate to" (a non-preset value) appears, it is determined to be non-compliant.

[0219] Step S154: If the format validation passes, generate a standard-compliant instruction parsing result; if the format validation fails, correct the non-compliant fields until the format validation passes.

[0220] After the format verification is passed, the structured data record is confirmed as a standard instruction parsing result, for example: {“Instruction ID”:“CMD20230510001”,“Instruction Source”:“Controller A”,“Operation Type”:“Ascent to”,“Target Parameter”:“Altitude 8000 meters”,“Time Constraint”:“Immediately”,“Recipient”:“Unit B”,“Generation Time”:“2023-05-10 10:05:30”}.

[0221] If the validation fails, the non-compliant fields can be corrected. For example, if the "Target Parameter" field contains "8000" (missing a unit), the unit can be automatically added to correct it to "Height 8000 meters"; if the "Operation Type" is "Accelerate to", manual confirmation can be prompted or it can be replaced with the closest preset value "Adjust Speed ​​to".

[0222] Step S155: Push the instruction parsing result to the air traffic control automation system and receive the reception confirmation signal returned by the air traffic control automation system to complete the instruction parsing result pushing process.

[0223] When pushing command parsing results, the system sends the structured command parsing results to the designated interface of the air traffic control automation system through data transmission protocols within the air traffic control system, such as TCP / IP. During the pushing process, data can be encrypted to ensure transmission security.

[0224] After receiving the parsed results, the air traffic control automation system can return a reception confirmation signal, which includes information such as the reception time and the result status (success / failure). For example, the confirmation signal might be: {"Reception Status": "Success", "Reception Time": "2023-05-10 10:05:32", "Command ID": "CMD20230510001"}.

[0225] After receiving a successful confirmation signal, record the push completion time and end the current instruction parsing process; if the reception fails, you can retry pushing until you succeed or reach the maximum number of retries (the number of retries can be set according to actual needs).

[0226] Figure 2 The illustration shows exemplary hardware and software components of a voice command intelligent parsing system 100 for air traffic control scenarios, which can implement the ideas of this application, according to some embodiments of this application. For example, processor 120 can be used in the voice command intelligent parsing system 100 for air traffic control scenarios and to perform the functions in this application.

[0227] For example, the intelligent voice command parsing system 100 applied to air traffic control scenarios may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the intelligent voice command parsing system 100 applied to air traffic control scenarios may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The intelligent voice command parsing system 100 applied to air traffic control scenarios also includes an I / O interface 150 between the computer and other input / output devices.

[0228] Furthermore, this embodiment of the invention also provides a readable storage medium, which has computer-executable instructions pre-set in it. When the processor executes the computer-executable instructions, the above-mentioned intelligent voice command parsing method applied to air traffic control scenarios is implemented.

[0229] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for intelligent parsing of voice instructions applied to an air traffic control scene, characterized in that, The method includes: A command interaction expectation model for air traffic control scenarios is constructed, which includes a set of command sequence association rules and semantic constraints generated based on historical air traffic control call data. The system captures the voice interaction process between air traffic controllers and flight crew members in real time and extracts a set of interaction features during the voice interaction process. The set of interaction features includes speaker identity features, temporal features of instruction components, and voice tone fluctuation features. The set of interactive features is input into the instruction interaction expectation model, and dynamic matching and deviation identification operations are performed to generate an interaction deviation analysis result containing the expected deviation type and deviation confidence level. Based on the interaction deviation analysis results, the corresponding intent clarification and instruction confirmation process is triggered. Missing instruction element information is obtained through supplementary questioning interaction, and a complete set of confirmed instruction components is generated. The complete set of instruction components is subjected to structured conversion processing to generate instruction parsing results that conform to the air traffic control data exchange standard, and the instruction parsing results are pushed to the air traffic control automation system; The construction of the command interaction expectation model in the air traffic control scenario includes: Collect a set of historical air traffic control call data in air traffic control scenarios. The set of historical air traffic control call data includes voice call recordings with labeled instruction types and interaction intentions, as well as corresponding text transcription results. The text transcription results in the historical control call data set are processed by instruction component segmentation to identify the instruction body component, operation object component, constraint condition component and execution requirement component in each call segment; The co-occurrence and temporal dependencies among different instruction components are analyzed, and an association rule mining algorithm is used to extract instruction component combination patterns that meet a preset support threshold, thereby generating instruction sequence association rules. Based on air traffic control regulations and standard operating procedures, a set of semantic constraints is constructed, which includes allowed combinations, prohibited conflicts, and priority ordering relationships among instruction components. The instruction sequence association rules and the set of semantic constraints are input into the model training framework, and an instruction interaction expectation model is constructed through machine learning algorithms. The instruction interaction expectation model can predict the subsequent possible instruction component sequences based on the current input interaction features. The analysis of co-occurrence and temporal dependencies among different instruction components employs an association rule mining algorithm to extract instruction component combination patterns that satisfy a preset support threshold, generating instruction sequence association rules, including: The segmented instruction components are encoded, and each instruction component is mapped to a unique component identifier, generating a set of instruction component sequences composed of component identifiers. Calculate the support parameters for any two instruction components that appear simultaneously in the instruction component sequence set, and generate a component co-occurrence frequency matrix; Based on the component co-occurrence frequency matrix, the Apriori algorithm is used to mine frequent itemsets that satisfy the minimum support threshold. The frequent itemsets represent combinations of instruction components whose co-occurrence probability reaches a preset threshold. Sequence pattern analysis is performed on the frequent itemsets to identify the order of appearance of different instruction components on the time axis, and to generate time sequence pattern rules that include the order of appearance of components and the interval distance. The time-series pattern rules are associated and stored with their corresponding support and confidence parameters to generate an instruction sequence association rule library. Each rule in the instruction sequence association rule library includes a preceding component sequence, a following component sequence, and a rule strength index.

2. The voice instruction intelligent parsing method applied to the air traffic control scene according to claim 1, characterized in that, The set of semantic constraints, constructed based on air traffic control regulations and standard operating procedures, includes: Analyze air traffic control regulations and standard operating procedures documents, extract normative clauses related to instruction interaction, and convert each normative clause into a structured constraint description statement; Semantic analysis is performed on the constraint description statement to identify the subject components, object components, and constraint relationship types in the statement. The constraint relationship types include allowed collocation relationships, prohibited conflict relationships, and priority ordering relationships. To determine the permissible collocations, a permissible collocation matrix is ​​constructed, where each element represents whether a permissible collocation exists between two corresponding instruction components. For prohibited conflict relationships, a prohibited conflict list is constructed, which contains multiple combinations of mutually conflicting instruction components; To determine the priority ranking relationship, a priority directed graph is constructed, where nodes in the priority directed graph represent instruction components, and directed edges represent the priority order between different instruction components. By integrating the allowed collocation matrix, the prohibited conflict list, and the priority directed graph, a set of semantic constraints is generated.

3. The intelligent voice command parsing method for air traffic control scenarios according to claim 1, characterized in that, The real-time capture of the voice interaction process between the controller and the crew, and the extraction of the interaction feature set during the voice interaction process, include: The voice interaction signals between the controller and the crew are acquired in real time by the voice acquisition device. The voice interaction signals are then processed into frames to obtain a voice frame sequence with timestamp information. Mel frequency cepstral coefficient features are extracted from each speech frame in the speech frame sequence to obtain the Mel frequency cepstral coefficient vector of each speech frame. A Gaussian mixture model is used to perform cluster analysis on the Mel frequency cepstral coefficient vector to aggregate speech frames with similar features into candidate speech segments of the same speaker. Calculate the similarity between different candidate speech segments, merge adjacent candidate speech segments belonging to the same speaker, and obtain the initial speaker segmentation result; The initial speaker segmentation results are post-processed by analyzing the duration and interval characteristics of speech segments to eliminate speech segments with durations below a preset threshold and noise interference segments. Assign a unique speaker identifier to each speaker, record the start and end timestamps of each speaker's speech segment, and generate speaker segmentation results that include speaker identifiers and time intervals; The speech segments of each speaker are extracted from the speaker segmentation results, and voiceprint features are extracted to generate speaker identity features for identifying the speaker's identity. Speech recognition processing is performed on the speech segments of each speaker, which are converted into text sequences. The text sequences are labeled with instruction components, and the components such as instruction subject, operation object, constraint conditions and execution requirements in the text sequences are identified. The temporal features of instruction components containing component types and timestamps are generated. The fundamental frequency and energy features are extracted from the speech frame sequence, the fundamental frequency change rate and energy fluctuation amplitude are calculated, and speech intonation fluctuation features representing speech intonation changes are generated. The speaker identity features, the temporal features of the instruction components, and the speech intonation fluctuation features are aligned to generate an interactive feature set containing time dimension markers.

4. The intelligent voice command parsing method for air traffic control scenarios according to claim 1, characterized in that, The step of inputting the set of interactive features into the instruction interaction expectation model, performing dynamic matching and deviation identification operations, and generating interaction deviation analysis results containing expected deviation types and deviation confidence levels includes: Extract the temporal features of the currently appearing instruction components from the interaction feature set and input them into the sequence prediction module of the instruction interaction expectation model; The sequence prediction module predicts the types and probabilities of subsequent possible instruction components based on instruction sequence association rules, thereby generating an expected instruction component sequence. The temporal features of the real-time instruction components in the interaction feature set are compared with the expected instruction component sequence to calculate the component type matching degree and temporal order similarity. Based on the component type matching degree and temporal sequence similarity, the deviation between the actual instruction component and the expected instruction component is identified, and the deviation location and deviation type are determined. The deviation types include component missing deviation, component redundancy deviation, and temporal sequence disorder deviation. Calculate the deviation confidence level corresponding to each deviation type, which is determined based on the importance weight of the deviation component and the difference in matching degree; By integrating information on deviation location, deviation type, and deviation confidence level, interactive deviation analysis results are generated.

5. The intelligent voice command parsing method for air traffic control scenarios according to claim 4, characterized in that, The step of identifying the deviation between the actual instruction component and the expected instruction component based on the component type matching degree and temporal sequence similarity, and determining the deviation location and type, includes: The real-time instruction component time-series features and the expected instruction component sequence are converted into component type sequences, where each element represents the instruction component type at the corresponding position; The dynamic time warping algorithm is used to align the two component type sequences and find the optimal matching path; Based on the optimal matching path, identify components that exist in the real-time instruction component sequence but not in the expected instruction component sequence, and mark them as potential redundant components; Identify components that exist in the expected instruction component sequence but not in the real-time instruction component sequence and mark them as potential missing components; Analyze the inconsistencies in component types at corresponding positions in the aligned component type sequence and mark them as potentially type-incorrect components; Based on the distribution of potential redundant components, potential missing components, and potential type error components, determine the location of the deviation; Based on the component characteristics of the deviation location, the deviation types are divided into component missing deviation, component redundancy deviation, and temporal sequence disorder deviation. Component missing deviation indicates that the expected component has not appeared, component redundancy deviation indicates that an unexpected component has appeared, and temporal sequence disorder deviation indicates that the order of components does not match the expectation.

6. The intelligent voice command parsing method for air traffic control scenarios according to claim 1, characterized in that, The process of triggering the corresponding intent clarification and instruction confirmation based on the interaction deviation analysis results obtains missing instruction element information through supplementary questioning interactions, and generates a complete set of confirmed instruction components, including: The interaction deviation analysis results are analyzed to extract the deviation location and the corresponding expected instruction component type for deviation types that are component missing deviations; According to the expected instruction component type, a preset clarification question template library is queried, a clarification question statement matching the missing component type is selected, the clarification question statement is converted into a speech signal through speech synthesis technology, and played to the corresponding speaker through a speech output device; Collect the speaker's response speech signal, perform speech recognition and semantic analysis on the response speech signal, extract supplementary instruction element information, identify the component type of the supplementary instruction element information, and determine the instruction component type to which it belongs. Based on the deviation position information in the interaction deviation analysis results, the insertion position of the supplementary instruction element information in the instruction component time series feature is determined, and the supplementary instruction element information is inserted into the insertion position to update the component sequence of the instruction component time series feature. Check whether the temporal features of the instruction components after inserting supplementary information satisfy the allowed collocation relationships in the semantic constraint condition set. If the allowed collocation relationships are satisfied, the updated temporal features of the instruction components are retained. If the allowed collocation relationships are not satisfied, they are marked as potential conflicts and a further clarification and confirmation process is triggered. The timestamp information in the updated instruction component timing features is recalibrated to ensure that the timing order of each instruction component is consistent with the actual interaction process. Repeat the deviation identification and clarification inquiry process until there are no missing component deviations or the deviation confidence is lower than the preset threshold in the interactive deviation analysis results. Then, summarize all the instruction components in the final instruction component time-series features to generate a confirmed complete instruction component set.

7. The intelligent voice command parsing method for air traffic control scenarios according to claim 1, characterized in that, The step of performing structured transformation on the complete set of instruction components to generate instruction parsing results conforming to air traffic control data exchange standards, and pushing the instruction parsing results to the air traffic control automation system, includes: Analyze the complete set of instruction components to extract the specific contents of the instruction body component, operation object component, constraint condition component, and execution requirement component; Based on the data fields defined in the air traffic control data exchange standard, the extracted instruction components are mapped to the corresponding fields to generate structured data records. The structured data records are format-validated to check whether the field length, data type, and value range meet the requirements of the air traffic control data exchange standard. If the format validation passes, a standard-compliant instruction parsing result is generated; if the format validation fails, the non-compliant fields are corrected until the format validation passes. The command parsing result is pushed to the air traffic control automation system, and a reception confirmation signal is received from the air traffic control automation system to complete the command parsing result push process.

8. A voice command intelligent parsing system for air traffic control scenarios, characterized in that, The system includes a processor and a memory, the memory being connected to the processor. The memory is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the memory to implement the intelligent voice command parsing method for air traffic control scenarios as described in any one of claims 1-7.

Images