Retrospective verification method, device and electronic equipment for NOTAM analysis results

Abstract

The present application discloses a retrospective verification method, device, and electronic device for NOTAM parsing results, relating to the field of artificial intelligence. The method comprises: obtaining the parsing results of the NOTAM by the target system; determining the key information of the NOTAM from the parsing results through an inverse model, and reconstructing and generating the target NOTAM based on the key information; detecting the similarity between the target NOTAM and the NOTAM; performing dependency grammar analysis on the target NOTAM and the NOTAM to obtain an analysis result, wherein the dependency grammar analysis is used to analyze the word dependency relationship and grammatical role relationship between the target NOTAM and the NOTAM; and determining the parsing accuracy of the NOTAM by the target system based on the similarity and analysis result. The present application solves the technical problem that the prior art mainly uses expert rules to verify the NOTAM parsing results, resulting in low verification efficiency of the NOTAM parsing results, thereby affecting aviation safety.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a method, device, and electronic device for retrospectively verifying the parsing results of a NOTAM. Background Art

[0002] As a key source of information for safe aviation operations, NOTAMs comprehensively cover all factors affecting flight safety within the flight information region (FIR), such as the status of airport facilities, the operating conditions of navigation equipment, and changes to air traffic service procedures. This information is crucial for flight safety and the smooth operation of flights, helping pilots and other personnel stay informed of any potential flight impacts.

[0003] With the rapid development of the civil aviation industry and the continuous increase in flight volume, the number and complexity of NOTAMs are also increasing. Traditionally, NOTAM processing relied primarily on manual analysis and verification. However, this approach is not only inefficient but also prone to human error. Manual analysis and verification requires significant time and effort, especially when faced with massive and complex NOTAMs. Mistakes, omissions, and difficulty updating these NOTAMs are common, posing a significant risk to aviation safety.

[0004] As can be seen from the above, current verification methods mostly rely on manual or simple automated tools to check the parsing results. This approach is difficult to fully cover all possible error types, especially for complex NOTAM content, which is prone to omissions or misjudgments. Furthermore, existing verification methods lack effective means to verify the comprehensiveness and accuracy of parsing results, cannot guarantee the integrity and correctness of parsing results, and cannot achieve automated output of verification results. Manual intervention not only increases the workload and results in low verification efficiency of NOTAM parsing results, but can also lead to inconsistencies and delays in verification results, thereby affecting aviation safety.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] The embodiments of the present application provide a method, device, and electronic device for retrospectively verifying NOTAM parsing results, so as to at least solve the technical problem in the prior art of mainly using expert rules to verify NOTAM parsing results, resulting in low verification efficiency of NOTAM parsing results, thereby affecting aviation safety.

[0007] According to one aspect of an embodiment of the present application, a retrospective verification method for a NOTAM parsing result is provided, comprising: obtaining a parsing result of a NOTAM by a target system; determining key information of the NOTAM from the parsing result through an inverse model, and reconstructing and generating a target NOTAM based on the key information of the NOTAM, wherein the key information of the NOTAM includes at least navigation time information and geographic location information; detecting the similarity between the target NOTAM and the NOTAM; performing dependency grammar analysis on the target NOTAM and the NOTAM to obtain an analysis result, wherein the dependency grammar analysis is used to analyze the word dependency relationship and grammatical role relationship between the target NOTAM and the NOTAM; and determining the parsing accuracy of the NOTAM by the target system based on the similarity and the analysis result.

[0008] Optionally, the target system's parsing accuracy of the navigation notice is determined based on the similarity and analysis results, including: based on the analysis results and the similarity, when it is detected that the target notice and the navigation notice simultaneously meet the following judgment conditions, determining that the target system's parsing accuracy of the navigation notice is greater than a target threshold, wherein the judgment conditions include: a first condition, used to limit the navigation time deviation described in the target notice and the navigation notice to be less than or equal to a preset threshold; a second condition, used to limit the navigation geographical location ranges described in the target notice and the navigation notice to be free of conflict; a third condition, used to limit the flight safety information described in the target notice and the navigation notice to be consistent; and a fourth condition, used to limit the similarity between the target notice and the navigation notice to be greater than a preset similarity.

[0009] Optionally, the parsing accuracy of the target system for the navigation notice is determined based on the similarity and analysis results, including: based on the analysis results and the similarity, when it is detected that the target notice and the navigation notice do not meet at least one of the first condition, the second condition, the third condition and the fourth condition, determining that the parsing accuracy of the target system for the navigation notice is less than or equal to the target threshold.

[0010] Optionally, dependency grammar analysis includes the following steps: identifying the grammatical role of each word in the target notice and the navigation notice; determining the dependency relationship between different words in the notice text sentence based on the grammatical role of each word and the position of each word in the notice text sentence; determining the semantic structure of the notice text sentence based on the dependency relationship between different words in the notice text sentence; extracting key semantic information of the notice text sentence based on the semantic structure of the notice text sentence; and determining the analysis result based on the key semantic information.

[0011] Optionally, the training steps of the inverse model include: obtaining multiple sample data pairs, wherein the sample data pairs include: historical navigation notices and parsed texts of the historical navigation notices; determining labels of the parsed texts, wherein the labels of the parsed texts are used to evaluate the parsing quality of the parsed texts for the historical navigation notices; based on the labels of the parsed texts, screening out target sample data pairs from the multiple sample data pairs; and based on the target sample data pairs, performing multiple iterative training on the neural network to obtain the inverse model.

[0012] Optionally, in the process of performing multiple iterative training on the neural network according to the target sample data pairs to obtain the inverse model, when the neural network is subjected to the i-th iterative training, the ratio of the current number of iterations to the set maximum number of iterations is calculated, where i is an integer greater than 1; the model learning rate during the i-th iterative training is determined according to the ratio; and the i-th iterative training is performed according to the model learning rate.

[0013] Optionally, determining the model learning rate during the i-th iterative training based on the ratio includes: calculating the product of the ratio and pi to obtain a target value; calculating the cosine value of the target value; calculating the difference between the maximum learning rate and the minimum learning rate set for the neural network; and determining the model learning rate during the i-th iterative training based on the cosine value of the target value, the difference between the maximum learning rate and the minimum learning rate set for the neural network, and the minimum learning rate.

[0014] Optionally, before performing the i-th iterative training according to the model learning rate, the latest network parameters of the neural network before the i-th iterative training are obtained; the latest loss function value of the neural network before the i-th iterative training is detected; the gradient information of the latest loss function value based on the latest network parameters is determined; and the model parameter information updated during the i-th iterative training is determined based on the latest network parameters, gradient information of the neural network before the i-th iterative training and the model learning rate during the i-th iterative training.

[0015] Optionally, detecting the latest loss function value of the neural network before the i-th iterative training includes: obtaining the weight of each target sample data pair; obtaining the notice text predicted by the current neural network based on the parsed text in the target sample data pair; determining the target difference value between the notice text predicted by the current neural network and the actual historical navigation notice in the target sample data pair; and determining the latest loss function value of the neural network before the i-th iterative training based on the weight of each target sample data pair and the target difference value.

[0016] Optionally, determining the label of the parsed text includes: determining the label of the parsed text according to at least one of the following labeling rules: a first labeling rule for detecting whether the field information of the parsed text under the key field is consistent with the field information of the historical navigation notice under the key field; a second labeling rule for detecting whether the content in the parsed text includes the navigation-related information of the historical navigation notice; a third labeling rule for detecting whether the logical description of the parsed text in terms of time sequence, navigation range and geographical location is correct.

[0017] According to another aspect of an embodiment of the present application, a retrospective verification device for parsing results of a navigation notice is also provided, including: an acquisition unit, used to obtain the parsing result of the navigation notice by the target system; a first processing unit, used to determine the key information of the notice from the parsing result through an inverse model, and reconstruct and generate a target notice based on the key information of the notice, wherein the key information of the notice includes at least navigation time information and geographic location information; a detection unit, used to detect the similarity between the target notice and the navigation notice; an analysis unit, used to perform dependency grammar analysis on the target notice and the navigation notice to obtain an analysis result, wherein the dependency grammar analysis is used to analyze the word dependency relationship and grammatical role relationship between the target notice and the navigation notice; a second processing unit, used to determine the parsing accuracy of the navigation notice by the target system based on the similarity and the analysis result.

[0018] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned retrospective verification method of the navigation notice parsing result.

[0019] According to another aspect of an embodiment of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors execute the above-mentioned retrospective verification method of the navigation notice parsing results.

[0020] As can be seen from the above, the technical solution provided by this application, compared to manual verification methods that rely on expert rules, can automatically extract key information from the analysis results, including but not limited to flight time and geographic location information, and quickly reconstruct the target announcement without manual intervention. This automated processing greatly shortens the verification cycle, ensuring that key information can be verified and confirmed immediately, meeting the strict information timeliness requirements of flight operations.

[0021] In addition, the combination of the inverse model and dependency parsing technology improves the accuracy and comprehensiveness of the verification of parsing results. Based on a large amount of training data, the model can capture the subtle and complex semantic logic in the navigation notice, avoiding implicit information that traditional expert rules may overlook. Dependency grammar analysis further verifies the consistency of the notice at the syntactic structure level, ensuring that even in the face of complex contexts or non-standard expressions, the correctness of key information can be accurately judged. This two-layer verification mechanism effectively compensates for the blind spots and errors that may exist in a single expert rule. Through the automated verification mechanism, this application eliminates the "missing and forgetting" problems caused by fatigue, negligence or differences in rule understanding during the manual verification process, significantly reducing aviation safety risks. The high similarity between the reconstructed target notice and the original notice and the consistent results of the dependency grammar analysis ensure the reliability and integrity of information transmission, providing pilots and related personnel with more accurate navigation information, thereby avoiding potential flight safety hazards caused by information errors or delays.

[0022] Finally, compared to fixed expert rules, the inverse model based on deep learning has the ability to self-learn and iterate. This means that in the face of new message formats or changes in aviation information, the model can continuously improve its parsing and verification performance through continuous fine-tuning and optimization. This flexibility and adaptability enables the system to seamlessly integrate new information processing processes without interrupting services, better serving the intelligent upgrade of the aviation field. Moreover, the automated verification method eliminates the subjective judgment differences that may occur in manual verification, and achieves a high degree of standardization and consistency in the verification of navigation notice parsing results. No matter when and where, as long as the same parsing result is input, the system can produce consistent verification results, avoiding confusion and contradictions caused by regional differences or personal interpretations, and providing a unified and reliable guarantee for aviation safety on a global scale.

[0023] In summary, the navigation notice parsing result verification method based on the inverse model and dependency grammar analysis proposed in this application not only solves the problems of low efficiency and error-proneness of traditional expert rule verification, but also greatly improves the efficiency and security of aviation information processing while ensuring the accuracy and comprehensiveness of information, providing strong technical support for information management and security assurance in the modern aviation industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 is a flowchart of an optional retrospective verification method of NOTAM parsing results according to an embodiment of the present application;

[0026] Figure 2 1 is a schematic diagram of an optional training process of a reverse inference model according to an embodiment of the present application;

[0027] Figure 3 This is a schematic diagram of an optional comparison agent execution process according to an embodiment of the present application;

[0028] Figure 4 Schematic diagram of an optional retrospective verification device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] It should also be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected by this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation portals for users to choose to authorize or refuse. For example, an interface is set up between this system and the relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain the relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0032] According to an embodiment of the present application, an embodiment of a method for retrospective verification of navigation notice parsing results is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] In an optional embodiment, a retroactive verification system for NOTAM parsing results (hereinafter referred to as the retroactive verification system) may serve as the execution entity of the retroactive verification method for NOTAM parsing results in the embodiments of this application. The retroactive verification system may be a software system or an embedded system combining software and hardware. Of course, those skilled in the art will appreciate that, in addition to using the retroactive verification system as the execution entity, other forms, such as the execution entity of an apparatus or device, may also be used to execute the retroactive verification method for NOTAM parsing results. The embodiments of this application do not specifically limit the specific form of the method execution entity.

[0034] For the sake of convenience, the following describes the solution using the retrospective verification system as the execution entity.

[0035] Figure 1 is a flowchart of an optional retrospective verification method for NOTAM parsing results according to an embodiment of the present application, such as Figure 1 As shown, the method includes the following steps:

[0036] Step S101: Obtain the parsing result of the target system on the NOTAM.

[0037] Alternatively, the target system refers to an automated software system or platform specifically designed to parse NOTAMs. Such systems typically have text processing and information extraction capabilities, capable of reading raw NOTAM messages issued by aviation agencies and converting them into structured data, such as JSON format, for further processing and analysis.

[0038] A NOTAM is a standardized aviation information bulletin that contains information critical to flight safety, such as temporary runway closures, navigation equipment failures, and changes to air traffic regulations. The target system can use natural language processing technologies, including text parsing, keyword recognition, and semantic understanding, to analyze and understand the content of NOTAM messages layer by layer, thereby extracting key information.

[0039] After the target system parses the NOTAM, the generated results are structured data, typically including navigation time, geographic location, restrictions, impact range, etc. This information is organized in a pre-set format and structure to facilitate subsequent verification and comparison operations.

[0040] Optionally, the backtracking verification system often obtains parsing results from the target system through automated interfaces, ensuring timely data transmission and seamless integration. This means that the backtracking verification system can instantly receive the latest parsing results sent by the target system without the need for manual downloading or uploading, improving the efficiency and responsiveness of the overall process. Furthermore, after obtaining the parsing results, the backtracking verification system can also perform a series of data preprocessing tasks, including data cleaning, format conversion, and information classification, to ensure data accuracy and consistency, providing high-quality input for subsequent inverse model verification and dependency grammar analysis.

[0041] Optionally, when obtaining the parsing results, pay special attention to data security and integrity. Ensure the secure encryption of the data transmission channel to prevent information tampering or leakage during transmission. At the same time, check the integrity of the parsing results to confirm that all key information has been correctly extracted to avoid verification errors caused by missing data.

[0042] Step S102: determining key information of the notification from the parsing result through an inverse model, and reconstructing and generating a target notification based on the key information of the notification.

[0043] In step S102, the key information announced includes at least the flight time information and the geographical location information.

[0044] A NOTAM typically contains a wealth of critical information, such as airport codes, effective and expiration dates, geographic coordinates, and facility change status. Upon receiving the original NOTAM, the target system (i.e., the NOTAM parsing system) performs structured parsing to extract this critical information. The unstructured message is then converted into a structured data format, such as JSON, to facilitate subsequent processing and verification. The inverse model is a deep learning model that has been specifically trained to reverse engineer the original message content from the structured parsed results. This process first requires the model to understand key data points in the parsed results, including flight time information (e.g., effective and expiration dates) and geographic information (e.g., airport coordinates and impact range). Through analysis and learning, the model develops rules and patterns for reconstructing the NOTAM message based on these data points. During this phase, the inverse model carefully analyzes the structured parsed results to identify and extract critical information. This information goes beyond time and location, and also includes the specific content of the NOTAM (e.g., facility status, service changes), conditional statements (e.g., restrictions within a specific timeframe), and any details that may impact flight safety. This process ensures that all necessary information is captured accurately, laying the data foundation for subsequent reconstruction generation.

[0045] Optionally, based on the identified key information, the inverse model will attempt to reconstruct a notification message. This reconstructed message aims to mimic the content and format of the original message, but is derived from key data points in the parsed results. The model considers the notification's grammatical rules, industry-standard expressions, and any contextual logic to ensure that the generated message is not only similar in form to the original message, but also highly consistent in semantic content.

[0046] in, Figure 2 is a schematic diagram of an optional training process of the inverse model according to an embodiment of the present application, such as Figure 2 As shown, first prepare an initial model, input data pairs into the initial model, and fine-tune the initial model. After fine-tuning the model, the model can be expanded using sample data, and finally an inverse model is obtained.

[0047] From the above content, it can be seen that by determining key information and reconstructing the process of generating target notifications through the inverse model, intelligent verification of the navigation notification analysis results is achieved, ensuring the accurate transmission of key information, thereby greatly improving flight safety and flight operation efficiency.

[0048] Step S103: detecting the similarity between the target notice and the navigation notice.

[0049] Alternatively, similarity detection can be implemented through a variety of methods, including but not limited to: text-based string matching technology, word vector-based cosine similarity, and deep learning-based sequence alignment network.

[0050] In this application, given the specialized nature and complexity of NOTAMs, a pre-trained deep learning model can be used to calculate the similarity score between the target NOTAM and the NOTAM. This process involves converting the text into embedding vectors and then measuring the similarity of the text using the mathematical distance or angle between the vectors.

[0051] Furthermore, similarity testing must not only focus on the consistency of the notice's content, but also on the similarity of its format. NOTAMs adhere to specific international standard formats, including but not limited to standard formatting for the opening, the order in which key information is presented, and specific abbreviation and encoding rules. Therefore, similarity calculations involve more than just comparing textual content; they also ensure that the generated target notice matches the original notice in format, which directly impacts how aviation personnel understand and use the information.

[0052] Optionally, after completing the similarity calculation, a reasonable threshold needs to be set to determine whether the generated target notice is sufficiently close to the original notice. The threshold setting should take into account the characteristics of navigation notices and the industry's high demand for information accuracy. If the similarity score between the target notice and the original notice is higher than the set threshold, it indicates that the information reconstructed by the inverse model is relatively accurate. Conversely, if the score is lower than the threshold, it may indicate that information was lost or distorted during the parsing process, requiring further inspection and correction.

[0053] Furthermore, NOTAMs often contain rich contextual information that is crucial for understanding their meaning. Therefore, similarity detection should employ a context-sensitive approach, one that understands the specific meaning of words in the NOTAM and their role in context, rather than relying solely on literal similarity. The semantic understanding capabilities of large models ensure accurate assessment of the similarity between two NOTAMs, even in complex contexts or with industry-specific abbreviations.

[0054] In summary, calculating the similarity between the target NOTAM and the original NOTAM using a deep model provides a quantitative metric for evaluating the accuracy of NOTAM parsing and reconstruction. This process is a key component of the NOTAM intelligent verification system, simplifying the traditional expert rule comparison process while also improving its reliability and efficiency.

[0055] Step S104 , performing dependency grammar analysis on the target notice and the NOTAM to obtain an analysis result, wherein the dependency grammar analysis is used to analyze word dependency relationships and grammatical role relationships between the target notice and the NOTAM.

[0056] Optionally, dependency parsing aims to gain a deeper understanding of the internal logical relationships and grammatical structure of the announcement text. By analyzing the dependencies between words, it is possible to identify which words are subject-verb, which are objects, which words are modifiers, and so on. This information is crucial for understanding the true meaning of the announcement, especially when the announcement contains complex conditions or multiple restrictive clauses. Dependency parsing can reveal the precise scope and logical order of these conditions and restrictive clauses.

[0057] During dependency parsing, the dependency type of each word in the announcement can be identified, for example, the dependency between a verb and its subject and object, or the dependency between a noun and its modifiers. Identifying these dependency relationships can help users or models understand the underlying structure of a sentence, ensuring that key information (such as the subject, time, and location details of the "runway closure" event) is correctly presented in the target announcement. Furthermore, each word in the announcement not only has its dependency relationships but also a grammatical role within the sentence. For example, a word may be the subject, object, attributive, or adverbial of a sentence. Identifying these grammatical roles helps ensure the semantic integrity and logical consistency of the announcement information, preventing misinterpretation or omission of information during the reverse reconstruction process.

[0058] It should be noted that by comparing the dependency grammar analysis results of the target notice with the original notice, it is possible to check whether the sentence structure and lexical relationships of the two are consistent. If the dependency structures of the two notices are the same and the grammatical roles of each word match, it can be preliminarily considered that the structure of the notice information has been correctly reconstructed. Dependency grammar analysis can reveal the correctness of the logical relationships in the notice, such as time conditions, location restrictions, etc. If the notice mentions that the "runway closure" event occurred within a specific time period and was limited to a specific geographical area, dependency grammar analysis will check whether this logic is accurately retained and expressed in the target notice.

[0059] Alternatively, dependency parsing can be performed by deep learning models such as recurrent neural networks, long short-term memory networks, or transformer models. These models learn the structure and rules of language through large amounts of training data and can accurately parse the grammatical structure of announcement text.

[0060] In the processing of highly standardized and semantically complex text such as NOTAMs, dependency grammar analysis can not only ensure the integrity of the notification information during the parsing and reconstruction process, but also further guarantee the correctness and coherence of the notification content through the verification of logical relationships, thereby greatly improving the safety and reliability of aviation information processing.

[0061] Step S105 : determining the parsing accuracy of the target system for the NOTAM based on the similarity and the analysis result.

[0062] Optionally, in this application, the determination of parsing accuracy is not based solely on one evaluation criterion, but rather combines two methods: similarity detection and dependency grammar analysis. Similarity detection focuses on content matching at the text level. It intuitively reflects the consistency of the target announcement with the original announcement in text form by quantifying the structural and lexical similarities of the two texts. Dependency grammar analysis, on the other hand, focuses more on semantic and logical details, and ensures that the logical structure of the key information of the announcement is correctly reconstructed by checking word dependencies and grammatical roles.

[0063] For example, after obtaining the similarity score and dependency parsing results between the target and original notices, these two metrics can be considered together. The similarity score can serve as a rough quantitative indicator of the degree of literal match between the two notices, while the dependency parsing results provide a deeper perspective, helping us understand the semantic correctness of the notice content.

[0064] If the target and original notices achieve a high similarity score, and the dependency parsing results for both indicate consistent relationships and grammatical roles between vocabulary, then the target system can be judged to have a high parsing accuracy for the NOTAM, indicating that the parsing process effectively preserves the key information and logical structure of the notice. Conversely, if the similarity score is low, or if the dependency parsing reveals missing key information, confusing logic, or mismatched grammatical roles, this would indicate that the target system made significant errors in parsing the NOTAM. In this case, the parsing accuracy would be considered insufficient, and adjustments and optimizations would be required to the parsing system (i.e., the target system) or the inverse model to correct the errors and improve future parsing accuracy.

[0065] Optionally, Figure 3 This is a schematic diagram of an optional comparison agent execution process according to an embodiment of the present application, wherein the retrospective verification system can use the comparison agent to complete the comparison of the original message (i.e., the original navigation notice) and the target message reconstructed by the inverse model (i.e., the above-mentioned target notice), including: after the LLM (Large Language Model) model (corresponding to the above-mentioned inverse model) obtains the message parsing result of the original message, calling the comparison agent, then extracting key information from the message parsing result, and using the key information to reconstruct the target message ( Figure 3 This step is omitted in the previous example. The comparison agent then calculates the similarity between the original and target messages and performs dependency parsing on both messages to obtain the analysis results. Finally, the comparison agent uses the judgment rules to determine the consistency between the target and original messages, thereby verifying the correctness of the original message parsing.

[0066] In an optional embodiment, determining the parsing accuracy of the target system for the NOTAM based on the similarity and the analysis result includes: determining, based on the analysis result and the similarity, that the target system's parsing accuracy for the NOTAM is greater than a target threshold when it is detected that the target NOTAM and the NOTAM simultaneously meet the following judgment conditions, wherein the judgment conditions include:

[0067] The first condition is used to limit the deviation of the navigation time described in the target notice and the navigation notice to be less than or equal to a preset threshold;

[0068] The second condition is to ensure that there is no conflict between the geographical locations described in the target notice and the navigation notice;

[0069] The third condition is to ensure that the flight safety information described in the target notice and the navigation notice is consistent;

[0070] The fourth condition is used to limit the similarity between the target notice and the navigation notice to be greater than a preset similarity.

[0071] Optionally, the time information involved in the NOTAM, such as the effective time and expiration time, is crucial to aviation safety. Therefore, the first condition checks whether there is a slight deviation between the flight time described in the target NOTAM and the original NOTAM, and whether this deviation is within a preset time threshold. The setting of the time threshold must take into account the actual situation of aviation operations. For example, a difference of a few hours or days may be acceptable, while a deviation exceeding the preset threshold may affect the flight plan. Through this conditional restriction, the accurate transmission of time information is ensured, avoiding safety hazards caused by time differences.

[0072] Optionally, the geographic location information in the NOTAM, including airport codes, latitude and longitude coordinates, relates to the NOTAM's effective range and applicable area. The second condition ensures that the target NOTAM and the original NOTAM describe identical geographic locations. This is particularly important for preventing aircraft from entering restricted or hazardous areas.

[0073] Alternatively, the core purpose of a NOTAM is to convey flight safety-related changes, such as airport runway status, navigation equipment operating conditions, and adjustments to air traffic service procedures. The third condition requires that the target NOTAM fully matches the flight safety information described in the original NOTAM, ensuring that no critical safety instructions are omitted from the parsed results. Meeting this condition is crucial for evaluating parsing accuracy, as any misinterpretation or omission of flight safety information could have a serious impact on aviation operations.

[0074] Optionally, in addition to the consistency check of the above three dimensions, this application also introduces the evaluation of overall text similarity as an indicator for comprehensive judgment of parsing accuracy. The calculation of overall similarity can be performed through a deep learning model, which not only considers literal matching but also covers semantic understanding. The selection of the preset similarity threshold should be based on the complexity of the notice and industry standards to ensure that the parsed notice is highly semantically similar to the original text, thereby ensuring the integrity and accuracy of information transmission.

[0075] Finally, when both the target NOTAM and the original NOTAM (i.e., the original NOTAM) meet the aforementioned four conditions, the retroactive verification system can determine that the target system's NOTAM parsing accuracy exceeds the target threshold, indicating that the parsing process is of high quality. The design of the comprehensive judgment mechanism fully considers the characteristics of NOTAMs. By combining quantitative and logical judgment, it ensures that the parsed results are highly consistent with the original NOTAM in terms of time, location, safety information, and overall semantics. This mechanism not only improves verification efficiency but also significantly enhances the accuracy of flight safety information processing.

[0076] In an optional embodiment, the retrospective verification system may determine that the target system's parsing accuracy of the navigation notice is less than or equal to the target threshold based on the analysis results and similarity when it detects that the target notice and the navigation notice do not meet at least one of the first condition, the second condition, the third condition, and the fourth condition.

[0077] Optionally, when the target NOTAM does not meet the preset conditions with the original NOTAM in terms of time, geographic location, consistency of safety information, or overall similarity, the retrospective verification system will deem the parsing accuracy insufficient, which triggers further review procedures or optimization needs of the parsing system to continuously improve the safety and reliability of NOTAM processing.

[0078] In an optional embodiment, dependency parsing includes the following steps:

[0079] Identify the grammatical role of each word in the target notice and navigation notice; determine the dependency relationship between different words in the notice text sentence based on the grammatical role of each word and the position of each word in the notice text sentence; determine the semantic structure of the notice text sentence based on the dependency relationship between different words in the notice text sentence; extract the key semantic information of the notice text sentence based on the semantic structure of the notice text sentence; determine the analysis result based on the key semantic information.

[0080] Optionally, first, the retrospective verification system can scan each word in the target notice and the original navigation notice to determine its grammatical identity in the sentence. For example, it can identify whether a word is a noun, verb, adjective, or more specifically, whether the word acts as a subject, object, attributive, adverbial, etc. This recognition can determine the function of the word in the sentence. Next, the retrospective verification system can establish dependency links between words based on the grammatical role of each word and the relative position of the word in the sentence. Dependency relationships describe the direct connection between words in a sentence, such as "verb-object", "noun-attributive", etc. These links constitute the dependency tree of the notice sentence, which can clearly show the logical and grammatical ties between words.

[0081] Optionally, once the dependencies are determined, the retroactive verification system can construct a semantic structure framework for the announcement statement. The semantic structure reveals the relationships between key elements such as participants, actions, time and place of the event or state described in the announcement, as well as how these elements are combined to express specific meanings. Through the semantic structure, the retroactive verification system can more intuitively understand the internal logic of the announcement. After understanding the semantic structure of the announcement, the next step is to extract key semantic information. This includes identifying the core content in the announcement, such as changes in facility status, descriptions of time ranges, limitations on geographical areas, etc. Key semantic information is the core value of the announcement, and its accuracy and completeness directly affect the effectiveness of the announcement.

[0082] Finally, the backtracking verification system produces the final analysis results based on the extracted key semantic information. This includes a comprehensive understanding of the announcement sentence and an assessment of parsing accuracy. If the target announcement is consistent with the original announcement in terms of dependencies and key semantic information, the parsing process can be considered correct. Otherwise, it may indicate a problem in the parsing, requiring further verification and correction.

[0083] Through dependency parsing, the technical solution of this application ensures that the parsed NOTAM information is not only superficially consistent with the original text, but more importantly, accurate at a deeper semantic and logical level. This is crucial for ensuring flight safety and smooth aviation operations. The introduction of dependency parsing significantly improves the accuracy and reliability of the NOTAM intelligent processing system.

[0084] In an optional embodiment, the training steps of the inverse model include: obtaining multiple sample data pairs, wherein the sample data pairs include: historical navigation notices and parsed texts of the historical navigation notices; determining labels of the parsed texts, wherein the labels of the parsed texts are used to evaluate the parsing quality of the parsed texts for the historical navigation notices; screening target sample data pairs from the multiple sample data pairs based on the labels of the parsed texts; and performing multiple iterative training on the neural network based on the target sample data pairs to obtain the inverse model.

[0085] Optionally, in order to train the inverse model, a large number of historical NOTAMs and their corresponding parsed texts are first collected. Each NOTAM and parsed text constitute a sample data pair. These sample data pairs are the basis for training the inverse model. Next, the retrospective verification system can automatically perform a quality assessment on each parsed text and determine its label based on the assessment results. This label reflects the quality of the parsed text for the original NOTAM, which can be classified as "accurate", "partially accurate" or "inaccurate". The quality assessment is usually based on specific criteria, such as whether all key information is fully parsed, whether the description of time, place and events is correct, and whether the semantics of the text is consistent with the original notice. These labels provide the goal of model learning, that is, the model needs to learn to generate the original NOTAM corresponding to the "accurate" label when given a parsed text.

[0086] From all collected sample data pairs, the retrospective validation system selects those with the "accurate" label as training data, known as target sample data pairs. These samples represent high-quality parsing results and serve as excellent learning material for training the model to achieve high accuracy. By focusing on these target samples, the model learns to establish an accurate correspondence between the parsed text and the original NOTAM.

[0087] Finally, the backtracking validation system uses the selected target sample data pairs to train the neural network. The training process typically involves multiple iterations. In each iteration, the model attempts to infer the original NOTAM from the parsed text. By comparing the generated message with the actual original message, the model adjusts its parameters to minimize the loss function. As the number of iterations increases, the model gradually learns how to perform the inference task more accurately.

[0088] By using labeled historical NOTAMs and parsed results to train a neural network, the aim is to train an inverse model that can accurately reconstruct historical NOTAMs from parsed texts, thereby providing an effective way to verify the accuracy and completeness of NOTAM parsing results.

[0089] In an optional embodiment, in the process of performing multiple iterative training on the neural network based on the target sample data pairs to obtain the inverse model, the backtracking verification system can calculate the ratio of the current number of iterations to the set maximum number of iterations when performing the i-th iterative training on the neural network, where i is an integer greater than 1, and then determine the model learning rate during the i-th iterative training based on the ratio. Finally, the backtracking verification system performs the i-th iterative training based on the model learning rate.

[0090] Optionally, in the i-th iteration of model training, the backtracking verification system can calculate the current iteration number (e.g. ) and the preset maximum number of iterations The ratio r between them is Here, r can be considered an indicator of training progress, gradually increasing from 0 (start) to 1 (end). This ratio helps the backtracking verification system understand the current stage of training and thus determine the learning rate adjustment strategy.

[0091] Optionally, the backtracking verification system can adjust the learning rate based on the iteration ratio r, where the model learning rate is an important parameter in the model training process, which controls the amplitude of the model parameter update. In the early stages, a higher learning rate can speed up the convergence of the model, but as the training progresses, an excessively high learning rate may cause the model to oscillate around the optimal solution and be difficult to stabilize in the optimal state. Therefore, the system will adjust the learning rate based on the current iteration ratio r. The specific adjustment strategy can be referred to the following formula (1):

[0092]

[0093] in, is the lower limit of the learning rate in the later stage of iteration, Is the upper limit of the learning rate at the beginning of the iteration. The increase in learning rate It will show a trend of first high and then low. Helps the model learn quickly, while the lower This ensures that the model can fine-tune parameters to avoid overfitting and achieve optimal generalization capabilities.

[0094] After adjusting the learning rate, the backtracking validation system continues with the i-th iteration of training, using the parsed text and the corresponding original NOTAM from the target sample data pair as input and output pairs to train the neural network model. In each iteration, the model updates its parameters based on the quality of the parsed text and the degree of match between the inferred results and the inferred results, gradually improving its inferred capabilities until the maximum number of iterations is reached.

[0095] Optionally, throughout the training process, the backtesting validation system can continuously monitor the model's learning status, including metrics such as the trend of the loss function and the degree of match between the backtested NOTAMs and the original ones. If the model's performance improves slowly or shows signs of overfitting during training, the system can take measures, such as terminating training early, adding regularization terms, or adjusting training parameters, to optimize the model's performance and stability.

[0096] In summary, the backtracking verification system optimizes the training process of the neural network model by dynamically adjusting the learning rate, ensuring that the model can learn efficiently and gradually converge over multiple iterations, thereby obtaining a reverse prediction model that can accurately reverse the navigation notice.

[0097] In an optional embodiment, determining the model learning rate during the i-th iterative training based on the ratio includes: calculating the product of the ratio and pi to obtain a target value; calculating the cosine value of the target value; calculating the difference between the maximum learning rate and the minimum learning rate set for the neural network; and determining the model learning rate during the i-th iterative training based on the cosine value of the target value, the difference between the maximum learning rate and the minimum learning rate set for the neural network, and the minimum learning rate.

[0098] Optionally, as in formula (1), the backtracking verification system calculates the iteration progress ratio and pi The product of . is a constant, approximately equal to 3.14159. By multiplying the iterative progress ratio by pi, the target value can be obtained. Then, the system calculates the cosine of the target value The cosine function is widely used in mathematics and physics to describe periodic changes. In this case, the cosine value range is [-1, 1], and as The increase, It will increase from 0 to Then from Increase to , so the cosine value will first drop from 1 to -1, and then rise back to 1. In this way, the change of the cosine value intuitively reflects the cyclical change trend of the learning rate from high to low and then back up.

[0099] In an optional embodiment, before performing the i-th iterative training according to the model learning rate, the backtracking verification system can obtain the latest network parameters of the neural network before the i-th iterative training; detect the latest loss function value of the neural network before the i-th iterative training; determine the gradient information of the latest loss function value based on the latest network parameters; and determine the model parameter information updated during the i-th iterative training based on the latest network parameters, gradient information of the neural network before the i-th iterative training and the model learning rate during the i-th iterative training.

[0100] Optionally, formula (2) shows the updating process of model parameter information:

[0101]

[0102] As shown in formula (2), is the latest network parameter of the neural network before the i-th iteration training, Represents the latest loss function value of the neural network before the i-th iteration training; Indicates the gradient information of the latest loss function value based on the latest network parameters. The model parameter information updated during the i-th iteration training can be calculated using the above formula (2) .

[0103] In summary, the backtracking verification system ensures that each training iteration effectively updates model parameters and optimizes model performance by acquiring the latest neural network parameters, monitoring the loss function value, calculating gradient information, and adjusting the learning rate. This strategy not only helps improve the model's ability to infer NOTAM parsing results, but also ensures stable convergence during training, avoiding overfitting, and ultimately achieving automated and intelligent verification of NOTAM parsing results. Through continuous iterative training, the model's accuracy and efficiency in processing various complex NOTAM parsing results will be significantly improved.

[0104] In an optional embodiment, detecting the latest loss function value of the neural network before the i-th iterative training includes: obtaining the weight of each target sample data pair; obtaining the notice text predicted by the current neural network based on the parsed text in the target sample data pair; determining the target difference value between the notice text predicted by the current neural network and the actual historical navigation notice in the target sample data pair; and determining the latest loss function value of the neural network before the i-th iterative training based on the weight of each target sample data pair and the target difference value.

[0105] Alternatively, formula (3) is an optional way to calculate the loss function:

[0106]

[0107] In formula (3), Represents the weight of any target sample data pair, such as the i-th target sample data pair; represents the announcement text predicted by the current neural network based on the parsed text in the i-th target sample data pair; represents the actual historical NOTAM in the i-th target sample data pair. “N” in formula (3) refers to the total number of target data pairs.

[0108] Optionally, weights are assigned to each target sample data pair, allowing the system to differentiate the influence of different samples during training. For example, NOTAMs containing complex conditions and exceptions may be weighted higher, as correctly interpreting such NOTAMs is critical to flight safety. By assigning higher weights to key samples, the model pays more attention to them during training, enhancing its robustness and ability to handle complex NOTAMs.

[0109] By dynamically adjusting the weights in the loss function, the system ensures that the model adapts to the current learning state at each stage of training. Initially, the model may learn equally well for all samples, but as training progresses, certain samples may require more attention. Dynamic weight adjustment ensures that the model continuously optimizes its processing of key samples during learning, avoiding excessive attention to low-value samples. This facilitates efficient resource allocation and maximizes the model's learning efficiency within limited training time.

[0110] In an optional embodiment, determining the label of the parsed text includes: determining the label of the parsed text according to at least one of the following labeling rules:

[0111] The first marking rule is used to detect whether the field information under the key field of the parsed text is consistent with the field information under the key field of the historical navigation notice;

[0112] The second annotation rule is used to detect whether the content in the parsed text includes navigation-related information of the historical NOTAM;

[0113] The third annotation rule is used to detect whether the logical description of the parsed text in terms of time sequence, navigation range and geographical location is correct.

[0114] Optionally, the first annotation rule is mainly used to check whether the information of the key fields in the parsed text is consistent with the corresponding field information in the original historical navigation notice. The key fields may include but are not limited to airport code, effective time, restriction type, navigation equipment status, etc. By comparing the values ​​of these fields, the retrospective verification system can evaluate the accuracy of the parsing results in terms of basic information. For example, if the effective time of a notice is " "to" ", and the corresponding effective time field in the parsed text accurately reflects this information, then the parsed text will be given the label "accurate"; conversely, if the field information in the parsed result is incorrect or missing, the parsed text will be marked as "inaccurate".

[0115] Optionally, a second annotation rule checks whether the parsed text fully captures all the navigation-related information in the original historical NOTAM. This includes not only basic field information but also detailed descriptions within the NOTAM, such as runway maintenance status, temporary route changes, and navigation equipment availability. If the parsed text fully captures this navigation-related information, without omissions or deletions, it is marked as "complete." Otherwise, it is marked as "incomplete." This rule ensures that the model learns how to fully parse NOTAM content during training.

[0116] Optionally, the third annotation rule is used to check whether the logical descriptions in the parsed text, such as time sequence, flight range, and geographic location, are consistent with the original notice. For example, if a notice describes an airport as closed at night, but the parsed text omits the time zone information, resulting in a logically incorrect time description, the parsed text will be labeled "logically incorrect description." By identifying and annotating these logical errors, the model learns how to accurately understand and parse complex logical descriptions during training, which is particularly important for processing notices involving time, space, and conditions.

[0117] By labeling the parsed text using the above rules, we provide the neural network with a series of labeled sample data, which we use to train the model to infer the original NOTAM from the parsed results and judge the quality of the parsed results. These labels help the model distinguish between correct and incorrect parsed results, thereby optimizing the model structure and parameters, making it more accurate and reliable when handling the inference and verification of NOTAM parsed results.

[0118] In an optional embodiment, the retrospective verification system may include the following modules:

[0119] Data collection module: responsible for collecting the results of navigation notice analysis. This module connects with various navigation notice analysis systems to obtain the analyzed information and make preliminary annotations on the quality of the analysis results.

[0120] Computing and processing module: This module carries out computational tasks for model training and data validation. During the model training phase, it uses model micro-debugging technology to fine-tune the large language model. During the data validation phase, it runs the fine-tuned model to verify whether messages can be correctly generated in the test set.

[0121] Storage module: used to store various data such as NOTAM analysis results, original message data, training data, fine-tuned models, and comparison results.

[0122] Comparison execution module: Built-in comparison agent, using a large model to compare two message information to determine whether they are consistent.

[0123] Verification tool module: Integrates the back-inference model and comparison agent, receives the message parsing results and the original message as input, and outputs the evaluation results.

[0124] Optionally, by connecting to various NOTAM parsing systems through the data acquisition module, the retrospective verification system can obtain structured parsing results (JSON format) in real time. Automated annotation tools are used to assess the quality of the parsing results.

[0125] Optionally, during the model fine-tuning and verification phase, the following steps are mainly included:

[0126] Collect data: Collect the NOTAM parsing results with high-quality annotations, including the original NOTAM messages and parsed results.

[0127] Data cleaning and processing: The collected data is further cleaned and processed, including unifying the encoding format, clearing unnecessary blank characters, and aligning the parsing results and original messages.

[0128] Model selection: Due to its outstanding performance on text tasks, the large language model can be selected as the base model.

[0129] Construct data pairs: The data pairs contain [input: parsing result, output: original message].

[0130] Model fine-tuning: Set appropriate training parameters to optimize model performance. Please refer to formula (1), formula (2) and formula (3).

[0131] Optionally, the backtracking verification system can extract key information from the message results based on the fine-tuned and trained large model as the base of the Agent; the fine-tuned model is used to extract key information, including but not limited to the status of airport facilities, the working status of navigation equipment, etc.

[0132] For example, refer to formula (4):

[0133]

[0134] Where k is the set of extracted key information, M is the fine-tuned large model, and T is the parsed result of the input text, i.e., the navigation notice. The loss value t based on cosine similarity is calculated as shown in formula (5):

[0135]

[0136] Here, A represents the key information extracted by the comparison agent, and B represents the original message. Dependency parsing tools are then used to analyze the sentence structure and obtain the dependency relationships and grammatical roles of each word. The following consistency judgment rules are formulated based on the loss function and the results of dependency parsing:

[0137]

[0138] Optionally, the retroactive verification system can also define clear API endpoints to handle different requests. For example, set POST / compare: Receive the original message and the retroactively generated message, and return the comparison results. GET / report / {id}: Get the comparison report based on the comparison task ID. The retroactive verification system can also use a thread pool to manage concurrent requests to ensure that system resources are effectively utilized. Each comparison task is assigned to an independent thread or coroutine for processing. For tasks that take a long time, an asynchronous processing mechanism is used to put the task into a background queue for execution and notify the client through a callback.

[0139] According to one aspect of the embodiment of the present application, a retrospective verification device for NOTAM parsing results is also provided, wherein: Figure 4 is a schematic diagram of an optional retrospective verification device according to an embodiment of the present application, such as Figure 4 As shown, the backtracking verification device includes: an acquisition unit 401, which is used to obtain the parsing result of the target system on the navigation notice; a first processing unit 402, which is used to determine the key information of the notice from the parsing result through an inverse model, and reconstruct and generate the target notice based on the key information of the notice, wherein the key information of the notice includes at least navigation time information and geographical location information; a detection unit 403, which is used to detect the similarity between the target notice and the navigation notice; an analysis unit 404, which is used to perform dependency grammar analysis on the target notice and the navigation notice to obtain an analysis result, wherein the dependency grammar analysis is used to analyze the word dependency relationship and grammatical role relationship between the target notice and the navigation notice; a second processing unit 405, which is used to determine the parsing accuracy of the navigation notice by the target system based on the similarity and the analysis result.

[0140] Optionally, the second processing unit 405 includes: a first determination subunit, used to determine, based on the analysis results and similarity, that the target system's parsing accuracy of the navigation notice is greater than a target threshold when it is detected that the target notice and the navigation notice simultaneously meet the following judgment conditions, wherein the judgment conditions include: a first condition, used to limit the navigation time deviation described in the target notice and the navigation notice to be less than or equal to a preset threshold; a second condition, used to limit the navigation geographical location ranges described in the target notice and the navigation notice to be free of conflict; a third condition, used to limit the flight safety information described in the target notice and the navigation notice to be consistent; and a fourth condition, used to limit the similarity between the target notice and the navigation notice to be greater than a preset similarity.

[0141] Optionally, the second processing unit 405 includes: a second determination subunit, which is used to determine, based on the analysis results and the similarity, that the parsing accuracy of the target system for the navigation notice is less than or equal to the target threshold when it is detected that the target notice and the navigation notice do not meet at least one of the first condition, the second condition, the third condition and the fourth condition.

[0142] Optionally, the retrospective verification device for the NOTAM parsing result is further configured to perform the following steps:

[0143] Identify the grammatical role of each word in the target notice and navigation notice; determine the dependency relationship between different words in the notice text sentence based on the grammatical role of each word and the position of each word in the notice text sentence; determine the semantic structure of the notice text sentence based on the dependency relationship between different words in the notice text sentence; extract the key semantic information of the notice text sentence based on the semantic structure of the notice text sentence; determine the analysis result based on the key semantic information.

[0144] Optionally, the retrospective verification device for the NOTAM parsing result is further configured to perform the following steps:

[0145] A plurality of sample data pairs are obtained, wherein the sample data pairs include: a historical NOTAM and a parsed text of the historical NOTAM; a label of the parsed text is determined, wherein the label of the parsed text is used to evaluate the parsing quality of the parsed text for the historical NOTAM; a target sample data pair is screened from the plurality of sample data pairs based on the label of the parsed text; and a neural network is iteratively trained multiple times based on the target sample data pair to obtain an inverse model.

[0146] Optionally, the retrospective verification device for the navigation notice parsing result also includes: a first calculation unit, used to calculate the ratio of the current number of iterations to the set maximum number of iterations when the neural network is trained for the i-th iterative training, where i is an integer greater than 1; a first determination unit, used to determine the model learning rate during the i-th iterative training based on the ratio; and a first training unit, used to perform the i-th iterative training based on the model learning rate.

[0147] Optionally, the first determination unit includes: a first calculation subunit, used to calculate the product of the ratio and pi to obtain a target value; a second calculation subunit, used to calculate the cosine value of the target value; a third calculation subunit, used to calculate the difference between the maximum learning rate and the minimum learning rate set for the neural network; the first determination subunit, used to determine the model learning rate during the i-th iterative training based on the cosine value of the target value, the difference between the maximum learning rate and the minimum learning rate set for the neural network, and the minimum learning rate.

[0148] Optionally, the retrospective verification device for the parsing results of the navigation notice also includes: a first acquisition unit, used to obtain the latest network parameters of the neural network before the i-th iterative training; a first detection unit, used to detect the latest loss function value of the neural network before the i-th iterative training; a second determination unit, used to determine the gradient information of the latest loss function value based on the latest network parameters; and a third determination unit, used to determine the model parameter information updated during the i-th iterative training based on the latest network parameters, gradient information of the neural network before the i-th iterative training and the model learning rate during the i-th iterative training.

[0149] Optionally, the first detection unit includes: a first acquisition subunit, used to obtain the weight of each target sample data pair; a second acquisition subunit, used to obtain the notice text predicted by the current neural network based on the parsed text in the target sample data pair; a second determination subunit, used to determine the target difference value between the notice text predicted by the current neural network and the actual historical navigation notice in the target sample data pair; and a third determination subunit, used to determine the latest loss function value of the neural network before the i-th iteration training based on the weight of each target sample data pair and the target difference value.

[0150] Optionally, determining the label of the parsed text includes: determining the label of the parsed text according to at least one of the following labeling rules: a first labeling rule for detecting whether the field information of the parsed text under the key field is consistent with the field information of the historical navigation notice under the key field; a second labeling rule for detecting whether the content in the parsed text includes the navigation-related information of the historical navigation notice; a third labeling rule for detecting whether the logical description of the parsed text in terms of time sequence, navigation range and geographical location is correct.

[0151] According to another aspect of an embodiment of the present application, a retrospective verification device for parsing results of a navigation notice is also provided, including: an acquisition unit, used to obtain the parsing result of the navigation notice by the target system; a first processing unit, used to determine the key information of the notice from the parsing result through an inverse model, and reconstruct and generate a target notice based on the key information of the notice, wherein the key information of the notice includes at least navigation time information and geographic location information; a detection unit, used to detect the similarity between the target notice and the navigation notice; an analysis unit, used to perform dependency grammar analysis on the target notice and the navigation notice to obtain an analysis result, wherein the dependency grammar analysis is used to analyze the word dependency relationship and grammatical role relationship between the target notice and the navigation notice; a second processing unit, used to determine the parsing accuracy of the navigation notice by the target system based on the similarity and the analysis result.

[0152] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned retrospective verification method of the navigation notice parsing result.

[0153] According to another aspect of an embodiment of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors execute the above-mentioned retrospective verification method of the navigation notice parsing results.

[0154] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0155] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0157] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program code.

[0160] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A retrospective verification method for NOTAM analysis results, characterized in that: include: Obtain the target system's analysis results of the NOTAM; Determining key information of the announcement from the analysis results through an inverse model, and reconstructing and generating a target announcement based on the key information of the announcement, wherein the key information of the announcement includes at least navigation time information and geographic location information; detecting similarity between the target notice and the NOTAM; performing dependency grammar analysis on the target notice and the NOTAM to obtain an analysis result, wherein the dependency grammar analysis is used to analyze word dependency relationships and grammatical role relationships between the target notice and the NOTAM; The parsing accuracy of the target system for the NOTAM is determined based on the similarity and the analysis result, including: if the similarity between the target NOTAM and the original NOTAM is greater than a preset similarity, and the dependency grammatical analysis results of both parties show that the relationship between the vocabulary and the grammatical roles are consistent, then it is determined that the parsing accuracy of the target system for the NOTAM is greater than a preset threshold, indicating that the target system has effectively retained the key information and logical structure of the NOTAM during the parsing process of the NOTAM; if the similarity between the target NOTAM and the original NOTAM is less than or equal to the preset similarity, or the dependency grammatical analysis result reveals the absence of key information, logical confusion or grammatical role mismatch, then it is determined that the target system has an error in parsing the NOTAM, and the target system is adjusted.

2. The method according to claim 1, characterized in that Determining the parsing accuracy of the target system for the NOTAM according to the similarity and the analysis result includes: Based on the analysis result and the similarity, if it is detected that the target notice and the navigation notice simultaneously meet the following judgment conditions, it is determined that the parsing accuracy of the target system for the navigation notice is greater than a target threshold, wherein the judgment conditions include: The first condition is used to limit the deviation of the navigation time described in the target notice and the navigation notice to be less than or equal to a preset threshold; The second condition is used to ensure that there is no conflict between the navigation geographical location ranges described in the target notice and the navigation notice; The third condition is used to ensure that the flight safety information described in the target notice and the navigation notice is consistent; The fourth condition is used to limit the similarity between the target notice and the navigation notice to be greater than a preset similarity.

3. The method according to claim 2, characterized in that Determining the parsing accuracy of the target system for the NOTAM according to the similarity and the analysis result includes: Based on the analysis result and the similarity, when it is detected that the target notice and the navigation notice do not meet at least one of the first condition, the second condition, the third condition and the fourth condition, it is determined that the resolution accuracy of the target system for the navigation notice is less than or equal to the target threshold.

4. The method according to claim 1, wherein The dependency grammar analysis includes the following steps: identifying a grammatical role of each word in the target notice and the NOTAM; Determining dependency relationships between different words in the announcement text sentence according to the grammatical role of each word and the position of each word in the announcement text sentence; determining the semantic structure of the announcement text sentence according to the dependency relationship between different words in the announcement text sentence; Extracting key semantic information of the notice text sentence according to the semantic structure of the notice text sentence; The analysis result is determined according to the key semantic information.

5. The method according to claim 1, wherein The training steps of the inverse model include: Acquire a plurality of sample data pairs, wherein the sample data pairs include: historical NOTAMs and parsed texts of the historical NOTAMs; Determining a label of the parsed text, wherein the label of the parsed text is used to evaluate the parsing quality of the parsed text for the historical NOTAM; Filtering a target sample data pair from a plurality of sample data pairs according to the label of the parsed text; According to the target sample data pairs, the neural network is iteratively trained multiple times to obtain the inverse model.

6. The method according to claim 5, characterized in that In the process of performing multiple iterative training on the neural network according to the target sample data pairs to obtain the inverse model, the method further includes: When performing the i-th iteration training on the neural network, calculating the ratio of the current number of iterations to the set maximum number of iterations, where i is an integer greater than 1; Determine the model learning rate during the i-th iterative training according to the ratio; The i-th iteration training is performed according to the model learning rate.

7. The method according to claim 6, characterized in that Determining the model learning rate during the i-th iterative training according to the ratio includes: Calculating the product of the ratio and pi to obtain a target value; Calculating the cosine value of the target value; Calculating the difference between the maximum learning rate and the minimum learning rate set for the neural network; The model learning rate during the i-th iterative training is determined according to the cosine value of the target value, the difference between the maximum learning rate and the minimum learning rate set by the neural network, and the minimum learning rate.

8. The method according to claim 6, characterized in that Before performing the i-th iterative training according to the model learning rate, the method further includes: Obtaining the latest network parameters of the neural network before the i-th iterative training; Detecting the latest loss function value of the neural network before the i-th iterative training; Determining the gradient information of the latest loss function value based on the latest network parameters; The model parameter information updated during the i-th iterative training is determined based on the latest network parameters of the neural network before the i-th iterative training, the gradient information, and the model learning rate during the i-th iterative training.

9. The method according to claim 8, characterized in that Detecting the latest loss function value of the neural network before the i-th iteration training, including: Obtaining the weight of each target sample data pair; Obtaining the notification text predicted by the current neural network based on the parsed text in the target sample data pair; Determining a target difference between the notification text predicted by the current neural network and the actual historical NOTAMs in the target sample data pair; Determine the latest loss function value of the neural network before the i-th iterative training according to the weight of each target sample data pair and the target difference value.

10. The method according to claim 5, characterized in that Determining a label for the parsed text includes: The label of the parsed text is determined according to at least one of the following labeling rules: A first marking rule is used to detect whether the field information under the key field of the parsed text is consistent with the field information under the key field of the historical navigation notice; A second marking rule is used to detect whether the content in the parsed text includes the navigation-related information of the historical navigation notice; The third annotation rule is used to detect whether the logical description of the time sequence, navigation range and geographical location of the parsed text is correct.

11. A retrospective verification device for NOTAM analysis results, characterized in that: include: An acquisition unit is used to obtain the parsing result of the target system on the navigation notice; a first processing unit, configured to determine key information of the notification from the parsing result through an inverse model, and reconstruct and generate a target notification based on the key information of the notification, wherein the key information of the notification includes at least navigation time information and geographic location information; a detection unit, configured to detect the similarity between the target notice and the navigation notice; an analyzing unit, configured to perform dependency grammar analysis on the target notice and the NOTAM to obtain an analysis result, wherein the dependency grammar analysis is used to analyze word dependency relationships and grammatical role relationships between the target notice and the NOTAM; The second processing unit is used to determine the parsing accuracy of the target system for the NOTAM based on the similarity and the analysis result, including: if the similarity between the target NOTAM and the original NOTAM is greater than a preset similarity, and the dependency grammar analysis results of both parties show that the relationship between the vocabulary and the grammatical roles are consistent, then it is determined that the parsing accuracy of the target system for the NOTAM is greater than a preset threshold, indicating that the target system has effectively retained the key information and logical structure of the NOTAM during the parsing process of the NOTAM; if the similarity between the target NOTAM and the original NOTAM is less than or equal to the preset similarity, or the dependency grammar analysis result reveals the absence of key information, logical confusion or grammatical role mismatch, then it is determined that the target system has an error in parsing the NOTAM, and the target system is adjusted.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the retrospective verification method of the navigation notice parsing result according to any one of claims 1 to 10.

13. An electronic device, characterized in that: The invention comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the retrospective verification method of the navigation notice parsing result according to any one of claims 1 to 10.

Images