Backtracking verification method and device for navigation announcement analysis result and electronic equipment

Abstract

The invention discloses a backtracking verification method and device for a navigation announcement analysis result and electronic equipment, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring an analysis result of a target system on a navigation announcement; determining announcement key information from the analysis result through a backstepping model, and reconstructing and generating a target announcement based on the announcement key information; detecting the similarity between the target announcement and the navigation announcement; dependency grammar analysis is carried out on the target announcement and the navigation announcement to obtain an analysis result, and the dependency grammar analysis is used for analyzing a word dependency relationship and a grammar role relationship between the target announcement and the navigation announcement; and determining the analysis accuracy of the target system to the navigation announcement according to the similarity and the analysis result. According to the method and the device, the technical problem that the verification efficiency of the navigation announcement analysis result is relatively low and the aviation safety is influenced due to the fact that the expert rule is mainly utilized to verify the navigation announcement analysis result in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a method, device and electronic device for retrospectively verifying the parsing results of a navigation notice. Background Art

[0002] As a key source of information to ensure safe aviation operations, NOTAMs comprehensively cover all factors that affect flight safety within the flight information region, such as the status of airport facilities, the working conditions of navigation equipment, changes in air traffic service procedures, etc. This information is of great significance to flight safety and the normal operation of flights, and can assist pilots and relevant staff to promptly grasp various situations that may affect flight.

[0003] As the civil aviation industry develops rapidly and the number of flights continues to rise, the number and complexity of navigation notices are also increasing. In the past, the traditional way of processing navigation notices mainly relied on manual analysis and verification. However, this method is not only inefficient, but also prone to human errors. Manual analysis and verification requires a lot of time and manpower, especially when faced with massive and complex navigation notices. Problems such as "mistakes, omissions" and difficulty in updating frequently occur, posing hidden dangers to aviation safety.

[0004] From the above content, it can be seen that the current verification methods are mostly to check the analysis results with the help of manual or simple automated tools. This method is difficult to fully cover all possible types of errors, especially for complex navigation notices, which are prone to omissions or misjudgments. In addition, the existing verification methods lack effective verification methods for the comprehensiveness and accuracy of the analysis results, and cannot guarantee the integrity and correctness of the analysis results, nor can they achieve automatic output of the verification results. Manual intervention not only increases the workload, resulting in low verification efficiency of navigation notice analysis results, but may also cause inconsistencies and delays in the verification results, thereby affecting aviation safety.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present application provide a method, device and electronic device for retrospectively verifying the results of NOTAM parsing, so as to at least solve the technical problem that the prior art mainly uses expert rules to verify the results of NOTAM parsing, resulting in low verification efficiency of the NOTAM parsing results, thereby affecting aviation safety.

[0007] According to one aspect of an embodiment of the present application, a retrospective verification method for parsing results of a NOTAM is provided, comprising: obtaining parsing results of a target system for the NOTAM; determining key information of the NOTAM from the parsing results through a reverse model, and reconstructing and generating a target NOTAM based on the key information of the NOTAM, wherein the key information of the NOTAM includes at least navigation time information and geographic location information; detecting the similarity between the target NOTAM and the NOTAM; performing dependency grammar analysis on the target NOTAM and the NOTAM to obtain analysis results, wherein the dependency grammar analysis is used to analyze word dependency relationships and grammatical role relationships between the target NOTAM and the NOTAM; and determining the parsing accuracy of the NOTAM by the target system based on the similarity and the analysis results.

[0008] Optionally, determining the parsing accuracy of the target system for the navigation notice based on the similarity and the analysis result includes: based on the analysis result and the similarity, when it is detected that the target notice and the navigation notice simultaneously meet the following judgment conditions, determining that the parsing accuracy of the target system for the navigation notice is greater than a target threshold, wherein the judgment conditions include: a first condition, used to limit the navigation time deviation described in the target notice and the navigation notice to be less than or equal to a preset threshold; a second condition, used to limit the navigation geographical location ranges described in the target notice and the navigation notice to be free of conflict; a third condition, used to limit the flight safety information described in the target notice and the navigation notice to be consistent; and a fourth condition, used to limit the similarity between the target notice and the navigation notice to be greater than a preset similarity.

[0009] Optionally, the parsing accuracy of the target system for the navigation notice is determined based on the similarity and the analysis results, including: based on the analysis results and the similarity, when it is detected that the target notice and the navigation notice do not satisfy at least one of the first condition, the second condition, the third condition and the fourth condition, determining that the parsing accuracy of the target system for the navigation notice is less than or equal to the target threshold.

[0010] Optionally, dependency grammatical analysis includes the following steps: identifying the grammatical role of each word in the target notice and the navigation notice; determining the dependency relationship between different words in the notice text sentence based on the grammatical role of each word and the position of each word in the notice text sentence; determining the semantic structure of the notice text sentence based on the dependency relationship between different words in the notice text sentence; extracting key semantic information of the notice text sentence based on the semantic structure of the notice text sentence; and determining the analysis result based on the key semantic information.

[0011] Optionally, the training step of the inverse model includes: obtaining multiple sample data pairs, wherein the sample data pairs include: historical navigation notices and parsed texts of the historical navigation notices; determining labels of the parsed texts, wherein the labels of the parsed texts are used to evaluate the parsing quality of the parsed texts for the historical navigation notices; based on the labels of the parsed texts, screening out target sample data pairs from the multiple sample data pairs; and based on the target sample data pairs, performing multiple iterative training on the neural network to obtain the inverse model.

[0012] Optionally, in the process of performing multiple iterative training on the neural network according to the target sample data pairs to obtain the inverse model, when the neural network is subjected to the i-th iterative training, the ratio of the current number of iterations to the set maximum number of iterations is calculated, where i is an integer greater than 1; the model learning rate during the i-th iterative training is determined according to the ratio; and the i-th iterative training is performed according to the model learning rate.

[0013] Optionally, determining the model learning rate during the i-th iterative training according to the ratio includes: calculating the product of the ratio and pi to obtain a target value; calculating the cosine value of the target value; calculating the difference between the maximum learning rate and the minimum learning rate set for the neural network; determining the model learning rate during the i-th iterative training according to the cosine value of the target value, the difference between the maximum learning rate and the minimum learning rate set for the neural network, and the minimum learning rate.

[0014] Optionally, before performing the i-th iterative training according to the model learning rate, the latest network parameters of the neural network before the i-th iterative training are obtained; the latest loss function value of the neural network before the i-th iterative training is detected; the gradient information of the latest loss function value based on the latest network parameters is determined; and the model parameter information updated during the i-th iterative training is determined based on the latest network parameters and gradient information of the neural network before the i-th iterative training and the model learning rate during the i-th iterative training.

[0015] Optionally, detecting the latest loss function value of the neural network before the i-th iteration training includes: obtaining the weight of each target sample data pair; obtaining the notice text predicted by the current neural network based on the parsed text in the target sample data pair; determining the target difference value between the notice text predicted by the current neural network and the actual historical navigation notices in the target sample data pair; and determining the latest loss function value of the neural network before the i-th iteration training based on the weight of each target sample data pair and the target difference value.

[0016] Optionally, determining the label of the parsed text includes: determining the label of the parsed text according to at least one of the following labeling rules: a first labeling rule is used to detect whether the field information of the parsed text under the key field is consistent with the field information of the historical navigation notice under the key field; a second labeling rule is used to detect whether the content in the parsed text includes the navigation-related information of the historical navigation notice; a third labeling rule is used to detect whether the logical description of the parsed text in terms of time sequence, navigation range and geographical location is correct.

[0017] According to another aspect of the embodiment of the present application, a retrospective verification device for parsing results of navigation notices is also provided, including: an acquisition unit, used to obtain the parsing results of the navigation notice by the target system; a first processing unit, used to determine the key information of the notice from the parsing results through a reverse model, and reconstruct and generate the target notice based on the key information of the notice, wherein the key information of the notice includes at least navigation time information and geographic location information; a detection unit, used to detect the similarity between the target notice and the navigation notice; an analysis unit, used to perform dependency grammar analysis on the target notice and the navigation notice to obtain an analysis result, wherein the dependency grammar analysis is used to analyze the word dependency relationship and grammatical role relationship between the target notice and the navigation notice; a second processing unit, used to determine the parsing accuracy of the navigation notice by the target system based on the similarity and the analysis result.

[0018] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned retrospective verification method of the parsing result of the navigation notice.

[0019] According to another aspect of an embodiment of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors execute the above-mentioned retrospective verification method of the navigation notice parsing results.

[0020] From the above content, it can be seen that based on the technical solution provided by this application, compared with the manual verification method relying on expert rules, the reverse inference model can automatically extract key information from the analysis results, including but not limited to flight time information and geographic location information, and quickly reconstruct the target notice without manual intervention. This automated processing greatly shortens the verification cycle, ensures that key information can be verified and confirmed in the first place, and meets the strict requirements of flight operations for information timeliness.

[0021] In addition, the combination of the inverse model and dependency grammar analysis technology improves the accuracy and comprehensiveness of the verification of parsing results. Based on a large amount of training data, the model can capture the subtle and complex semantic logic in the navigation notice, avoiding the implicit information that may be ignored by traditional expert rules. Dependency grammar analysis further verifies the consistency of the notice from the syntactic structure level, ensuring that even in the face of complex contexts or non-standard expressions, the correctness of key information can be accurately judged. This two-layer verification mechanism effectively makes up for the blind spots and errors that may exist in a single expert rule. Through the automated verification mechanism, this application eliminates the "forgetting and missing" problems caused by fatigue, negligence or differences in rule understanding during manual verification, and significantly reduces aviation safety risks. The high similarity between the reconstructed target notice and the original notice and the consistency results of the dependency grammar analysis ensure the reliability and integrity of information transmission, providing pilots and related personnel with more accurate navigation information, thereby avoiding potential flight safety hazards caused by information errors or delays.

[0022] Finally, compared to fixed expert rules, the inverse model based on deep learning has the ability of self-learning and iteration. This means that in the face of new message formats or changes in aviation information, the model can continuously improve its parsing and verification performance through continuous fine-tuning and optimization. This flexibility and adaptability enables the system to seamlessly integrate into new information processing processes without interrupting services, and better serve the intelligent upgrade of the aviation field. Moreover, the automated verification method eliminates the subjective judgment differences that may occur in manual verification, and achieves a high degree of standardization and consistency in the verification of navigation notice parsing results. No matter when and where, as long as the same parsing results are input, the system can produce consistent verification results, avoiding confusion and contradictions caused by regional differences or personal interpretations, and providing unified and reliable protection for aviation safety on a global scale.

[0023] In summary, the navigation notice parsing result verification method based on the inverse model and dependency grammar analysis proposed in this application not only solves the problems of low efficiency and error-proneness of traditional expert rule verification, but also greatly improves the efficiency and security of aviation information processing while ensuring the accuracy and comprehensiveness of information, providing strong technical support for information management and security assurance in the modern aviation industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 is a flowchart of an optional retrospective verification method of NOTAM parsing results according to an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of an optional training process of a reverse inference model according to an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of an optional comparison Agent execution process according to an embodiment of the present application;

[0028] Figure 4 It is a schematic diagram of an optional backtracking verification device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] It should also be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected by this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation portals are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0032] According to an embodiment of the present application, an embodiment of a method for retrospective verification of navigation notice parsing results is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0033] In an optional embodiment, a retrospective verification system for the parsing results of a navigation notice (hereinafter referred to as the retrospective verification system) can be used as the execution subject of the retrospective verification method for the parsing results of the navigation notice of the embodiment of the present application, wherein the retrospective verification system can be a software system or an embedded system combining software and hardware. Of course, those skilled in the art should know that in addition to using the retrospective verification system as the execution subject, other forms can also be used, such as the execution subject of a device or equipment to execute the retrospective verification method for the parsing results of the navigation notice, and the embodiment of the present application does not specifically limit the specific form of expression of the method execution subject.

[0034] For the sake of convenience, the following describes the solution using the backtracking verification system as the execution entity.

[0035] Figure 1 is a flowchart of an optional retrospective verification method of the analysis result of the navigation notice according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0036] Step S101, obtaining the parsing result of the target system on the navigation notice.

[0037] Optionally, the target system refers to an automated software system or platform specifically used to parse NOTAM. Such a system usually has text processing and information extraction capabilities, and can read the original NOTAM messages issued by aviation agencies and convert them into structured data, such as JSON format, for further processing and analysis.

[0038] A NOTAM is a standardized aviation information notification that contains information that is critical to flight safety, such as temporary closure of airport runways, failure of navigation equipment, changes in air traffic rules, etc. The target system can use natural language processing technology, including text parsing, keyword recognition, semantic understanding, etc., to analyze and understand the content of the NOTAM message layer by layer, thereby extracting key information.

[0039] After the target system completes the parsing of the NOTAM, the generated parsing result is in the form of structured data, which usually includes navigation time information, geographic location information, restrictions, impact range, etc. This information is organized in a preset format and structure to facilitate subsequent verification and comparison operations.

[0040] Optionally, the process of the backtracking verification system obtaining the parsing results from the target system is often implemented through an automated interface to ensure timely data transmission and seamless connection. This means that the backtracking verification system can instantly receive the latest parsing results sent by the target system without manual downloading or uploading, which improves the efficiency and response speed of the overall process. In addition, after obtaining the parsing results, the backtracking verification system can also perform a series of data preprocessing tasks, including data cleaning, format conversion, information classification, etc., to ensure the accuracy and consistency of the data, and provide high-quality input for subsequent inverse model verification and dependency grammar analysis.

[0041] Optionally, in the process of obtaining the analysis results, special attention should be paid to the security and integrity of the data. Ensure the security encryption of the data transmission channel to prevent information from being tampered with or leaked during transmission. At the same time, check the integrity of the analysis results to confirm that all key information has been correctly extracted to avoid verification errors caused by missing data.

[0042] Step S102, determining the key information of the announcement from the parsing result through the inverse model, and reconstructing and generating the target announcement based on the key information of the announcement.

[0043] In step S102, the key information announced includes at least the flight time information and the geographical location information.

[0044] Optionally, a NOTAM usually contains a lot of key information, such as airport code, effective time, expiration time, geographic coordinates, facility change status, etc. After receiving the original NOTAM, the target system (i.e., NOTAM parsing system) will perform structured parsing on it, extract these key information, and convert the unstructured message into a structured data format, such as JSON format, for subsequent processing and verification. The inverse model is a deep learning model that has been specially trained to reversely generate the original message content from the structured parsing results. This process first requires the model to understand the key data points in the parsing results, including flight time information (such as effective and expiration time) and geographic location information (such as airport coordinates, impact range). Through analysis and learning, the model understands the rules and patterns of how to reconstruct the notification message based on these data points. At this stage, the inverse model will carefully analyze the structured parsing results to identify and extract key information. This information is not limited to time and location, but also includes the specific content of the notice (such as facility status, service changes, etc.), conditional statements (such as restrictions within a specific time range), and any details that may affect flight safety. This process ensures that all necessary information is captured accurately, laying the data foundation for subsequent reconstruction generation.

[0045] Optionally, based on the identified key information, the inverse model will also attempt to reconstruct a notification message. This generated message is intended to mimic the content and format of the original message, but is derived from key data points of the parsed results. The model will consider the grammatical rules of the notification, industry standard expressions, and possible contextual logic to ensure that the generated message is not only close to the original message in form, but also highly consistent in semantic content.

[0046] in, Figure 2 is a schematic diagram of an optional training process of a reverse inference model according to an embodiment of the present application, such as Figure 2 As shown, first prepare an initial model, and then fine-tune the initial model by inputting data pairs into the initial model. After the model is fine-tuned, the model can be expanded using sample data, and finally an inverse model is obtained.

[0047] From the above content, it can be seen that by determining key information through the inverse model and reconstructing the process of generating the target notice, the intelligent verification of the navigation notice parsing results is realized, ensuring the accurate transmission of key information, thereby greatly improving flight safety and flight operation efficiency.

[0048] Step S103, detecting the similarity between the target notice and the navigation notice.

[0049] Optionally, similarity detection can be implemented by a variety of methods, including but not limited to: text-based string matching technology, word vector-based cosine similarity, and deep learning-based sequence alignment network.

[0050] In this application, considering the professionalism and complexity of NOTAM, a pre-trained deep model can also be used to calculate the similarity score between the target NOTAM and the NOTAM. This process involves converting the text into an embedding vector and then measuring the similarity of the text by the mathematical distance or angle between the vectors.

[0051] In addition, similarity detection should not only focus on the consistency of the notice content, but also take into account the similarity of the format. Navigation notices follow a specific international standard format, including but not limited to the standard format mark at the beginning, the order of presentation of key information, and specific abbreviations and coding rules. Therefore, similarity calculation is not only about comparing the text content, but also about ensuring that the generated target notice is consistent with the original notice in format, which will directly affect the understanding and use of the information by aviation personnel.

[0052] Optionally, after completing the similarity calculation, a reasonable threshold needs to be set to determine whether the generated target notice is close enough to the original notice. The setting of this threshold needs to take into account the characteristics of navigation notices and the industry's high requirements for information accuracy. If the similarity score between the target notice and the original notice is higher than the set threshold, it indicates that the information reconstructed by the inverse model is relatively accurate; conversely, if the score is lower than the threshold, it may mean that information loss or deformation has occurred during the parsing process, which requires further inspection and correction.

[0053] Alternatively, NOTAMs often contain rich contextual information that is crucial to understanding the meaning of the NOTAM. Therefore, similarity detection should adopt a context-sensitive approach, that is, it should be able to understand the specific meaning of the words in the NOTAM and their role in the context, rather than just based on literal similarity. The semantic understanding ability of the large model can ensure that the similarity between two NOTAMs can be accurately assessed even in complex contexts or industry-specific abbreviations.

[0054] In summary, calculating the similarity between the target notice and the original navigation notice through the deep model can provide a quantitative indicator to judge the accuracy of the notice parsing and reconstruction. This process is a key link in the navigation notice intelligent verification system, which not only simplifies the traditional expert rule comparison process, but also improves the reliability and efficiency of the comparison.

[0055] Step S104, performing dependency grammar analysis on the target notice and the navigation notice to obtain analysis results, wherein the dependency grammar analysis is used to analyze word dependency relations and grammatical role relations between the target notice and the navigation notice.

[0056] Optionally, the purpose of dependency parsing is to deeply understand the internal logical relationship and grammatical structure of the notice text. By analyzing the dependency relationship between words, it is possible to identify which words are subject-predicate relationships, which are objects, which words have a modifying relationship, and so on. This information is crucial to understanding the true meaning of the notice, especially when the notice contains complex conditions or multiple restrictive statements, dependency parsing can reveal the exact scope and logical order of these conditions and restrictive statements.

[0057] During the dependency grammar analysis process, the dependency type of each word in the notice can be identified, for example, the dependency between a verb and its subject and object, or the dependency between a noun and its modifier. The identification of this dependency relationship can help users or models understand the deep structure of the sentence and ensure that key information (such as the subject, time, location and other details of the "runway closure" event) is correctly presented in the target notice. In addition, each word in the notice not only has its dependency relationship, but also has a grammatical role in the sentence. For example, a word may be the subject, object, attributive or adverbial of a sentence. Identifying these grammatical roles helps ensure the semantic integrity and logical consistency of the notice information and prevents misreading or omission of information during the reverse reconstruction process.

[0058] It should be noted that by comparing the dependency grammar analysis results of the target notice with the original notice, it is possible to check whether the sentence structure and lexical relationships of the two are consistent. If the dependency structures of the two notices are the same and the grammatical roles of each word match, it can be preliminarily considered that the structure of the notice information has been correctly reconstructed. Dependency grammar analysis can reveal the correctness of the logical relationships in the notice, such as time conditions, location restrictions, etc. If the notice mentions that the event of "runway closure" occurred within a specific time period and was limited to a specific geographical area, dependency grammar analysis will check whether this logic is accurately retained and expressed in the target notice.

[0059] Optionally, dependency parsing can be performed by deep learning models, such as recursive neural networks, long short-term memory networks, or transformer models. These models learn the structure and rules of language through a large amount of training data and can accurately parse the grammatical structure of the announcement text.

[0060] In the processing of highly standardized and semantically complex text such as navigation notices, dependency grammar analysis can not only ensure the integrity of the notice information during the parsing and reconstruction process, but also further ensure the correctness and coherence of the notice content through the verification of logical relationships, thereby greatly improving the safety and reliability of aviation information processing.

[0061] Step S105, determining the parsing accuracy of the target system for the NOTAM according to the similarity and the analysis result.

[0062] Optionally, in the present application, the determination of parsing accuracy is not based solely on one evaluation standard, but rather combines two methods: similarity detection and dependency grammar analysis. Similarity detection focuses on content matching at the text level. It intuitively reflects the consistency of the target announcement with the original announcement in text form by quantifying the structural and lexical similarities of the two texts. Dependency grammar analysis, on the other hand, focuses more on semantic and logical details, and ensures that the logical structure of the key information of the announcement is correctly reconstructed by checking word dependencies and grammatical roles.

[0063] For example, after obtaining the similarity score and dependency parsing results of the target notice and the original notice, these two indicators can be considered together. The similarity score can be used as a rough quantitative indicator to indicate the degree of literal match between the two notices; while the dependency parsing results provide a deeper perspective to help us understand the semantic correctness of the notice content.

[0064] Among them, if the target notice and the original notice obtain a high score in the similarity test, and the dependency grammar analysis results of both sides show that the relationship between words and the grammatical roles are consistent, then it can be determined that the target system has a high parsing accuracy for the navigation notice, indicating that the parsing process effectively retains the key information and logical structure of the notice. On the contrary, if the similarity score is low, or the dependency grammar analysis reveals the lack of key information, logical confusion, or grammatical role mismatch, this will indicate that the target system has significant errors in parsing the navigation notice. At this time, the parsing accuracy will be considered insufficient, and the parsing system (i.e., the target system) or the inverse model needs to be adjusted and optimized to correct the errors and improve the accuracy of future parsing.

[0065] Optionally, Figure 3 : is an optional comparison agent execution flow diagram according to an embodiment of the present application, wherein the retrospective verification system can use the comparison agent to complete the comparison of the original message (i.e., the original navigation notice) and the target message reconstructed by the reverse model (i.e., the above-mentioned target notice), including: after the LLM (Large Language Model) model (corresponding to the above-mentioned reverse model) obtains the message parsing result of the original message, calling the comparison agent, then extracting key information from the message parsing result, and reconstructing the target message ( Figure 3 This step is omitted in the text). Then the comparison agent calculates the similarity between the original message and the target message, and also performs dependency syntax analysis on the target message and the original message to obtain the analysis result. Finally, the comparison agent can make a consistency judgment on the target message and the original message according to the judgment rule, thereby verifying the correctness of the parsing of the original message.

[0066] In an optional embodiment, determining the parsing accuracy of the target system for the NOTAM according to the similarity and the analysis result includes: determining that the parsing accuracy of the target system for the NOTAM is greater than a target threshold when it is detected that the target NOTAM and the NOTAM simultaneously meet the following judgment conditions based on the analysis result and the similarity, wherein the judgment conditions include:

[0067] The first condition is used to limit the navigation time deviation described in the target notice and the navigation notice to be less than or equal to a preset threshold;

[0068] The second condition is to limit the scope of navigation geographical location described in the target notice and the navigation notice to be free of conflict;

[0069] The third condition is to ensure that the flight safety information described in the target notice and the navigation notice is consistent;

[0070] The fourth condition is used to limit the similarity between the target notice and the navigation notice to be greater than a preset similarity.

[0071] Optionally, the time information involved in the navigation notice, such as the effective time and the expiration time, is crucial to aviation safety. Therefore, the first condition checks whether there is a slight deviation between the target notice and the navigation time described in the original notice, and whether this deviation is within the preset time threshold. The setting of the time threshold needs to take into account the actual situation of aviation operations. For example, a difference of a few hours or a few days may be acceptable, while a deviation exceeding the preset threshold may have an impact on the flight plan. Through the restriction of this condition, the accurate transmission of time information is ensured to avoid safety hazards caused by time differences.

[0072] Optionally, the geographical location information in the NOTAM, including the airport code, longitude and latitude coordinates, etc., involves the effective range and applicable area of ​​the NOTAM. The second condition ensures that there is no conflict between the target NOTAM and the original NOTAM in terms of the geographical location described, that is, the geographical location range involved in the NOTAM is completely consistent. This is particularly important to prevent aircraft from entering restricted or dangerous areas.

[0073] Optionally, the core of the NOTAM is to convey flight safety-related change information, such as airport runway status, navigation equipment operating conditions, and adjustments to air traffic service procedures. The third condition requires that the target NOTAM is completely consistent with the flight safety information described in the original NOTAM, ensuring that no key safety instructions are omitted in the parsing results. The satisfaction of this condition is the core of evaluating the accuracy of the parsing, because any misreading or omission of flight safety information may have a serious impact on aviation operations.

[0074] Optionally, in addition to the consistency check of the above three dimensions, this application also introduces the evaluation of overall text similarity as an indicator for comprehensive judgment of parsing accuracy. The calculation of overall similarity can be performed through a deep learning model, which not only considers literal matching but also covers semantic understanding. The selection of the preset similarity threshold should be based on the complexity of the announcement and industry standards to ensure that the parsed announcement is highly similar to the original text in semantics, thereby ensuring the integrity and accuracy of information transmission.

[0075] Finally, when the target notice and the original notice (i.e., the original navigation notice) meet the above four conditions at the same time, the retrospective verification system can determine that the accuracy of the target system's analysis of the navigation notice is greater than the target threshold, that is, the analysis process is of high quality. The design of the comprehensive judgment mechanism fully considers the characteristics of the navigation notice, and through the combination of quantification and logical judgment, ensures that the analysis results are highly consistent with the original notice in terms of time, location, safety information and overall semantics. This mechanism not only improves the efficiency of verification, but also significantly improves the accuracy of flight safety information processing.

[0076] In an optional embodiment, the retrospective verification system may determine that the target system's parsing accuracy of the navigation notice is less than or equal to the target threshold based on the analysis results and the similarity when it detects that the target notice and the navigation notice do not satisfy at least one of the first condition, the second condition, the third condition, and the fourth condition.

[0077] Optionally, when the target NOTAM does not meet preset conditions with the original NOTAM in terms of time, geographic location, consistency of safety information, or overall similarity, the retrospective verification system will deem the resolution accuracy insufficient, which triggers further review procedures or optimization requirements of the resolution system to continuously improve the safety and reliability of NOTAM processing.

[0078] In an optional embodiment, dependency parsing includes the following steps:

[0079] Identify the grammatical role of each word in the target notice and the navigation notice; determine the dependency relationship between different words in the notice text sentence according to the grammatical role of each word and the position of each word in the notice text sentence; determine the semantic structure of the notice text sentence according to the dependency relationship between different words in the notice text sentence; extract the key semantic information of the notice text sentence according to the semantic structure of the notice text sentence; determine the analysis result according to the key semantic information.

[0080] Optionally, first, the retroactive verification system can scan each word in the target notice and the original navigation notice to determine its grammatical identity in the sentence. For example, it can identify whether a word is a noun, a verb, an adjective, or more specifically, whether the word acts as a subject, an object, an attributive, an adverbial, etc. This recognition can determine the function of the word in the sentence. Next, the retroactive verification system can establish dependency links between words based on the grammatical role of each word and the relative position of the word in the sentence. Dependency relationships describe the direct connection between words in a sentence, such as "verb-object", "noun-attributive", etc. These links constitute the dependency tree of the notice sentence, which can clearly show the logical and grammatical ties between words.

[0081] Optionally, once the dependencies are determined, the backtracking verification system can construct a semantic structural framework of the announcement statement. The semantic structure reveals the relationships between key elements such as participants, actions, time and place of the event or state described in the announcement, and how these elements are combined to express a specific meaning. Through the semantic structure, the backtracking verification system can more intuitively understand the internal logic of the announcement. After understanding the semantic structure of the announcement, the next step is to extract key semantic information. This includes identifying the core content in the announcement, such as changes in facility status, descriptions of time ranges, limitations on geographical areas, etc. Key semantic information is the core value of the announcement, and its accuracy and completeness directly affect the effectiveness of the announcement.

[0082] Finally, the backtracking verification system obtains the final analysis results based on the extracted key semantic information. This includes a comprehensive understanding of the announcement sentence and an evaluation of the parsing accuracy. If the target announcement is consistent with the original announcement in terms of dependencies and key semantic information, then the parsing process can be considered correct; otherwise, it may indicate that there is a problem in the parsing, which requires further verification and correction.

[0083] Through dependency grammar analysis, the technical solution of this application can ensure that the parsed information of the notice is not only consistent with the original text on the surface, but more importantly, accurate at the deep semantic and logical level, which is crucial to ensuring flight safety and smooth aviation operations. The introduction of dependency grammar analysis has greatly improved the accuracy and reliability of the navigation notice intelligent processing system.

[0084] In an optional embodiment, the training step of the inverse model includes: obtaining multiple sample data pairs, wherein the sample data pairs include: historical navigation notices and parsed texts of the historical navigation notices; determining labels of the parsed texts, wherein the labels of the parsed texts are used to evaluate the parsing quality of the parsed texts for the historical navigation notices; based on the labels of the parsed texts, screening out target sample data pairs from the multiple sample data pairs; and based on the target sample data pairs, performing multiple iterative training on the neural network to obtain the inverse model.

[0085] Optionally, in order to train the inverse model, a large number of historical NOTAMs and their corresponding parsed texts are first collected. Each NOTAM and parsed text constitute a sample data pair. These sample data pairs are the basis for training the inverse model. Next, the retrospective verification system can automatically perform a quality assessment on each parsed text and determine its label based on the assessment results. This label reflects the quality of the parsed text for the original NOTAM, which can be classified as "accurate", "partially accurate" or "inaccurate". Quality assessment is usually based on specific criteria, such as whether all key information is fully parsed, whether the description of time, place and events is correct, and whether the semantics of the text is consistent with the original notice. These labels provide the goal of model learning, that is, the model needs to learn to generate the original NOTAM corresponding to the "accurate" label when given a parsed text.

[0086] From all the sample data pairs collected, the retrospective verification system can filter out those samples that contain the "accurate" label as training data, that is, the target sample data pairs. These samples represent high-quality parsing results and are good learning materials for training the model to achieve high accuracy goals. By focusing on these target samples, the model learns how to establish accurate correspondence between the parsed text and the original NOTAM.

[0087] Finally, the back-testing system can use the selected target sample data pairs to train the neural network. The training process usually includes multiple iterations, in each of which the model tries to reverse the original NOTAM from the parsed text and adjusts its parameters by comparing the difference between the generated message and the actual original message to minimize the loss function. As the number of iterations increases, the model gradually learns how to complete the reverse task more accurately.

[0088] By training the neural network using labeled historical NOTAMs and parsed results, the aim is to train an inverse model that can accurately reconstruct historical NOTAMs from parsed texts, thereby providing an effective way to verify the accuracy and completeness of NOTAM parsing results.

[0089] In an optional embodiment, in the process of performing multiple iterative training on the neural network according to the target sample data pairs to obtain the inverse model, the backtracking verification system can calculate the ratio of the current number of iterations to the set maximum number of iterations when performing the i-th iterative training on the neural network, where i is an integer greater than 1, and then determine the model learning rate during the i-th iterative training based on the ratio. Finally, the backtracking verification system performs the i-th iterative training based on the model learning rate.

[0090] Optionally, in the i-th iteration of model training, the backtracking verification system can calculate the current iteration number (e.g. ) and the preset maximum number of iterations The ratio r between them is Here r can be regarded as an indicator of training progress, which gradually increases from 0 (start) to 1 (end). This ratio can help the backtracking verification system understand the current stage of training and thus determine the learning rate adjustment strategy.

[0091] Optionally, the backtracking verification system can adjust the learning rate according to the iteration ratio r, where the model learning rate is an important parameter in the model training process, which controls the amplitude of the model parameter update. In the early stage, a higher learning rate can speed up the convergence of the model, but as the training progresses, an excessively high learning rate may cause the model to oscillate around the optimal solution and be difficult to stabilize in the optimal state. Therefore, the system will adjust the learning rate according to the current iteration ratio r. The specific adjustment strategy can refer to the following formula (1):

[0092]

[0093] in, is the lower limit of the learning rate in the later iteration, is the upper limit of the learning rate at the beginning of the iteration. The increase in learning rate It will show a trend of first high and then low. Helps the model learn quickly, while the lower This ensures that the model can fine-tune parameters to avoid overfitting and achieve optimal generalization capabilities.

[0094] After adjusting the learning rate, the backtracking verification system continues to perform the i-th iteration training, using the parsed text and the corresponding original navigation notice in the target sample data pair as input and output pairs to train the neural network model. In each iteration, the model updates parameters based on the matching degree of the parsing quality and the back-propagation result, gradually improving its back-propagation ability until the maximum number of iterations is reached.

[0095] Optionally, during the entire training process, the backtracking verification system can continuously monitor the learning status of the model, including indicators such as the changing trend of the loss function, the matching degree between the back-generated NOTAM and the original NOTAM, etc. If the model performance is found to be improving slowly or showing signs of overfitting during the training process, the system can take measures, such as terminating the training early, adding regularization terms, or adjusting the training parameters, to optimize the performance and stability of the model.

[0096] In summary, the backtracking verification system optimizes the training process of the neural network model by dynamically adjusting the learning rate, ensuring that the model can learn efficiently and gradually converge in multiple iterations, thereby obtaining a reverse prediction model that can accurately reverse the navigation notice.

[0097] In an optional embodiment, the model learning rate during the i-th iterative training is determined according to the ratio, including: calculating the product of the ratio and pi to obtain a target value; calculating the cosine value of the target value; calculating the difference between the maximum learning rate and the minimum learning rate set for the neural network; determining the model learning rate during the i-th iterative training according to the cosine value of the target value, the difference between the maximum learning rate and the minimum learning rate set for the neural network, and the minimum learning rate.

[0098] Optionally, as in formula (1), the backtracking verification system calculates the iteration progress ratio and pi The product of . is a constant, approximately equal to 3.14159. By multiplying the iteration progress ratio by pi, we can get the target value The system then calculates the cosine of the target value The cosine function is widely used in mathematics and physics to describe periodic changes. In this case, the cosine value ranges from [-1, 1], as increase, It will increase from 0 to Then from Increase to , so the cosine value will first drop from 1 to -1, and then rise back to 1. In this way, the change of the cosine value intuitively reflects the cyclical change trend of the learning rate from high to low and then rise again.

[0099] In an optional embodiment, before performing the i-th iterative training according to the model learning rate, the backtracking verification system can obtain the latest network parameters of the neural network before the i-th iterative training; detect the latest loss function value of the neural network before the i-th iterative training; determine the gradient information of the latest loss function value based on the latest network parameters; determine the model parameter information updated during the i-th iterative training based on the latest network parameters and gradient information of the neural network before the i-th iterative training and the model learning rate during the i-th iterative training.

[0100] Optionally, formula (2) shows the updating process of model parameter information:

[0101]

[0102] As shown in formula (2), is the latest network parameter of the neural network before the i-th iteration training, Represents the latest loss function value of the neural network before the i-th iteration training; Indicates the gradient information of the latest loss function value based on the latest network parameters. The above formula (2) can be used to calculate the model parameter information updated during the i-th iteration training: .

[0103] In summary, the backtracking verification system ensures that each iteration of training can effectively update the model parameters and optimize the model performance by obtaining the latest parameters of the neural network, monitoring the loss function value, calculating the gradient information and adjusting the learning rate. This strategy not only helps to improve the model's ability to reverse the results of the NOTAM analysis, but also ensures that the model can converge stably during the training process, avoid overfitting, and ultimately realize the automation and intelligence of the verification of NOTAM analysis results. Through continuous iterative training, the accuracy and efficiency of the model in processing various complex notifications and analysis results will be significantly improved.

[0104] In an optional embodiment, detecting the latest loss function value of the neural network before the i-th iteration training includes: obtaining the weight of each target sample data pair; obtaining the notice text predicted by the current neural network based on the parsed text in the target sample data pair; determining the target difference value between the notice text predicted by the current neural network and the actual historical navigation notices in the target sample data pair; and determining the latest loss function value of the neural network before the i-th iteration training based on the weight of each target sample data pair and the target difference value.

[0105] Optionally, formula (3) is an optional way to calculate the loss function:

[0106]

[0107] In formula (3), Represents the weight of any target sample data pair, such as the i-th target sample data pair; represents the announcement text predicted by the current neural network based on the parsed text in the i-th target sample data pair; represents the actual historical NOTAM in the i-th target sample data pair. The “N” in formula (3) refers to the total number of target data pairs.

[0108] Optionally, obtaining weights for each target sample data pair means that the system allows differentiating the influence of different samples during the training process. For example, for navigation notices containing complex conditions and exceptions, their weights may be higher, because correctly interpreting such notices is critical to flight safety. By giving higher weights to key samples, the model will pay more attention to these samples during training, thereby enhancing the model's robustness and ability to handle complex notices.

[0109] By dynamically adjusting the weights in the loss function, the system ensures that the current learning state can be adapted at each stage of model training. In the early stages of training, the model may learn all samples equally well, but as training progresses, some samples may require more attention. Dynamic adjustment of weights ensures that the model continuously optimizes its processing capabilities for key samples during the learning process, avoids excessive attention to low-value samples, and is conducive to the efficient allocation of resources, thereby maximizing the model's learning efficiency within a limited training time.

[0110] In an optional embodiment, determining a label of the parsed text includes: determining the label of the parsed text according to at least one of the following annotation rules:

[0111] The first annotation rule is used to detect whether the field information of the parsed text under the key field is consistent with the field information of the historical navigation notice under the key field;

[0112] The second annotation rule is used to detect whether the content in the parsed text includes navigation related information of the historical navigation notice;

[0113] The third annotation rule is used to detect whether the logical description of the parsed text in terms of time sequence, navigation range and geographical location is correct.

[0114] Optionally, the first annotation rule is mainly used to check whether the information of the key fields in the parsed text is consistent with the corresponding field information in the original historical navigation notice. The key fields may include but are not limited to airport code, effective time, restriction type, navigation equipment status, etc. By comparing the values ​​of these fields, the retrospective verification system can evaluate the accuracy of the parsing results in terms of basic information. For example, if the effective time of a notice is " "to" ", and the corresponding effective time field in the parsed text accurately reflects this information, then the parsed text will be given an "accurate" label; conversely, if the field information in the parsed result is wrong or missing, the parsed text will be marked as "inaccurate".

[0115] Optionally, the second annotation rule is mainly used to check whether the parsed text fully covers all the navigation-related information in the original historical navigation notice. This includes not only basic field information, but also detailed descriptions in the notice, such as the maintenance status of the runway, temporary changes in routes, availability of navigation equipment, etc. If the parsed text can fully reflect these navigation-related information without omissions or mistaken deletions, it will be marked as "complete", otherwise it will be marked as "incomplete". This rule ensures that the model can learn how to fully parse the notice content during the training process.

[0116] Optionally, the third annotation rule is mainly used to check whether the logical description in the parsed text, such as time sequence, flight range, geographical location, etc., is consistent with the information of the original notice. For example, if a notice describes that an airport is closed at night, but the description in the parsed text ignores the time zone information, resulting in a logically incorrect time description, then the parsed text will be marked as "logical description incorrect". By identifying and annotating these logical errors, the model can learn how to accurately understand and parse complex logical descriptions during training, which is especially important for processing notices involving time, space, and conditions.

[0117] By labeling the parsed text according to the above rules, a series of labeled sample data can be provided to the neural network to train the model on how to reversely generate the original NOTAM from the parsed results and judge its parsing quality. These labels help the model distinguish between correct and incorrect parsing results, thereby optimizing the model structure and parameters, making it more accurate and reliable when processing the reverse and verification tasks of the NOTAM parsing results.

[0118] In an optional embodiment, the backtracking verification system may include the following modules:

[0119] Data collection module: responsible for collecting the results of navigation notice analysis. This module connects with various navigation notice analysis systems to obtain the analyzed information and make preliminary annotations on the quality of the analysis results.

[0120] Computing and processing module: It carries out computing tasks of model training and data verification. In the model training phase, it uses model micro-debugging technology to fine-tune the large language model. In the data verification phase, it runs the fine-tuned model to verify whether the message can be correctly generated in the test set.

[0121] Storage module: used to store various data such as navigation notice analysis results, original message data, training data, fine-tuned models and comparison results.

[0122] Comparison execution module: Built-in comparison agent, using a large model to compare two message information to determine whether they are consistent.

[0123] Verification tool module: integrates the back-inference model and the comparison agent, receives the message parsing results and the original message as input, and outputs the evaluation results.

[0124] Optionally, the data collection module can be connected to various navigation notice analysis systems, and the retrospective verification system can obtain structured analysis results (JSON format) in real time. The quality of the analysis results is evaluated using automated annotation tools.

[0125] Optionally, the model fine-tuning and verification phase mainly includes the following steps:

[0126] Collect data: Collect the analysis results of NOTAM with high-quality annotations, including the original NOTAM messages and the parsed results.

[0127] Data cleaning and processing: The collected data is further cleaned and processed, including unifying the encoding format, removing unnecessary blank characters, and aligning the parsing results and original messages.

[0128] Model selection: Due to its outstanding performance on text tasks, the large language model can be selected as the base model.

[0129] Construct a data pair: The data pair contains [input: parsing result, output: original message].

[0130] Model fine-tuning: Set appropriate training parameters to optimize model performance. Please refer to formula (1), formula (2) and formula (3).

[0131] Optionally, the backtracking verification system can extract key information from the message results based on the above-mentioned fine-tuned and trained large model as the base of the Agent; use the fine-tuned model to extract key information, which includes but is not limited to the status of airport facilities, the working status of navigation equipment, etc.

[0132] For example, refer to formula (4):

[0133]

[0134] Among them, k is the set of extracted key information, M is the fine-tuned large model, and T is the parsed result of the input text, i.e., the navigation notice. The loss value t calculation formula based on cosine similarity is set as formula (5):

[0135]

[0136] Among them, A represents the key information extracted by the comparison agent, and B represents the original message. The sentence structure is further analyzed using the dependency syntactic analysis tool to obtain the dependency relationship and grammatical role of each word. The following consistency judgment rules are formulated based on the value of the loss function and the results of the dependency syntactic analysis:

[0137]

[0138] Optionally, the retroactive verification system can also define clear API endpoints to handle different requests, such as setting POST / compare: receiving the original message and the message generated by the retroactive push, and returning the comparison result. GET / report / {id}: Get the comparison report based on the comparison task ID. The retroactive verification system can also use thread pools to manage concurrent requests to ensure that system resources are used effectively. Each comparison task is assigned an independent thread or coroutine for processing. For tasks that take a long time, an asynchronous processing mechanism is used to put the task into the background queue for execution, and notify the client through callbacks.

[0139] According to one aspect of the embodiment of the present application, a retrospective verification device for the analysis result of the navigation notice is also provided, wherein: Figure 4 is a schematic diagram of an optional backtracking verification device according to an embodiment of the present application, such as Figure 4 As shown, the backtracking verification device includes: an acquisition unit 401, which is used to obtain the parsing result of the target system on the navigation notice; a first processing unit 402, which is used to determine the key information of the notice from the parsing result through the inverse model, and reconstruct and generate the target notice based on the key information of the notice, wherein the key information of the notice includes at least the navigation time information and the geographical location information; a detection unit 403, which is used to detect the similarity between the target notice and the navigation notice; an analysis unit 404, which is used to perform dependency grammar analysis on the target notice and the navigation notice to obtain the analysis result, wherein the dependency grammar analysis is used to analyze the word dependency relationship and grammatical role relationship between the target notice and the navigation notice; a second processing unit 405, which is used to determine the parsing accuracy of the navigation notice by the target system according to the similarity and the analysis result.

[0140] Optionally, the second processing unit 405 includes: a first determination subunit, used to determine, based on the analysis results and the similarity, that the target system's parsing accuracy of the navigation notice is greater than a target threshold when it is detected that the target notice and the navigation notice simultaneously meet the following judgment conditions, wherein the judgment conditions include: a first condition, used to limit the navigation time deviation described in the target notice and the navigation notice to be less than or equal to a preset threshold; a second condition, used to limit the navigation geographical location ranges described in the target notice and the navigation notice to not conflict; a third condition, used to limit the flight safety information described in the target notice and the navigation notice to be consistent; and a fourth condition, used to limit the similarity between the target notice and the navigation notice to be greater than a preset similarity.

[0141] Optionally, the second processing unit 405 includes: a second determination subunit, for determining, based on the analysis results and the similarity, that the resolution accuracy of the target system for the navigation notice is less than or equal to a target threshold when it is detected that the target notice and the navigation notice do not satisfy at least one of the first condition, the second condition, the third condition and the fourth condition.

[0142] Optionally, the retrospective verification device of the NOTAM parsing result is further used to perform the following steps:

[0143] Identify the grammatical role of each word in the target notice and the navigation notice; determine the dependency relationship between different words in the notice text sentence according to the grammatical role of each word and the position of each word in the notice text sentence; determine the semantic structure of the notice text sentence according to the dependency relationship between different words in the notice text sentence; extract the key semantic information of the notice text sentence according to the semantic structure of the notice text sentence; determine the analysis result according to the key semantic information.

[0144] Optionally, the retrospective verification device of the NOTAM parsing result is further used to perform the following steps:

[0145] Acquire multiple sample data pairs, wherein the sample data pairs include: historical navigation notices and parsed texts of the historical navigation notices; determine labels of the parsed texts, wherein the labels of the parsed texts are used to evaluate the parsing quality of the parsed texts for the historical navigation notices; select target sample data pairs from the multiple sample data pairs based on the labels of the parsed texts; and perform multiple iterative training on the neural network based on the target sample data pairs to obtain an inverse model.

[0146] Optionally, the retrospective verification device for the parsing results of the navigation notice also includes: a first calculation unit, used to calculate the ratio of the current number of iterations to the set maximum number of iterations when the neural network is trained for the i-th iteration, wherein i is an integer greater than 1; a first determination unit, used to determine the model learning rate during the i-th iteration training based on the ratio; and a first training unit, used to perform the i-th iteration training based on the model learning rate.

[0147] Optionally, the first determination unit includes: a first calculation subunit, used to calculate the product of the ratio and pi to obtain a target value; a second calculation subunit, used to calculate the cosine value of the target value; a third calculation subunit, used to calculate the difference between the maximum learning rate and the minimum learning rate set for the neural network; the first determination subunit, used to determine the model learning rate during the i-th iterative training based on the cosine value of the target value, the difference between the maximum learning rate and the minimum learning rate set for the neural network, and the minimum learning rate.

[0148] Optionally, the retrospective verification device for the parsing result of the navigation notice also includes: a first acquisition unit, used to acquire the latest network parameters of the neural network before the i-th iterative training; a first detection unit, used to detect the latest loss function value of the neural network before the i-th iterative training; a second determination unit, used to determine the gradient information of the latest loss function value based on the latest network parameters; and a third determination unit, used to determine the model parameter information updated during the i-th iterative training based on the latest network parameters and gradient information of the neural network before the i-th iterative training and the model learning rate during the i-th iterative training.

[0149] Optionally, the first detection unit includes: a first acquisition subunit, used to obtain the weight of each target sample data pair; a second acquisition subunit, used to obtain the notice text predicted by the current neural network based on the parsed text in the target sample data pair; a second determination subunit, used to determine the target difference value between the notice text predicted by the current neural network and the actual historical navigation notices in the target sample data pair; and a third determination subunit, used to determine the latest loss function value of the neural network before the i-th iteration training based on the weight of each target sample data pair and the target difference value.

[0150] Optionally, determining the label of the parsed text includes: determining the label of the parsed text according to at least one of the following labeling rules: a first labeling rule is used to detect whether the field information of the parsed text under the key field is consistent with the field information of the historical navigation notice under the key field; a second labeling rule is used to detect whether the content in the parsed text includes the navigation-related information of the historical navigation notice; a third labeling rule is used to detect whether the logical description of the parsed text in terms of time sequence, navigation range and geographical location is correct.

[0151] According to another aspect of the embodiment of the present application, a retrospective verification device for parsing results of navigation notices is also provided, including: an acquisition unit, used to obtain the parsing results of the navigation notice by the target system; a first processing unit, used to determine the key information of the notice from the parsing results through a reverse model, and reconstruct and generate the target notice based on the key information of the notice, wherein the key information of the notice includes at least navigation time information and geographic location information; a detection unit, used to detect the similarity between the target notice and the navigation notice; an analysis unit, used to perform dependency grammar analysis on the target notice and the navigation notice to obtain an analysis result, wherein the dependency grammar analysis is used to analyze the word dependency relationship and grammatical role relationship between the target notice and the navigation notice; a second processing unit, used to determine the parsing accuracy of the navigation notice by the target system based on the similarity and the analysis result.

[0152] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned retrospective verification method of the parsing result of the navigation notice.

[0153] According to another aspect of an embodiment of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors execute the above-mentioned retrospective verification method of the navigation notice parsing results.

[0154] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0155] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0160] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A retrospective verification method for NOTAM analysis results, characterized in that: include: Obtain the target system's analysis results of the NOTAM; Determine the key information of the announcement from the analysis result through the reverse inference model, and reconstruct and generate the target announcement based on the key information of the announcement, wherein the key information of the announcement includes at least the navigation time information and the geographical location information; detecting the similarity between the target notice and the navigation notice; Performing dependency grammar analysis on the target notice and the navigation notice to obtain an analysis result, wherein the dependency grammar analysis is used to analyze word dependency relationship and grammatical role relationship between the target notice and the navigation notice; The parsing accuracy of the target system for the NOTAM is determined according to the similarity and the analysis result.

2. The method according to claim 1, characterized in that Determining the parsing accuracy of the target system for the navigation notice according to the similarity and the analysis result includes: According to the analysis result and the similarity, when it is detected that the target notice and the navigation notice simultaneously meet the following judgment conditions, it is determined that the resolution accuracy of the target system for the navigation notice is greater than a target threshold, wherein the judgment conditions include: The first condition is used to limit the navigation time deviation described in the target notice and the navigation notice to be less than or equal to a preset threshold; The second condition is used to limit that there is no conflict between the navigation geographical location ranges described in the target notice and the navigation notice; The third condition is used to ensure that the flight safety information described in the target notice and the navigation notice is consistent; The fourth condition is used to limit the similarity between the target notice and the navigation notice to be greater than a preset similarity.

3. The method according to claim 2, characterized in that Determining the parsing accuracy of the target system for the navigation notice according to the similarity and the analysis result includes: Based on the analysis result and the similarity, when it is detected that the target notice and the navigation notice do not satisfy at least one of the first condition, the second condition, the third condition and the fourth condition, it is determined that the resolution accuracy of the target system for the navigation notice is less than or equal to the target threshold.

4. The method according to claim 1, characterized in that The dependency grammar analysis comprises the following steps: identifying a grammatical role of each word in the target notice and the NOTAM; Determining the dependency relationship between different words in the announcement text sentence according to the grammatical role of each word and the position of each word in the announcement text sentence; Determining the semantic structure of the notice text sentence according to the dependency relationship between different words in the notice text sentence; Extracting key semantic information of the notice text sentence according to the semantic structure of the notice text sentence; The analysis result is determined according to the key semantic information.

5. The method according to claim 1, characterized in that The training steps of the inverse model include: Acquire a plurality of sample data pairs, wherein the sample data pairs include: historical navigation notices and parsed texts of the historical navigation notices; Determining a label of the parsed text, wherein the label of the parsed text is used to evaluate the parsing quality of the parsed text for the historical NOTAM; Filtering a target sample data pair from a plurality of sample data pairs according to the label of the parsed text; According to the target sample data pairs, the neural network is iteratively trained multiple times to obtain the inverse model.

6. The method according to claim 5, characterized in that In the process of performing multiple iterative training on the neural network according to the target sample data pair to obtain the inverse model, the method further includes: When performing the i-th iteration training on the neural network, calculating the ratio of the current number of iterations to the set maximum number of iterations, where i is an integer greater than 1; Determine the model learning rate during the i-th iteration training according to the ratio; The i-th iteration training is performed according to the model learning rate.

7. The method according to claim 6, characterized in that Determining the model learning rate during the i-th iteration training according to the ratio includes: Calculating the product of the ratio and pi to obtain a target value; Calculating the cosine value of the target value; Calculate the difference between the maximum learning rate and the minimum learning rate set for the neural network; The model learning rate during the i-th iterative training is determined according to the cosine value of the target value, the difference between the maximum learning rate and the minimum learning rate set by the neural network, and the minimum learning rate.

8. The method according to claim 6, characterized in that Before performing the i-th iteration training according to the model learning rate, the method further includes: Obtaining the latest network parameters of the neural network before the i-th iteration training; Detecting the latest loss function value of the neural network before the i-th iteration training; Determine the gradient information of the latest loss function value based on the latest network parameter; The model parameter information updated during the i-th iterative training is determined according to the latest network parameters of the neural network before the i-th iterative training, the gradient information, and the model learning rate during the i-th iterative training.

9. The method according to claim 8, characterized in that Detecting the latest loss function value of the neural network before the i-th iteration training, including: Obtaining the weight of each target sample data pair; Obtaining the notification text predicted by the current neural network based on the parsed text in the target sample data pair; Determining a target difference value between the notification text predicted by the current neural network and the actual historical navigation notification in the target sample data pair; According to the weight of each target sample data pair and the target difference value, the latest loss function value of the neural network before the i-th iterative training is determined.

10. The method according to claim 5, characterized in that Determining a label of the parsed text includes: The label of the parsed text is determined according to at least one of the following labeling rules: A first marking rule is used to detect whether the field information of the parsed text under the key field is consistent with the field information of the historical navigation notice under the key field; A second marking rule is used to detect whether the content in the parsed text includes the navigation related information of the historical navigation notice; The third annotation rule is used to detect whether the logical description of the parsed text in terms of time sequence, navigation range and geographical location is correct.

11. A retrospective verification device for NOTAM analysis results, characterized in that: include: An acquisition unit, used for acquiring the parsing result of the target system on the navigation notice; A first processing unit is used to determine the key information of the announcement from the analysis result through an inverse model, and reconstruct and generate a target announcement based on the key information of the announcement, wherein the key information of the announcement at least includes the navigation time information and the geographical location information; A detection unit, used for detecting the similarity between the target notice and the navigation notice; an analysis unit, configured to perform dependency grammar analysis on the target notice and the navigation notice to obtain an analysis result, wherein the dependency grammar analysis is used to analyze word dependency relations and grammatical role relations between the target notice and the navigation notice; The second processing unit is used to determine the parsing accuracy of the target system for the navigation notice according to the similarity and the analysis result.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the retrospective verification method of the navigation notice parsing result as described in any one of claims 1 to 10.

13. An electronic device, characterized in that: It comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the retrospective verification method of the navigation notice parsing result as described in any one of claims 1 to 10.

Images