A two-way quality assessment method of pilot feedback report and airport processing result

Abstract

The application discloses a two-way quality evaluation method for pilot feedback report and airport processing result. The method is related to feedback report, airport processing event, expert evaluation and satisfaction data; the contradiction relationship between feedback text clauses is identified through a local natural language reasoning model, the information quantity is calculated through word segmentation and text information entropy, and the content quality score is obtained in combination with factual accuracy and suggestion feasibility; the feedback frequency and change trend are calculated based on the report quantity in a statistical time window to obtain the behavior quality score; the quality score of the feedback report is obtained by weighted calculation according to the content quality score and the behavior quality score; the processing timeliness is calculated according to the first response duration and the processing duration, the problem solving effectiveness is calculated according to the satisfaction and the repeated problem identification result, and the airport processing result quality score is further obtained according to the processing timeliness and the problem solving effectiveness. The method can generate a machine-readable two-way quality evaluation vector.

Description

Technical Field

[0001] This invention relates to the field of quality assessment technology, specifically to a two-way quality assessment method for pilot feedback reports and airport processing results. Background Technology

[0002] Aviation safety is a long-standing and crucial issue in the civil aviation sector. With the continuous growth of air transport volume, the airport operating environment, support processes, and collaborative relationships are becoming increasingly complex. Timely identification and resolution of safety hazards and support deficiencies during operations are of paramount importance for improving air transport safety and airport operational efficiency. Pilots, being on the front lines of aviation operations, have direct contact with aircraft taxiing, takeoff and landing, apron operations, ground support, and operational command. They can identify problems in airport facilities, operating procedures, support services, and collaborative management from a professional operational perspective. Pilot feedback reports therefore become an important source of information for airports to obtain frontline operational information, identify potential risks, and improve support work. After receiving feedback reports, airports typically need to go through stages such as review, response, investigation, processing, reply, and closure to formulate a resolution for the issues raised in the feedback.

[0003] Current pilot feedback management methods primarily focus on the collection, transfer, and archiving of feedback reports, with the quality of the reports themselves typically relying on manual judgment by managers or professionals. Because feedback reports are often written in natural language, significant differences exist between reports in terms of logical relationships, information richness, factual basis, completeness of problem description, and the implementation conditions of proposed suggestions. Manual evaluation is easily influenced by the evaluator's experience and subjective judgment, making it difficult to produce unified, stable, and comparable evaluation results. Furthermore, current evaluations usually focus on the textual content of individual feedback reports, rarely considering the number of reports submitted and their changes within a specific statistical period to assess feedback behavior. Therefore, they fail to comprehensively reflect the feedback from pilots or relevant evaluation subjects at different times. There is also a lack of mutually distinguishing and consistent evaluation methods for individual feedback reports and collections of reports within a statistical period.

[0004] On the other hand, airports typically record the processing and results of feedback reports in different data logs, such as report submission records, initial response records, processing completion records, satisfaction survey records, and subsequent feedback records. The lack of a unified correlation between these data makes it difficult to comprehensively evaluate information such as the timeliness of the initial response, the reasonableness of the processing cycle, pilot satisfaction, and whether the same problem recurs after processing. Existing methods often judge processing completion solely based on whether a report has been responded to or closed, failing to fully reflect the actual quality of the processing results. Furthermore, feedback reports submitted by pilots at different times may use different wording to describe the same or similar problems. Relying solely on keywords or manual comparison makes it difficult to accurately identify recurring problems after processing, thus affecting the assessment of the airport's processing effectiveness.

[0005] Therefore, current technology lacks a method to correlate pilot feedback report data, airport processing data, professional evaluation data, and pilot satisfaction data, and to generate quantitative evaluation results from both the perspectives of feedback report quality and airport processing outcome quality. Due to the lack of a mutually reinforcing two-way evaluation mechanism, it is difficult to screen high-value feedback reports and objectively reflect the airport's response to and resolution of feedback issues, hindering the formation of a closed-loop management process of "proposal, processing, feedback, and improvement." Summary of the Invention

[0006] The purpose of this invention is to provide a two-way quality assessment method for pilot feedback reports and airport processing results, thereby solving the problems existing in the prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a two-way quality assessment method for pilot feedback reports and airport processing results, comprising: S1. Obtain pilot feedback report data, airport incident handling data, expert evaluation data, and satisfaction data, and generate evaluation samples according to the report identifiers; S2. Segment the feedback text into clauses and construct clause pairs, input them into the local natural language reasoning model, and determine the logical consistency index based on the proportion of contradictory relationships in the clause pairs. S3. Perform word segmentation and information entropy calculation on the feedback text to obtain information content indicators; S4. Combine expert evaluations to obtain factual accuracy indicators and suggestion feasibility indicators, and then weight them to obtain a content quality score; S5. Within the statistical time window, the feedback frequency index is determined based on the normalization result of the number of feedback reports relative to the extreme value of the number of the baseline period. The trend index is determined based on the change in the number of the current statistical time window and the previous statistical time window and the Sigmoid function. Then, the behavior quality score is obtained and weighted with the content quality score to obtain the feedback report quality score. S6. Determine the processing timeliness score based on the report submission time, initial response time, and processing completion time; S7. Determine the effectiveness score for problem-solving based on pilot satisfaction and repeated feedback identified using problem identifiers or text semantic vectors, and then weight them to obtain the airport processing result quality score. S8. Output a two-way quality assessment result containing the feedback report quality score and the airport processing result quality score.

[0008] Specifically, the step of generating evaluation samples based on report identifiers includes: Unify the time zone and time unit for time fields from different data sources; Identify and delete duplicate records based on report identifiers, feedback text, and submission times; Identify abnormal records based on the temporal relationship between submission time, first response time, processing completion time, and processing status, and add data abnormality identifiers to the abnormal records; Anonymize pilot and expert identification identifiers; Establish correlations between pilot feedback report data, airport incident handling data, expert evaluation data, and satisfaction data based on report identifiers.

[0009] Specifically, the determination of the logical consistency index includes: The feedback text is divided into multiple valid clauses according to a preset text delimiter, and clause pairs are constructed based on the valid clauses; Each clause pair is input into the local natural language reasoning model to obtain the classification results of each clause pair corresponding to implication relation, contradiction relation, and neutral relation, respectively; The percentage of clause pairs identified as contradictory is counted among all clause pairs, and the percentage is reverse-mapped to obtain the logical consistency index. When there are no constructible clause pairs in the feedback text, the logical consistency index is determined according to the preset boundary processing rules.

[0010] Specifically, the determination of the information content index includes: The feedback text is segmented and invalid characters are filtered to obtain multiple valid terms; The frequency of occurrence of each effective term is counted, and the text information entropy is calculated based on the probability distribution of each effective term in the feedback text. Based on the information entropy distribution in the preset benchmark report set, the text information entropy is normalized to obtain an information quantity index within a preset numerical range. When the information entropy in the benchmark report set does not have an effective distinguishing range, the information content index is determined according to the preset boundary processing rules.

[0011] Specifically, the indicators for factual accuracy and recommendations for feasibility obtained by combining expert evaluation include: Obtain the first set of evaluation data to assess the objectivity, authenticity, completeness of background elements, and specificity of problem description in the feedback report. Normalize and weight the first set of evaluation data to obtain the factual accuracy index. Obtain a second set of evaluation data to assess the technical feasibility, resource feasibility, goal consistency, and value impact of the feedback recommendations. Normalize and weight the second set of evaluation data to obtain the feasibility index of the recommendations. The logical consistency index, information content index, factual accuracy index, and suggestion feasibility index are integrated according to preset weights to obtain the content quality score; When the same evaluation dimension corresponds to multiple expert evaluation results, the multiple expert evaluation results are aggregated based on at least one of the following: expert professional category, professional experience, and degree of professional matching with the problem to be evaluated.

[0012] Specifically, the determination of the behavioral quality score includes: The number of feedback reports corresponding to the statistical target evaluation object within the current statistical time window; Obtain the number of feedback reports corresponding to multiple unit time windows within the baseline period, and normalize the number of feedback reports in the current statistical time window relative to the extreme values ​​of the number within the baseline period to obtain the feedback frequency index. The rate of change of quantity is determined based on the number of feedback reports in the current statistical time window and the previous statistical time window, and the rate of change of quantity is mapped to a preset numerical range using the Sigmoid function to obtain a trend index. The feedback frequency index and the trend index are fused according to preset weights to obtain a behavior quality score; When there is no effective range in the number of feedback reports within the baseline period or the number of feedback reports in the previous statistical time window is zero, the calculation is completed according to the preset smoothing rule or boundary processing rule.

[0013] Specifically, the determination of the feedback report quality score includes: When the evaluation object is a single feedback report, the quality score of the feedback report is determined based on the content quality score of the single feedback report; When the evaluation object is a set of feedback reports within a statistical time window, the content quality scores of each feedback report in the set are aggregated, and the aggregated content quality scores are fused with the behavior quality scores according to preset weights to obtain the feedback report quality score of the set.

[0014] Specifically, the determination of the processing timeliness score includes: The initial response duration is determined based on the time difference between the report submission time and the initial response time, and the processing duration is determined based on the time difference between the report submission time and the processing completion time. For the processing result of a single airport, the initial response time and processing time are compared with the corresponding preset maximum time limit. The initial response timeliness index and processing cycle timeliness index are determined based on the remaining proportion of each actual time within the corresponding preset maximum time limit. For the processing results of multiple airports within the statistical time window, the percentage of processing results whose first response time meets the preset response time limit and the percentage of processing results whose processing time meets the preset processing time limit are calculated separately. The timeliness of the first response and the timeliness of the processing cycle, or the two proportions, are combined according to a preset weight to obtain a processing timeliness score.

[0015] Specifically, the method of identifying repetitive feedback using question identifiers or text semantic vectors includes: Extract at least two items from the feedback text: airport identifier, operating area, facility object, problem phenomenon, triggering condition, and affected object, to generate structured problem features; The feedback text or the text sequence composed of the structured question features is encoded using a local text encoding model to obtain a question feature vector; From the subsequent feedback reports after the target feedback report is processed, candidate feedback reports that meet the preset time sequence conditions and airport association conditions are selected; Calculate the semantic similarity between the problem feature vectors of the target feedback report and the candidate feedback reports; Based on the matching results of the structured question features and the semantic similarity, duplicate feedback belonging to the same question is identified; When a feedback report has a pre-set problem identifier, duplicate feedback will be identified first based on the problem identifier.

[0016] Specifically, the determination of the problem-solving effectiveness score and the airport processing result quality score includes: Normalize the pilot satisfaction score corresponding to the processing results of a single airport, or determine the pilot satisfaction index based on the percentage of satisfaction samples that meet the preset satisfaction conditions within the statistical time window. For a single airport's processing results, the duplicate feedback index is determined based on the number of times the same problem reappears between the completion of the processing result and the evaluation time point; for multiple airports' processing results within the statistical time window, the duplicate feedback index is determined based on the proportion of feedback reports that the same problem reappears. The repeated feedback index is processed in reverse and fused with the pilot satisfaction index according to a preset weight to obtain a problem-solving effectiveness score. The problem-solving effectiveness score and the processing timeliness score are combined according to preset weights to obtain the airport processing result quality score. The feedback report quality score and the airport processing result quality score are compared with the corresponding preset scoring thresholds, and information supplementation labels, low quality labels or airport processing abnormal tasks are generated based on the comparison results.

[0017] As can be seen from the above technical solution, the present invention has the following beneficial effects: This two-way quality assessment method for pilot feedback reports and airport handling results establishes a two-way quality assessment mechanism by correlating pilot feedback report data, airport handling event data, expert evaluation data, and satisfaction data. Specifically, by comprehensively evaluating clause logic, textual information content, factual accuracy, and the feasibility of suggestions, it helps reduce subjective differences arising from relying solely on human experience, improving the consistency and comparability of feedback report quality assessment. By combining feedback frequency and trends within a statistical time window, it further reflects the changes in feedback behavior over different time periods. A comprehensive analysis of initial response time, handling completion time, pilot satisfaction, and repeated feedback provides a more complete reflection of the timeliness and effectiveness of airport handling results. Furthermore, using problem identifiers or text semantic vectors to identify repeated feedback helps detect similar problems that recur after handling, improving the accuracy of judging the actual degree of problem resolution. Therefore, this application provides quantitative evidence for high-quality feedback report screening, airport handling process evaluation, and abnormal task identification, promoting a closed loop between pilot feedback submission, processing, feedback, and improvement stages. Attached Figure Description

[0018] Figure 1 This is a flowchart of the two-way quality assessment method for pilot feedback reports and airport processing results according to the present invention. Figure 2 This is a sequence diagram of the two-way quality assessment method for pilot feedback reports and airport processing results of the present invention; Figure 3 This is a diagram showing the core architecture of the local natural language reasoning model of this invention. Figure 4 This is a comparison chart of the method indicators and expert scores in an embodiment of the present invention; Figure 5 This is a scatter plot showing the correlation between the content quality score and the overall expert score in this embodiment of the invention. Figure 6This is a graph showing the change in the performance of repeated feedback recognition as a function of semantic similarity threshold in an embodiment of the present invention. Figure 7 This is a quadrant diagram showing the distribution of bidirectional quality assessment results according to an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figures 1-7 As shown, the present invention provides a technical solution: a two-way quality assessment method for pilot feedback reports and airport processing results, comprising: S1. Obtain pilot feedback report data, airport incident handling data, expert evaluation data, and satisfaction data, and generate evaluation samples according to the report identifiers; S2. Segment the feedback text into clauses and construct clause pairs, input them into the local natural language reasoning model, and determine the logical consistency index based on the proportion of contradictory relationships in the clause pairs. S3. Perform word segmentation and information entropy calculation on the feedback text to obtain information content indicators; S4. Combine expert evaluations to obtain factual accuracy indicators and suggestion feasibility indicators, and then weight them to obtain a content quality score; S5. Within the statistical time window, the feedback frequency index is determined based on the normalization result of the number of feedback reports relative to the extreme value of the number of the baseline period. The trend index is determined based on the change in the number of the current statistical time window and the previous statistical time window and the Sigmoid function. Then, the behavior quality score is obtained and weighted with the content quality score to obtain the feedback report quality score. S6. Determine the processing timeliness score based on the report submission time, initial response time, and processing completion time; S7. Determine the effectiveness score for problem-solving based on pilot satisfaction and repeated feedback identified using problem identifiers or text semantic vectors, and then weight them to obtain the airport processing result quality score. S8. Output a two-way quality assessment result containing the feedback report quality score and the airport processing result quality score.

[0021] The method described in this embodiment can be executed by a server, an airport operations management platform, a local computer, or other electronic devices with data processing capabilities. The natural language inference model and text encoding model used during execution are preferably deployed on local airport computing nodes to reduce the transmission of feedback text, pilot identification information, and airport operations information to external networks.

[0022] In one possible implementation, pilot feedback reports, airport incident handling data, expert evaluation data, and satisfaction data are first acquired.

[0023] Pilot feedback report data should include at least a report identifier, feedback text, submission time, pilot identification, associated airport identifier, and report status. Airport incident handling data should include at least a report identifier, first viewing time, first response time, processing completion time, processing status, and processing response content. Expert evaluation data should include at least a report identifier, expert identification, expert's specialty, and scores for each evaluation dimension. Satisfaction data should include at least a report identifier, satisfaction score, survey submission time, and survey completion time. The system associates the above data according to the report identifier. For the same feedback report, its feedback text, submission time, airport processing procedure, expert evaluation results, and pilot satisfaction results are combined into one evaluation sample. Establishing associations through report identifiers avoids association errors caused by approximate matching based solely on submission time, pilot name, or text content.

[0024] Before data association, preprocessing is performed on different data sources.

[0025] First, the times from different data sources are converted to a unified time zone, and then further converted to a unified time unit. In this implementation, both the initial response time and the processing time are calculated in minutes.

[0026] Secondly, duplicate records are identified based on report identifiers, feedback text, and submission times. When two records have the same report identifier, feedback text, and submission time, they are identified as duplicate records, and only one valid record is retained.

[0027] Next, examine the time relationships in the data. If the initial response time is earlier than the report submission time, the processing completion time is earlier than the initial response time, or the report status shows completion but no processing completion time exists, then the corresponding record will be marked as an anomaly. Anomalies will not be directly evaluated and will proceed to subsequent processing after manual verification or data correction.

[0028] In addition, pilot and expert identification identifiers are anonymized. Anonymization can be achieved using irreversible hash values, randomized replacement numbers, or other methods that maintain data relationships but do not directly reveal personnel identities.

[0029] After completing the above processing, the system establishes a one-to-one or one-to-many relationship between feedback report data, airport incident handling data, expert evaluation data, and satisfaction data based on the report identifier, and generates an evaluation sample for two-way quality assessment.

[0030] In this embodiment, the quality of the feedback report content is determined by four aspects: logical consistency, information content, factual accuracy, and feasibility of suggestions.

[0031] In one possible implementation, the feedback text undergoes punctuation standardization, converting different forms of periods, question marks, exclamation marks, semicolons, and line breaks into a uniform format. Subsequently, the feedback text is divided into multiple clauses using periods, question marks, exclamation marks, semicolons, or line breaks as text delimiters.

[0032] After segmentation, blank clauses, clauses containing only punctuation, and clauses without actual semantic content are deleted, resulting in multiple valid clauses. The system combines any two valid clauses to construct multiple clause pairs. Each clause pair includes a first clause as the premise text and a second clause as the text to be judged. In one implementation, all unique clause pairs can be constructed; when the feedback text is long, only adjacent clause pairs or clause pairs with high semantic relevance can be constructed to reduce the computational load of the model. Each clause pair is input into a local natural language inference model, which determines whether the two clauses have an implication relationship, a contradiction relationship, or a neutral relationship.

[0033] Among them, implication relations indicate that one clause supports or implies the content of another clause; contradictory relations indicate that two clauses describe the same fact, state, or condition in ways that cannot be simultaneously true; and neutral relations indicate that there is no explicit support or contradiction relationship between two clauses. The model can directly output the classification results of the three relations, or it can output the probabilities of each of the three relations, and use the relation with the highest probability as the final classification result. In another implementation, when the output probability of a contradictory relation reaches a preset threshold, the corresponding clause pair is identified as a contradictory relation. After classifying all clause pairs, the proportion of contradictory relation clause pairs is calculated. The higher the proportion of contradictory relations, the higher the probability that there is conflicting content within the feedback report. The system reverses the contradictory relation proportion by subtracting the contradictory relation proportion from 1 to obtain the logical consistency index.

[0034] For example, when there are no contradictory relationships among all clause pairs, the percentage of contradictory relationships is 0, and the logical consistency index is 1; when 20% of all clause pairs are identified as contradictory, the logical consistency index is 0.8. When the feedback text contains only one valid clause and clause pairs cannot be constructed, the logical consistency index can be set to 1. This setting only indicates that no internal contradictions were found in the text; it does not necessarily mean that the report has a high level of information content, completeness, or specificity. Other evaluation indicators are still calculated according to the actual situation.

[0035] The preprocessed feedback text is segmented into words. During segmentation, terms with practical operational meaning, such as airport name, taxiway number, runway number, time, frequency, distance, proportion, weather conditions, and facility names, are retained, while spaces, invalid symbols, and characters without analytical value are filtered out. The frequency of each valid term in the feedback text is counted, and the probability of that term's occurrence is obtained by dividing the frequency by the total number of valid terms. For each valid term, the natural logarithm of its probability is calculated, and then multiplied by the probability. The information entropy of the feedback text is obtained by summing the products of all terms and taking the negative of the sum. Information entropy reflects the richness of the term distribution in the feedback text. When the report repeatedly uses a small number of terms, the information entropy is usually low; when the report contains rich information on operational objects, problem phenomena, triggering conditions, and impacts, and the term distribution is relatively uniform, the information entropy is usually high.

[0036] To ensure that the information entropy of different reports can be compared within a uniform range, a baseline report set is pre-selected. This set can be valid pilot feedback reports received by the target airport within the past year, or valid reports received by airports of the same category within a set baseline period. The information entropy of each feedback report in the baseline report set is calculated, and the minimum and maximum values ​​are determined. The information entropy of the current feedback report is subtracted from the minimum information entropy in the baseline set, and then divided by the difference between the maximum and minimum information entropy to obtain the normalized information content index. When the normalization result is below 0, the information content index is set to 0; when the normalization result is above 1, the information content index is set to 1. When the maximum and minimum information entropies in the baseline report set are the same and cannot form a valid distinguishing range, the information content index of the current report can be set to 0.5, or normalization can be performed again based on a pre-set backup baseline set.

[0037] The accuracy of facts is evaluated based on three dimensions: objectivity, completeness of background information, and specificity of problem description. Each dimension is scored from 1 to 5 points. Objectivity is used to evaluate whether the feedback report accurately reflects the objective operational situation, and it accounts for 40% of the accuracy of facts evaluation.

[0038] A score of 5 for objectivity indicates that the core statements in the report can be fully verified by system data, operation records, work records, or other reliable evidence; a score of 3 for objectivity indicates that the core problem objectively exists, but there are certain deviations in the details of the cause, frequency, time, or scope of impact in the report, or that an occasional event is described as a common phenomenon; a score of 1 for objectivity indicates that the content of the report contradicts verifiable data records, or that it is a subjective judgment that cannot be supported by objective evidence.

[0039] The completeness of background elements is used to evaluate whether the feedback report includes background information such as time, location, triggering conditions, operation process, problem phenomenon, and impact results. Its weight is 30%. A score of 5 indicates that the key background elements are complete, and the conditions and process of the problem can be reconstructed based on the report content; a score of 3 indicates that only some key background elements are included, and supplementary information is needed; a score of 1 indicates that the report provides almost no background information and only contains vague or conclusive statements.

[0040] The specificity of the problem description is used to evaluate whether the problem description is clear, explicit, and has further analytical value, and its weight is 30%. A specificity score of 5 indicates that the report provides a clear and accurate description of the nature of the problem and provides quantitative information such as the number of occurrences, frequency, duration, proportion, location, or scope of impact; a specificity score of 3 indicates that the report only has a qualitative description or only a rough quantitative description; a specificity score of 1 indicates that the problem description is vague, lacks specific manifestations and quantitative information, and it is difficult to judge the actual situation of the problem.

[0041] When calculating the factual accuracy index, the objective truthfulness score is weighted at 40%, and the background element completeness score and the problem description specificity score are weighted at 30% each. The three weighted results are summed and then divided by the maximum score of 5 to obtain the normalized factual accuracy index.

[0042] The feasibility assessment is composed of four dimensions: technical feasibility, resource feasibility, goal consistency, and value and impact, each of which is scored from 1 to 5 points.

[0043] Specifically, technical feasibility, weighted at 25%, evaluates whether the airport's existing technical capabilities can realize the recommendations in the feedback report. A score of 5 indicates that implementation can be achieved using existing equipment, systems, or operational configurations, or requires only minor adjustments; a score of 3 indicates that implementation is feasible in principle, but requires certain development resources, construction arrangements, or specialized technical assessments; a score of 1 indicates that implementation is difficult under current technical conditions, or requires a fundamental restructuring of the core system. Resource feasibility, weighted at 25%, evaluates the funding, manpower, equipment, timeframe, and operational adjustment costs required to implement the recommendations. A score of 5 indicates that the required resources are minimal, and the cost is far lower than the expected benefits; a score of 3 indicates that the cost is roughly equivalent to the expected benefits, and further determination of implementation priorities is needed; a score of 1 indicates that the resource investment is enormous, significantly exceeding the potential value or existing budget capacity.

[0044] The goal consistency metric is used to evaluate whether the recommendations align with the airport's safety operation goals, efficiency goals, and development direction, and it accounts for 30% of the score. A goal consistency score of 5 indicates that the recommendations are highly consistent with the airport's positioning and development goals; a goal consistency score of 3 indicates that the recommendations do not directly conflict with the current goals, but the degree of correlation is moderate; a goal consistency score of 1 indicates that the recommendations deviate from the airport's development direction or may be detrimental to the airport's safe and efficient operation.

[0045] Furthermore, the value and impact score is used to evaluate the scope and extent to which the recommendations can solve problems, with a weight of 20%. A value and impact score of 5 indicates that the recommendations can solve common problems and improve the operational efficiency and user experience of a large number of airlines, passengers, or other stakeholders; a value and impact score of 3 indicates that the recommendations can solve problems for some stakeholders or bring about localized improvements; a value and impact score of 1 indicates that the recommendations are only applicable to a very few special cases and lack universal application value. When calculating the feasibility index of the recommendations, the technical feasibility score and resource feasibility score are each weighted at 25%, the goal consistency score at 30%, and the value and impact score at 20%. The weighted results are summed and then divided by the maximum score of 5 to obtain the normalized feasibility index of the recommendations. When the same evaluation dimension is scored by multiple experts, the scores can be aggregated according to expert weights. Expert weights can be determined based on the expert's profession, years of experience, and the degree of professional matching with the problem being evaluated. For taxiway lighting issues, the weighting of experts in airfield management, navigation lighting, and airport operations can be increased; for information system issues, the weighting of experts in information technology can be increased.

[0046] After obtaining the logical consistency index, information content index, factual accuracy index, and suggestion feasibility index, the four indexes are weighted and integrated.

[0047] In this implementation, the logical consistency index and the information content index each account for 20% of the content quality score, while the factual accuracy index and the suggestion feasibility index each account for 30%.

[0048] In the specific calculation, each of the four indicators is multiplied by its respective weight, and then the four weighted results are summed to obtain the content quality score of the feedback report. The above weights can be adjusted according to actual business needs. For example, for reports that only report safety incidents without requiring pilots to provide solutions, the weight of the feasibility of the recommendations can be reduced, while the weight of factual accuracy or completeness of background information can be increased accordingly. The sum of the adjusted weights should remain at 100%.

[0049] Feedback behavior quality is used to evaluate the quantity and trend of feedback from a target evaluation object within a certain statistical period. The target evaluation object can be an individual pilot, a group of pilots, an airline, an airport operating area, or a preset report category.

[0050] First, determine the current statistical time window, which can be 1 day, 1 week, 1 month, or a custom time period. Then, determine the number of feedback reports submitted by the target evaluation object within the current statistical time window. Next, select a baseline period and divide it into multiple unit time windows of the same length as the current statistical time window. Count the number of feedback reports within each unit time window and determine the minimum and maximum numbers. Subtract the minimum number within the baseline period from the number of feedback reports in the current statistical time window, and then divide by the difference between the maximum and minimum numbers within the baseline period to obtain the feedback frequency index. When the result is less than 0, set the feedback frequency index to 0; when the result is greater than 1, set the feedback frequency index to 1. When the number of feedback reports in all unit time windows within the baseline period is the same, resulting in no difference between the maximum and minimum numbers, the feedback frequency index can be set to 0.5, or the baseline period can be expanded and the maximum and minimum numbers redefined.

[0051] The method involves calculating the number of feedback reports submitted by the target evaluation object within the previous statistical time window. The rate of change is obtained by subtracting the number of feedback reports from the previous statistical time window from the number of feedback reports in the current statistical time window, and then dividing by the number of feedback reports from the previous statistical time window. When the number of feedback reports in the current statistical time window is 0, a preset positive number can be used as a smoothing value in the denominator to avoid the inability to perform division calculations. In this embodiment, this smoothing value can be set to 1. The rate of change is input to a smoothed monotonic mapping function with an output range of 0 to 1. This mapping function outputs 0.5 when the rate of change is 0; the output result is greater than 0.5 when the quantity increases; and the output result is less than 0.5 when the quantity decreases. A sensitivity parameter can be set in the mapping function. In this embodiment, the sensitivity parameter is set to 5, so that the month-on-month change in the number of feedback reports can be clearly reflected in the trend indicator.

[0052] In this embodiment, the feedback frequency index has a weight of 70% in the behavior quality score, and the trend index has a weight of 30%.

[0053] In the calculation, the feedback frequency indicator is included at 70%, and the trend indicator is included at 30%. The two weighted results are then added together to obtain the behavior quality score.

[0054] Behavioral quality is used to reflect feedback behavior over a certain period of time, and is not used to evaluate the textual content of a single feedback report. Therefore, when the evaluation object is a single feedback report, the feedback frequency and trend are not calculated for that single report.

[0055] Specifically, when the evaluation object is a single feedback report, the content quality score of that report is directly used as the feedback report quality score. When the evaluation object is a set of feedback reports within a statistical time window, the content quality scores of all feedback reports within the statistical time window are first averaged or weighted averaged to obtain the aggregated content quality score.

[0056] In this implementation, the aggregated content quality score accounts for 60% of the feedback report quality score, and the behavior quality score accounts for 40%. The two scores are weighted according to their respective proportions and then added together to obtain the overall feedback report quality score for the feedback report set. By setting separate evaluation modes for individual reports and report sets, it is possible to avoid forcibly applying indicators such as feedback frequency and trends, which are not suitable for individual reports, to individual reports.

[0057] Airport processing timeliness is determined by both the timeliness of the initial response and the timeliness of the processing cycle.

[0058] The actual initial response time is determined based on the time difference between the feedback report submission time and the airport's initial response time. A maximum response time limit is pre-set according to the report category. For example, the maximum initial response time limit for general reports can be set to 24 hours, or 1440 minutes. When the actual initial response time does not exceed the maximum response time limit, the actual initial response time is first subtracted from the maximum response time limit to obtain the remaining response time. Then, the remaining response time is divided by the maximum response time limit to obtain the initial response timeliness index. When the actual initial response time exceeds the maximum response time limit, the initial response timeliness index is set to 0.

[0059] The actual processing time is determined based on the time difference between the report submission time and the processing completion time. A maximum processing time limit is pre-set according to the report category. For example, the maximum processing time limit for a general report can be set to 14 days, or 20,160 minutes.

[0060] When the actual processing time does not exceed the maximum processing time limit, first subtract the actual processing time from the maximum processing time limit to obtain the remaining processing time, then divide the remaining processing time by the maximum processing time limit to obtain the processing cycle timeliness index. When the actual processing time exceeds the maximum processing time limit, set the processing cycle timeliness index to 0.

[0061] In this implementation, the timeliness of the first response and the timeliness of the processing cycle each account for 50% of the processing timeliness score. The two indicators are each weighted at 50% and then added together to obtain the processing timeliness score for a single airport's processing results.

[0062] For processing results from multiple airports within the statistical time window, the number of processing results whose actual first response time does not exceed the preset maximum response time limit is counted, and this number is divided by the total number of processing results participating in the evaluation to obtain the first response compliance rate. The number of processing results whose actual processing time does not exceed the preset maximum processing time limit is also counted, and this number is divided by the total number of processing results participating in the evaluation to obtain the processing cycle compliance rate. The first response compliance rate and the processing cycle compliance rate are each weighted according to preset weights to obtain a processing timeliness score for the processing results from multiple airports. In this embodiment, the weight of both compliance rates is 50%.

[0063] To determine whether the same problem recurs after the airport has addressed it, the feedback report is subjected to problem identifier matching and text semantic matching. First, structured problem features such as airport identifier, operating area, facility object, problem phenomenon, triggering conditions, and affected objects are extracted from the feedback text.

[0064] For example, for a report that “internal taxiways are difficult to see due to apron reflections in rainy weather,” you can extract information such as airport signs, apron or taxiway areas, taxiway or centerline lights, the problem that is difficult to identify, and the conditions that trigger it in rainy weather.

[0065] The feedback text or a text sequence composed of structured question features is input into the local text encoding model to obtain a question feature vector that represents the semantic content of the question. From subsequent feedback reports processed from the target feedback report, candidate feedback reports with the same associated airport and meeting a preset observation time range are selected. The semantic similarity between the question feature vectors of the target feedback report and the candidate feedback reports is calculated. When the target feedback report and the candidate feedback report share at least one preset structured question feature, and their semantic similarity reaches a preset threshold, the candidate feedback report is identified as a duplicate feedback belonging to the same question as the target feedback report. The preset semantic similarity threshold can be determined based on validation samples. In this embodiment, 0.85 can be used as the judgment threshold. In practical applications, it can also be adjusted between 0.75 and 0.95 depending on the question category. When the feedback report already has a manually set question identifier, duplicate feedback is identified first based on the question identifier. Only when the question identifier is missing or cannot be determined is the structured question features and text semantic vectors used for matching performed.

[0066] The effectiveness of problem-solving is determined by both pilot satisfaction and repeat feedback.

[0067] Furthermore, after the feedback report processing is completed, a 5-point satisfaction survey is sent to the pilots who submitted the report through the feedback platform.

[0068] For a single processing result, the pilot's satisfaction score is divided by the maximum score of 5 to obtain the normalized pilot satisfaction index. For example, a pilot's score of 5 corresponds to a satisfaction index of 1; a pilot's score of 3 corresponds to a satisfaction index of 0.6. For multiple processing results within the statistical time window, the number of valid satisfaction samples with a satisfaction score of 3 or higher is counted, and this number is divided by the total number of valid satisfaction samples actually received to obtain the pilot satisfaction index for the statistical time window. When the satisfaction survey is a mandatory field, the total number of valid satisfaction samples can be equal to the number of completed feedback reports. When the satisfaction survey is not a mandatory field, it is preferable to use the number of valid satisfaction samples actually received as the denominator, and the questionnaire return rate is also recorded.

[0069] For a single processing result, the number of times the problem recurred between the processing completion date and the evaluation time point is counted. This number is then divided by the total number of feedback reports generated between the processing completion date and the evaluation time point to obtain the duplicate feedback index. For multiple processing results within the statistical time window, the number of feedback reports that recurred before the evaluation time point is counted. This number is then divided by the total number of feedback reports participating in the evaluation within the statistical time window to obtain the duplicate feedback index. A higher degree of duplicate feedback indicates that the problem may persist even after the airport has completed its processing. Therefore, when calculating the effectiveness of problem resolution, the duplicate feedback index is reversed: 1 is subtracted from the duplicate feedback index to obtain the degree of problem non-repetition.

[0070] In this implementation, pilot satisfaction and the degree of problem non-repetition each account for 50% of the problem-solving effectiveness score. The pilot satisfaction index is weighted at 50%, the degree of problem non-repetition is weighted at 50%, and the two weighted results are added together to obtain the problem-solving effectiveness score.

[0071] The quality of airport processing results is determined by both processing timeliness and problem-solving effectiveness.

[0072] In this implementation, processing timeliness and problem-solving effectiveness each account for 50% of the airport processing result quality score. The processing timeliness score and problem-solving effectiveness score are each weighted at 50%, and then the two weighted results are summed to obtain the airport processing result quality score. The system writes the feedback report quality score and the airport processing result quality score into the evaluation result database and establishes a correlation with the report identifier, airport identifier, evaluation time, and evaluation mode. The system can also compare the score results with preset thresholds. When the feedback report quality score is lower than the first threshold, an information supplementation identifier or a low-quality identifier is generated for the report; when the airport processing result quality score is lower than the second threshold, an airport processing exception task is generated and written into the pending review task queue. Example

[0073] The following example, using a pilot feedback report and the airport's handling result, illustrates the method used in this application.

[0074] Feedback report and handling status: On October 10, 2025, a pilot submitted the following feedback report through the feedback platform: “The internal taxiways L1, L1, J, K, etc. do not have taxi lights, and the taxi routes are extremely difficult to see due to the reflection of the apron on rainy days.” The relevant airport responded on October 13, 2025: "Due to initial design reasons, taxiway centerline lights are not installed on taxiways L1, K, and J at the airport. This information is noted in the airport's detailed regulations. Taxiway L2 is equipped with taxiway centerline lights. The airport previously convened a special discussion with operations, ground support, and airlines, concluding that installing deep-cylinder centerline lights on the apron would have a significant impact on airport operations, and after assessment, it is currently not feasible. In response to the issues raised, the airport will increase the refresh frequency of taxiway lines at key locations on the apron to enhance visibility. During rainy or nighttime flight conditions, apron control will provide additional warnings; if aircraft require guidance, the airport can dispatch guidance vehicles." The final status of the feedback report is "Completed".

[0075] The system associates feedback text, submission time, airport viewing time, processing completion time, expert rating, satisfaction rating, and subsequent feedback data into the same evaluation sample based on the report identifier.

[0076] Feedback report content quality evaluation: After segmenting the feedback text into clauses and constructing clause pairs, each clause pair is input into the local natural language inference model. The model did not identify any contradictory clause pairs, therefore the proportion of contradictory clause pairs is 0. After reversing the processing of the contradictory clause proportion, the logical consistency index is 1.

[0077] The feedback text was segmented, word frequency counted, and information entropy calculated, yielding an information entropy of 3.95. After normalization based on the information entropy distribution in the benchmark report set, the information content index was 0.96.

[0078] Airport operations experts evaluated the accuracy of the facts. The evaluation criteria included: objectivity (5 points), completeness of background information (3 points), and specificity of the problem description (3 points).

[0079] In the calculation, the objective truthfulness score of 5 points is included at 40%, which is converted to 2 points; the completeness of background elements score of 3 points is included at 30%, which is converted to 0.9 points; and the specificity of the problem description score of 3 points is included at 30%, which is converted to 0.9 points. The total of the three weighted results is 3.8 points, which is then divided by the full score of 5 points to obtain the factual accuracy index of 0.76.

[0080] Airport operations experts evaluated the feasibility of the recommendations. The evaluation included: technical feasibility (3 points), resource feasibility (1 point), goal alignment (3 points), and value and impact (3 points).

[0081] In the calculation, technical feasibility (3 points) is weighted at 25%, equivalent to 0.75 points; resource feasibility (1 point) is weighted at 25%, equivalent to 0.25 points; goal consistency (3 points) is weighted at 30%, equivalent to 0.9 points; and value and impact (3 points) are weighted at 20%, equivalent to 0.6 points. The total weighted score of these four factors is 2.5 points. Dividing this by the maximum score of 5, the suggested feasibility index is 0.5.

[0082] When determining the content quality score, the logical consistency index 1 is included at 20%, resulting in a score of 0.2; the information content index 0.96 is included at 20%, resulting in a score of 0.192; the factual accuracy index 0.76 is included at 30%, resulting in a score of 0.228; and the suggestion feasibility index 0.5 is included at 30%, resulting in a score of 0.15.

[0083] Adding the four results together, the content quality score of the feedback report is 0.77.

[0084] This evaluation focuses on a single feedback report, without considering feedback frequency or trends. Therefore, the content quality score is directly used as the quality score for this feedback report. The quality score for this particular feedback report is 0.77.

[0085] The results indicate that the report contains no obvious logical contradictions, is rich in information, and can objectively reflect the problems existing in the airport operation. However, the proposed facility renovations are subject to certain limitations in terms of resource investment and implementation conditions.

[0086] Airport processing timeliness evaluation: The feedback report was submitted at 12:13 PM on October 10, 2025. The airport generated its first valid response event at 4:25 PM on October 10, 2025. The valid response event was defined as the airport's business system sending acceptance confirmation information to the feedback submitter. Therefore, 4:25 PM on October 10, 2025 is determined as the first response time.

[0087] The difference between the feedback report submission time and the initial response time was 252 minutes, therefore the actual initial response time was 252 minutes.

[0088] In this embodiment, the maximum first response time is set to 1440 minutes, or 24 hours.

[0089] Subtracting 252 minutes from 1440 minutes gives a remaining response time of 1188 minutes. Dividing 1188 minutes by 1440 minutes gives an initial response timeliness index of 0.825.

[0090] The feedback report was submitted at 12:13 on October 10, 2025, and the report status changed to "completed" at 17:23 on October 13, 2025, with an actual processing time of 4630 minutes.

[0091] The maximum processing time set in this embodiment is 20,160 minutes, or 14 days.

[0092] Subtracting 4630 minutes from 20160 minutes gives a remaining processing time of 15530 minutes. Dividing 15530 minutes by 20160 minutes gives a processing cycle timeliness index of approximately 0.77.

[0093] The timeliness of the first response (0.825) is weighted at 50%, and the timeliness of the processing cycle (0.77) is weighted at 50%. The sum of the two weighted results gives the airport a processing timeliness score of approximately 0.798.

[0094] Evaluation of the effectiveness of problem-solving: After the feedback report processing was completed, the platform sent a satisfaction survey to the pilot who submitted the report. The pilot gave a satisfaction rating of 5 out of 5.

[0095] Dividing the satisfaction score of 5 by the maximum score of 5, we get a pilot satisfaction index of 1.

[0096] After processing the initial report, the system performed problem identification and text semantic matching on subsequent feedback reports up to the evaluation time. No subsequent reports were found to meet the matching requirements for airport identification, operating area, facility object, problem phenomenon, and triggering conditions, nor were any similar problems found with semantic similarity reaching the preset threshold. Therefore, the number of times this problem recurred was 0, and the duplicate feedback index was 0.

[0097] After reverse processing of the duplicate feedback index, the degree of non-duplication of the problem is 1.

[0098] If the pilot satisfaction index 1 is included at 50%, and the degree of non-repetition of the problem 1 is included at 50%, the problem-solving effectiveness score is 1.

[0099] Airport processing result quality evaluation: The timeliness score of 0.798 is weighted at 50%, resulting in a score of 0.399; the problem-solving effectiveness score of 1 is weighted at 50%, resulting in a score of 0.5. Adding the two weighted results together, the airport's overall processing quality score is 0.899.

[0100] The evaluation results show that the quality score of this feedback report is 0.77. The report has high logical consistency and information content, and the factual accuracy is good, but the implementation of the recommendations is subject to certain resource constraints. The airport's handling result quality score is 0.899, indicating that the airport was able to complete the response and handling within the stipulated time, the pilots were satisfied with the handling result, and no recurrence of the same problem was found as of the evaluation time. Compared with the effectiveness of problem-solving, there is still room for improvement in the timeliness of handling.

[0101] Storage and application of evaluation results: After the evaluation is completed, the system generates a two-way quality assessment result record. This record includes at least the following indicators: logical consistency, information content, factual accuracy, feasibility of recommendations, content quality score, behavioral quality score, feedback report quality score, timeliness of initial response, timeliness of processing cycle, processing efficiency score, pilot satisfaction, repeat feedback, problem-solving effectiveness score, and airport processing result quality score.

[0102] For a single feedback report, inapplicable behavior quality scores can be set to null, and "Single Report Evaluation" should be noted in the evaluation mode field. For a collection of reports within a statistical time window, "Report Collection Evaluation" should be noted in the evaluation mode field. The system can sort reports according to their feedback report quality scores, prioritizing reports with higher quality, more comprehensive information, and clearer factual basis for relevant processing departments. For reports with lower content quality, the system can prompt the submitter to supplement information such as time, location, triggering conditions, operation process, and scope of impact. The system can also identify processing tasks with insufficient timeliness, low satisfaction, or recurring issues based on the airport processing result quality score, and push the corresponding tasks to airport operations management, safety management, or responsible departments for review. Through the above processing, pilot feedback report evaluations and airport processing result evaluations can be linked, forming a continuous data evaluation chain in the process of feedback report submission, processing, feedback, and improvement.

[0103] It should be noted that, in the above implementation, the local natural language reasoning model is used to determine whether two valid clauses in the same feedback report have an implication relationship, a contradiction relationship, or a neutral relationship.

[0104] In one specific implementation, the local natural language inference model includes a text input processing unit, a lexical encoding unit, a semantic encoding unit, and a classification output unit. The text input processing unit receives a clause pair consisting of two valid clauses, using the first clause as the premise text and the second clause as the text to be judged, and forms the model input text in the order of "first clause, separator, second clause". Subsequently, the model input text is segmented into lexical units, and lexical identifiers and attention mask information corresponding to each lexical unit are generated.

[0105] When the model has the function of segment differentiation, different segment identifiers are set for the first clause and the second clause respectively, so that the model can identify the position of the two clauses in the input text.

[0106] In one specific implementation, the length of the input text processed by the model at one time is limited to no more than 256 tokens. When the input text exceeds 256 tokens, the core entities, time information, location or region information, facility objects, problem phenomena, triggering conditions, and impact information in the first and second clauses are retained, while non-core modifying content exceeding the input length limit is deleted, ensuring that the final input length does not exceed 256 tokens. The token encoding unit converts each token in the input text into a numerical vector. The semantic encoding unit consists of several sequentially connected semantic encoding layers, each of which includes at least an attention processing unit, a feedforward processing unit, a normalization unit, and a residual connection unit. The semantic encoding unit jointly encodes the two clauses, allowing the word relationships within the same clause and between the two clauses to participate in the semantic representation calculation. After completing the semantic encoding, the classification output unit generates confidence levels corresponding to three classification results—implication relationship, contradictory relationship, and neutral relationship—based on the joint semantic representation, and determines the relationship category of the clause pair as the one with the highest confidence level among the three classification results.

[0107] In another method, a threshold for determining contradictory relationships is pre-set. When the confidence level corresponding to a contradictory relationship reaches or exceeds the threshold, the corresponding clause pair is identified as contradictory. When the confidence level corresponding to a contradictory relationship does not reach the threshold, the final classification result is determined based on the confidence levels corresponding to implication and neutral relationships. The threshold for determining contradictory relationships is determined based on pre-labeled validation samples. In one specific embodiment, 50%, 60%, 70%, and 80% are used as candidate thresholds, respectively. The accuracy and recall of contradictory relationship identification under each candidate threshold are statistically analyzed, and the candidate threshold that simultaneously meets the preset requirements for accuracy and recall is selected as the actual threshold for determining contradictory relationships.

[0108] In this embodiment, the natural language reasoning model is deployed on a local server or local computing node at the airport, and the feedback text does not need to be sent to a public model service outside the airport.

[0109] In one specific implementation, the natural language inference model uses a pre-trained language model that has already completed natural language inference task training as the base model, and further trains it using airport operations domain text. During airport operations domain training, clause pairs are extracted from historical pilot feedback reports, and each clause pair is labeled with implication, contradiction, or neutral relationships. Each clause pair is initially labeled by two annotators with expertise in airport or aviation operations. When the results of the two annotators are consistent, the consistent result is used as the sample label; when the results of the two annotators are inconsistent, a third person with relevant expertise reviews the label, and the final label is determined based on the review result. After labeling, the labeled samples are divided into training samples, validation samples, and test samples according to a preset ratio. In one specific implementation, training samples account for 70% of all labeled samples, validation samples account for 15%, and test samples account for 15%.

[0110] During training, the difference between the relation categories output by the model and the manually labeled relation categories is used as the basis for parameter adjustment, and training samples are repeatedly input in batches. In one specific implementation, each training batch includes 16 clause pairs, and all training samples participating in one parameter update process is counted as one round of training, for a total of 5 rounds of training. After each round of training, the classification accuracy and recall are calculated using validation samples. When the validation results do not meet the preset improvement conditions after two consecutive rounds of training, training ends and the model parameters that meet the preset requirements are saved. After the model is deployed, each clause pair in a feedback report is classified separately. The system counts the total number of clause pairs participating in the judgment and the number of clause pairs identified as contradictory relationships. Then, the number of contradictory clause pairs is divided by the total number of clause pairs participating in the judgment to obtain the proportion of contradictory relationships. The logical consistency index is then obtained by subtracting the contradictory relationship proportion from the value of 1.

[0111] For example, if a feedback report contains 10 valid clause pairs, and two of these clause pairs are judged to be contradictory, then the contradictory relationship accounts for 20%, and the logical consistency index is 80%.

[0112] When the feedback text forms only one valid clause and cannot construct a clause pair, clause relationship classification is not performed, and the logical consistency index is set to 1 according to the preset boundary processing rules. This value only indicates that no internal contradictions were found between clauses; information content, factual accuracy, and feasibility of suggestions are still calculated according to their respective evaluation rules.

[0113] In the above implementation, the local text encoding model is used to convert the problem descriptions in the pilot feedback reports into fixed-length problem feature vectors in order to determine whether the two feedback reports describe the same airport operation problem.

[0114] In one specific implementation, the local text encoding model includes a text preprocessing unit, a lexical encoding unit, a semantic encoding unit, a vector convergence unit, and a vector normalization unit. Before inputting into the local text encoding model, the system first extracts airport identification, operating area, facility objects, problem phenomena, triggering conditions, and affected objects from the feedback text. The structured problem features include at least two of the above six types of features.

[0115] Among them, airport identification is used to identify the airport to which the reported problem belongs; operating area is used to identify where the problem occurred on the runway, taxiway, apron, parking stand, terminal area or other airport operating area; facility object is used to identify the taxiway centerline lights, taxiways, signs, runway lights, guidance facilities, information systems or other operating facilities involved in the problem; problem phenomenon is used to identify the specific abnormal situation that occurred; triggering conditions are used to identify the weather, time, visibility, operating direction or traffic conditions at the time the problem occurred; affected objects are used to identify the aircraft, pilots, operating vehicles or other operating participants affected by the problem.

[0116] In one specific implementation, the structured problem features are arranged in a fixed order: airport identification, operating area, facility object, problem phenomenon, triggering condition, and affected object. Taking "internal taxiways are difficult to see the taxiway route due to apron reflections in rainy weather" as an example, the system organizes it into the following structured content: The airport sign indicates the target airport; The operating area is the internal taxiway; The facility in question is a taxiway signage facility; The problem is difficulty in identifying the gliding path; The trigger condition is rainy weather accompanied by tarmac reflection; The affected entities are aircraft and pilots operating in the relevant area.

[0117] For fields where the corresponding content does not exist in the feedback text, the field is marked as missing, and facts not present in the original feedback text are not automatically filled in through inference. After completing the structuring process, the original feedback text or the structured question text formed according to a fixed field order is input into the local text encoding model.

[0118] In one specific implementation, both the original feedback text and the structured question text are used simultaneously. The system performs semantic encoding on both parts of the text separately and connects the encoding results of the two parts in a preset order to form a unified question semantic representation. The lexical encoding unit first segments the input text into lexical units and converts each lexical unit into a numerical vector. The semantic encoding unit performs contextual encoding on the lexical relationships in the input text to obtain the semantic representation corresponding to each lexical unit. The vector aggregation unit averages the semantic representations of all valid lexical units. Specifically, the process is as follows: first, the values ​​corresponding to the same position of each valid lexical unit are added together, and then divided by the total number of valid lexical units to form a fixed-length text semantic vector. The vector normalization unit then normalizes the text semantic vector so that the vectors generated by different feedback reports can be compared for similarity according to a unified calculation rule.

[0119] In one specific implementation, the problem feature vector contains 768 numerical positions. For implementations using other local text encoding models, the number of numerical positions in the problem feature vector is determined according to the output dimension of the corresponding model, but the target feedback report and candidate feedback report used in the same repeated problem identification process use the same text encoding model and the same vector dimension. When the general text encoding model cannot identify airport operation professional expressions according to the preset accuracy, the text encoding model is trained in the airport operation domain using historical feedback reports. Domain training samples are constructed according to problem text pairs. When two feedback reports are manually confirmed to belong to the same problem, or have the same problem identifier, they are considered as training samples of the same type of problem; when two feedback reports correspond to different airports and do not have cross-airport unified facility problems, or their operating areas, facility objects, and problem phenomena are all different, they are considered as training samples of different problems. During training, the two problem feature vectors generated by the training samples of the same type of problem are adjusted to the same semantic direction, and the problem feature vectors generated by the training samples of different problems are adjusted to different semantic directions, so that the model can form a distinguishing result based on the actual semantic content of the airport operation problem.

[0120] In one specific implementation, each training batch includes 32 question text pairs, and a total of 5 rounds of training are performed. After each round of training, the accuracy and recall of duplicate question identification are statistically analyzed using validation samples, and model parameters that meet preset requirements are saved based on the validation results.

[0121] In the above implementation, the target feedback report and the candidate feedback report are processed by the same local text encoding model to obtain problem feature vectors of the same length. The system calculates the semantic similarity according to the correspondence between the numerical positions in the vectors.

[0122] Specifically, firstly, the first value in the feature vector of the target feedback report is multiplied by the first value in the feature vector of the candidate feedback report, then the second value is multiplied by the second value, and so on, processing all corresponding value positions in the same way, and then summing all the products. Next, the vector lengths of the feature vectors of the target and candidate feedback reports are calculated separately. The vector length is obtained through the following steps: multiplying each value in a problem feature vector by itself, summing all the results, and then taking the square root of the sum. The semantic similarity between the two feedback questions is obtained by dividing the sum of the products of corresponding positions in the two problem feature vectors by the product of the lengths of the two problem feature vectors.

[0123] When the problem feature vectors have been normalized to a unit length after encoding, the products of all corresponding positions of the two problem feature vectors can be directly added together, and the result is used as the semantic similarity. Before performing semantic similarity calculation, candidate feedback reports are first screened according to temporal and airport association conditions. Only feedback reports generated after the target feedback report has been processed are considered as candidate reports that meet the temporal conditions. In one specific implementation, the observation period for repeated questions is set to 180 days from the date the target feedback report has been processed. At the same time, only feedback reports corresponding to the same airport as the target feedback report are retained. Feedback reports with inconsistent airport identifiers are not included in the repeated question semantic matching process of the current target feedback report. After completing the screening based on temporal and airport association conditions, the structured problem features of the target feedback report and each candidate feedback report are compared.

[0124] In one specific implementation, the airport identifiers must be consistent, and at least one of the following must be consistent or belong to a pre-established equivalent expression relationship before the semantic similarity judgment can continue.

[0125] For example, "rainy day" and "rain conditions" are pre-registered as the same trigger condition; "gliding route is difficult to identify" and "gliding route is not visible" are pre-registered as the same problem phenomenon category.

[0126] In one specific implementation, 85% is used as the semantic similarity threshold. When the structured question features meet the above matching conditions, and the semantic similarity between two question feature vectors reaches or exceeds 85%, the candidate feedback report is identified as a duplicate feedback belonging to the same question as the target feedback report. The semantic similarity threshold is determined through validation samples. Specifically, 75%, 80%, 85%, 90%, and 95% are used as candidate thresholds to identify manually confirmed samples of the same question and samples of different questions, and the recognition accuracy and recall corresponding to each candidate threshold are calculated. The candidate threshold whose accuracy and recall meet the preset business requirements is selected as the actual operating threshold.

[0127] When a feedback report has a pre-set problem identifier, the problem identifiers are compared first. If the problem identifiers of the target feedback report and the candidate feedback report are the same, they are identified as the same problem. If the problem identifier is missing or the problem identifiers cannot form a definite matching relationship, then structured problem feature matching and semantic similarity judgment are performed. By first performing time range filtering, airport association filtering, and structured feature filtering, and then performing problem feature vector similarity judgment, feedback reports that do not belong to the target airport, are not in the evaluation observation period, or have significantly different problem objects can be excluded, so that semantic similarity calculation is focused on candidate reports that meet the basic association conditions.

[0128] In the above implementation, the trend indicator is determined based on the number of feedback reports in the current statistical time window and the number of feedback reports in the previous statistical time window.

[0129] First, count the number of feedback reports corresponding to the target evaluation object within the current statistical time window. Then, count the number of feedback reports corresponding to the target evaluation object within the previous statistical time window, which has the same length as the current statistical time window. If the number of feedback reports in the previous statistical time window is not zero, subtract the number of feedback reports in the previous statistical time window from the number of feedback reports in the current statistical time window to obtain the change in the number of feedback reports. Then, divide this change in the number of feedback reports in the previous statistical time window to obtain the rate of change in the number of feedback reports.

[0130] For example, if there were 10 feedback reports in the previous statistical time window and there are 15 feedback reports in the current statistical time window, then first subtract 10 from 15 to get an increase of 5 reports; then divide 5 by 10 to get a 50% change rate in the number.

[0131] When there are no feedback reports within the current statistical time window, in order to avoid the division by zero, the denominator is set to 1 according to the preset smoothing rule.

[0132] For example, if the number of feedback reports in the previous statistical time window was 0, and there are 3 feedback reports in the current statistical time window, subtracting 0 from 3 gives a change in quantity of 3. Then, dividing 3 by 1 gives the smoothed rate of change. After obtaining the rate of change, inputting it into a logistic smoothing mapping function converts the rate of change into a trend indicator between 0 and 1.

[0133] The smooth mapping function satisfies the following deterministic relationship: When the rate of change in quantity is 0, the trend indicator is 0.5. When the rate of change in quantity is positive, the trend indicator is greater than 0.5; When the rate of change in quantity is negative, the trend indicator is less than 0.5; When the absolute value of the rate of change of quantity increases, the trend indicator gradually approaches the boundary value in the corresponding direction, but does not directly exceed the range of 0 to 1 due to a single change in quantity.

[0134] In one specific implementation, the sensitivity parameter of the smoothing mapping function is set to 5. This parameter is stored fixedly in the evaluation rule configuration table and remains unchanged within the same evaluation period.

[0135] Through the above processing, the original rate of change is uniformly converted into a trend indicator between 0 and 1, allowing the trend indicator and the feedback frequency indicator to use the same numerical range for subsequent weighted calculations. The feedback frequency indicator for the baseline period is calculated as follows: First, determine the number of feedback reports in each unit time window within the baseline period, and identify the minimum and maximum numbers. Subtract the minimum number within the baseline period from the number of feedback reports in the current statistical time window, and then divide the difference by the difference between the maximum and minimum numbers within the baseline period to obtain the feedback frequency indicator.

[0136] When the calculation result is less than 0, the feedback frequency index is set to 0; when the calculation result is greater than 1, the feedback frequency index is set to 1. When the number of feedback reports in each unit time window within the baseline period is exactly the same, making the difference between the maximum and minimum numbers zero, the feedback frequency index is set to 0.5, or the baseline period is extended according to preset rules and recalculated. In actual operation, one of the processing rules is predetermined for the same evaluation task and remains consistent throughout the entire evaluation cycle.

[0137] In the above implementation, the first response time is the time when the airport processing unit first generates a valid response event to the feedback report submitted by the pilot.

[0138] The valid response events include the airport processor's first sending of a processing reply to the feedback submitter, the first generation of formal acceptance feedback, the first sending of an acceptance status notification to the feedback submitter, or other events recorded by the airport's business system that indicate the airport has formed a formal processing response to the feedback report. The system determines the initial response event based on the report identifier, event type, event occurrence time, and event status in the airport processing event data.

[0139] When the same feedback report corresponds to more than two response events, events whose types fall within the preset range of valid response events are first filtered out. Then, they are sorted according to their occurrence time, and the occurrence time of the earliest valid response event is determined as the initial response time. If airport staff only open or view the feedback report but do not generate acceptance information, status feedback, or processing reply to the feedback submitter, this viewing event is not considered an initial response event. When the airport business system is configured to automatically send acceptance confirmation information to the feedback submitter when a staff member views the feedback report for the first time, this initial viewing event simultaneously constitutes a valid response event. Under this business configuration, the initial viewing time and the initial response time have the same time value. Therefore, in the embodiments described in the specification, the initial response time should be clearly determined based on the actual business events.

[0140] When the event corresponding to 16:25 on October 10, 2025 only indicates that the staff opened the report, the time when the first formal acceptance information or processing reply was generated thereafter should be used as the first response time.

[0141] When the initial viewing operation at 16:25 on October 10, 2025 simultaneously triggers the system to send a notification to the pilot that "report has been accepted" or has the same processing meaning, 16:25 on October 10, 2025 will be determined as the initial response time.

[0142] By adopting the above event identification method, the first response time corresponds to a clear airport processing behavior, avoiding confusion between the internal operation of simply browsing the report and the formal response to the feedback submitter.

[0143] In the above implementation, natural language reasoning processing is performed on the factual descriptions of airport operations in the pilot feedback report, rather than classifying general text unrelated to airport operations. Valid clauses in the feedback text include one or more of the following: airport location, facility status, weather conditions, operating conditions, operational procedures, problem phenomena, and impact outcomes. The system first segments the complete feedback text into valid clauses, and then determines the relationships between the clauses.

[0144] For example, if a feedback report contains two clauses at the same time, namely "the target taxiway does not have centerline lights" and "the target taxiway has centerline lights that are working properly", and the two clauses describe opposite states of the same facility under the same evaluation conditions, then the natural language inference model will identify the two clauses as contradictory.

[0145] When a feedback report contains two clauses, "Difficulty in identifying target taxiway routes in rainy weather" and "Airport plans to increase taxiway refresh frequency," these two clauses describe the problem phenomenon and the corresponding handling measures, respectively. They do not constitute contradictory descriptions of the same factual state and are therefore not considered contradictory according to the aforementioned rules. By performing a pairwise relationship judgment on each clause, conflicting descriptions within the report concerning the same airport operation object, the same state, or the same condition can be identified individually. A logical consistency index is then formed based on the number of contradictory relationships in all clause pairs. In the process of identifying duplicate problems, the system first determines the scope of candidate feedback reports based on airport identification and time conditions, then extracts structured problem features such as operating area, facility object, problem phenomenon, triggering conditions, and affected objects, and finally performs text semantic similarity judgment.

[0146] For example, the target feedback report describes "the taxiway routes on the internal taxiway cannot be clearly identified in rainy weather," while the candidate feedback report describes "the route markings in the internal taxiway area are unclear under rainy conditions." Both reports correspond to the same airport, and the candidate feedback report is generated within a preset observation period after the target feedback report has been processed. Both reports indicate the internal taxiway area as the operating area, the problem is difficulty in route identification, and the triggering condition is rainy conditions. After performing structured feature matching, the system calculates the semantic similarity of the feature vectors of the two problems. When the semantic similarity reaches a preset threshold, the candidate feedback report is identified as a duplicate feedback of the same problem.

[0147] In cases where the target feedback report describes "difficulty in identifying internal taxiway routes" and the candidate feedback report describes "incorrect flight information displayed in the terminal," the corresponding operating areas, facilities, and problems in the two reports do not meet the preset structured matching conditions. Therefore, they are not included in the final semantic judgment result for the same problem. Through the above processing, the model input directly comes from the operating objects, operating conditions, and problem phenomena in the airport feedback reports, and the model output continues to be used for logical consistency judgment or duplicate problem judgment, thus forming a continuous data processing relationship between the natural language processing results and the airport feedback report quality evaluation and the airport processing result evaluation.

[0148] Data and Equipment Description: The feedback text comes from anonymized historical feedback reports from airports under an airport group from 2024 to 2025; both the natural language inference model and the text encoding model are deployed on local computing nodes at the airport (CPU inference, not dependent on external networks); expert scoring is done independently by three flight area operation managers. All experiments can be re-executed at similar airports using the same procedures, without involving computations exceeding conventional computing power.

[0149] Experiment 1: Consistency Verification of Content Quality Evaluation Experimental Setup: Sample: 120 randomly selected historical pilot feedback reports (covering common categories such as taxiway lighting, apron markings, information systems, and guidance services), anonymized; Method Scoring: Logical consistency, information content, factual accuracy, and suggestion feasibility indicators were calculated sequentially according to patent steps S2-S4, and then weighted at 20% / 20% / 30% / 30% to form a content quality score; Expert Scoring: Three airport operations experts independently scored the reports according to the evaluation dimensions specified in the patent (objective authenticity 40%, background element completeness 30%, problem description specificity 30%; technical feasibility 25%, resource feasibility 25%, goal consistency 30%, value and impact 20%), and the average of the three scores was normalized to 0-1; Evaluation Indicators: Pearson correlation coefficient, Spearman rank correlation coefficient, mean absolute error (MAE), and root mean square error (RMSE).

[0150] Experimental results: Table 1 shows the comparison between the indicators of this method and the mean expert scores on the four content quality dimensions.

[0151] Table 1. Content Quality Dimensions: Comparison of Indicators from This Method and Mean Expert Scores (n=120)

[0152] The overall consistency statistics between the content quality score and the expert overall score (normalized) of this method are shown in Table 2.

[0153] Table 2. Overall consistency between the scoring method and expert scoring (n=120)

[0154] like Figure 4 and Figure 6As shown, the difference between the method and the expert scores in all four dimensions does not exceed 0.02. The two broken lines in the dumbbell chart almost overlap, indicating that the automatically calculated indicators are highly consistent with the subjective evaluation of experts. The Pearson correlation coefficient r=0.869 (Spearmanρ=0.874) reaches the level of strong correlation, indicating that the method can effectively reproduce the overall judgment of experts on the quality of reports. MAE=0.073 and RMSE=0.089 are within the acceptable deviation range under the 0-1 scoring system. This method can complete the evaluation of report content quality in batches and objectively without increasing the workload of manual review, and experts can concentrate their energy on a few reports that need to be reviewed.

[0155] Experiment 2: Threshold Sensitivity Verification for Repeated Feedback Recognition Experimental Setup: Sample: 100 problem report pairs were constructed according to the method described in Patent S9, of which 35 pairs were manually confirmed to belong to the same problem (e.g., the L1 taxiway is not visible due to glare in rainy weather, and the internal taxiway is difficult to identify the taxiway route during rainfall), and 65 pairs belong to different problems; Features and Vectors: Structured problem features (at least 2 categories per pair) such as airport identification, operating area, facility object, problem phenomenon, triggering condition, and affected object were extracted from the feedback text, and a 768-dimensional problem feature vector was generated by the local text encoding model. Cosine similarity was used to calculate semantic similarity; Screening Criteria: Only candidate pairs that are from the same airport, within a 180-day observation period after the target report is processed, and have at least one consistent structured feature were retained to simulate real operating conditions; Variation Parameters: The semantic similarity thresholds were successively set to 0.75, 0.80, 0.85, 0.90, and 0.95; Evaluation Indicators: Precision, Recall, and F1 score. The experimental results are shown in Table 3.

[0156] Table 3. Performance of duplicate feedback identification under different semantic similarity thresholds (100 report pairs)

[0157] Figure 5The graph shows the change in repetitive feedback recognition performance as a function of semantic similarity threshold in this embodiment of the invention (100 report pairs). When the threshold increases from 0.75 to 0.95, the precision increases monotonically (0.739→1.000), while the recall decreases monotonically (0.971→0.686), exhibiting a typical precision-recall tradeoff. The F1 score reaches its maximum value of 0.914 at the threshold of 0.85, at which point both precision and recall reach 0.914. This perfectly matches the description in the patent specification that 0.85 is used as the default judgment threshold and can be adjusted between 0.75 and 0.95 according to the problem category. It is also consistent with the candidate threshold set (0.75, 0.80, 0.85, 0.90, 0.95) selected for the validation samples. This result demonstrates that under the three-level screening process of first completing time range screening, airport association screening, and structured feature screening, and then performing vector similarity judgment, the algorithm can detect similar problems that reappear after processing with high accuracy.

[0158] Experiment 3: Verification of the Application Effect of Two-Way Quality Assessment Experimental Setup: Sample: 120 feedback reports—airport processing result records, from anonymized data of the same airport from January to August 2025. Each record includes a feedback report quality score (content quality 60% + behavior quality 40%, single report evaluation mode) and an airport processing result quality score (processing timeliness 50% + problem-solving effectiveness 50%); Evaluation Thresholds: The first and second scoring thresholds are both set at 0.6 according to patent S8; Four Status Classifications: High-quality feedback report quality ≥ 0.6 and airport processing result quality ≥ 0.6 (high-quality feedback - efficient processing); Report Excellent - Processing Needs Improvement: Feedback report quality ≥ 0.6 and airport processing result quality < 0.6; Report to be supplemented - Efficient processing: Feedback report quality <0.6 and airport processing result quality ≥0.6; Low scores: Both scores are <0.6 (key focus), and the experimental results are shown in Table 4.

[0159] Table 4 Distribution of Two-Way Quality Assessment Results (120 Feedback-Processing Records, Evaluation Threshold 0.6)

[0160] Figure 7The quadrant diagram showing the distribution of two-way quality assessment results in this embodiment of the invention (120 feedback-processing records, evaluation threshold 0.6) indicates that 50.0% (60 records) are high in both aspects, meaning that more than half of the airport's feedback-processing links are operating well; 23.3% (28 records) are low in both aspects, representing key targets requiring priority review (simultaneously exhibiting insufficient reporting information and untimely / recurring issues). The quadrant diagram visually shows that the two dimensions are not redundant: there are two non-redundant categories: excellent reporting - processing needs improvement (9 records) and reports need supplementation - efficient processing (23 records). The status quo demonstrates that both the quality of feedback reports and the quality of airport processing results must be evaluated simultaneously to fully depict the operational chain; one-way evaluation will miss one type of problem. The results verify the application value of two-way quality assessment: the system can prioritize pushing high-risk reports to relevant processing departments as reference cases, prompt the submitter to supplement background information such as time, location, and triggering conditions for reports that need to be supplemented, and generate airport processing exception tasks for reports that are good but need improvement and low-risk records and write them into the queue for review, thus realizing a data closed loop between the feedback report submission, processing, feedback and improvement stages.

[0161] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A two-way quality assessment method for pilot feedback reports and airport processing results, characterized in that, include: S1. Obtain pilot feedback report data, airport incident handling data, expert evaluation data, and satisfaction data, and generate evaluation samples according to the report identifiers; S2. Segment the feedback text into clauses and construct clause pairs, input them into the local natural language reasoning model, and determine the logical consistency index based on the proportion of contradictory relationships in the clause pairs. S3. Perform word segmentation and information entropy calculation on the feedback text to obtain information content indicators; S4. Combine expert evaluations to obtain factual accuracy indicators and suggestion feasibility indicators, and then weight them to obtain a content quality score; S5. Within the statistical time window, the feedback frequency index is determined based on the normalization result of the number of feedback reports relative to the extreme value of the number of the baseline period. The trend index is determined based on the change in the number of the current statistical time window and the previous statistical time window and the Sigmoid function. Then, the behavior quality score is obtained and weighted with the content quality score to obtain the feedback report quality score. S6. Determine the processing timeliness score based on the report submission time, initial response time, and processing completion time; S7. Determine the effectiveness score for problem-solving based on pilot satisfaction and repeated feedback identified using problem identifiers or text semantic vectors, and then weight them to obtain the airport processing result quality score. S8. Output a two-way quality assessment result containing the feedback report quality score and the airport processing result quality score.

2. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The generation of evaluation samples based on the report identifier includes: Unify the time zone and time unit for time fields from different data sources; Identify and delete duplicate records based on report identifiers, feedback text, and submission times; Identify abnormal records based on the temporal relationship between submission time, first response time, processing completion time, and processing status, and add data abnormality identifiers to the abnormal records; Anonymize pilot and expert identification identifiers; Establish correlations between pilot feedback report data, airport incident handling data, expert evaluation data, and satisfaction data based on report identifiers.

3. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The determination of the logical consistency index includes: The feedback text is divided into multiple valid clauses according to a preset text delimiter, and clause pairs are constructed based on the valid clauses; Each clause pair is input into the local natural language reasoning model to obtain the classification results of each clause pair corresponding to implication relation, contradiction relation, and neutral relation, respectively; The percentage of clause pairs identified as contradictory is counted among all clause pairs, and the percentage is reverse-mapped to obtain the logical consistency index. When there are no constructible clause pairs in the feedback text, the logical consistency index is determined according to the preset boundary processing rules.

4. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The determination of the information content index includes: The feedback text is segmented and invalid characters are filtered to obtain multiple valid terms; The frequency of occurrence of each effective term is counted, and the text information entropy is calculated based on the probability distribution of each effective term in the feedback text. Based on the information entropy distribution in the preset benchmark report set, the text information entropy is normalized to obtain an information quantity index within a preset numerical range. When the information entropy in the benchmark report set does not have an effective distinguishing range, the information content index is determined according to the preset boundary processing rules.

5. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The indicators for factual accuracy and recommendations for feasibility obtained by combining expert evaluation include: Obtain the first set of evaluation data to assess the objectivity, authenticity, completeness of background elements, and specificity of problem description in the feedback report. Normalize and weight the first set of evaluation data to obtain the factual accuracy index. Obtain a second set of evaluation data to assess the technical feasibility, resource feasibility, goal consistency, and value impact of the feedback recommendations. Normalize and weight the second set of evaluation data to obtain the feasibility index of the recommendations. The logical consistency index, information content index, factual accuracy index, and suggestion feasibility index are integrated according to preset weights to obtain the content quality score; When the same evaluation dimension corresponds to multiple expert evaluation results, the multiple expert evaluation results are aggregated based on at least one of the following: expert professional category, professional experience, and degree of professional matching with the problem to be evaluated.

6. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The determination of the behavioral quality score includes: The number of feedback reports corresponding to the statistical target evaluation object within the current statistical time window; Obtain the number of feedback reports corresponding to multiple unit time windows within the baseline period, and normalize the number of feedback reports in the current statistical time window relative to the extreme values ​​of the number within the baseline period to obtain the feedback frequency index. The rate of change of quantity is determined based on the number of feedback reports in the current statistical time window and the previous statistical time window, and the rate of change of quantity is mapped to a preset numerical range using the Sigmoid function to obtain a trend index. The feedback frequency index and the trend index are fused according to preset weights to obtain a behavior quality score; When there is no effective range in the number of feedback reports within the baseline period or the number of feedback reports in the previous statistical time window is zero, the calculation is completed according to the preset smoothing rule or boundary processing rule.

7. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The determination of the quality score for the feedback report includes: When the evaluation object is a single feedback report, the quality score of the feedback report is determined based on the content quality score of the single feedback report; When the evaluation object is a set of feedback reports within a statistical time window, the content quality scores of each feedback report in the set are aggregated, and the aggregated content quality scores are fused with the behavior quality scores according to preset weights to obtain the feedback report quality score of the set.

8. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The determination of the processing timeliness score includes: The initial response duration is determined based on the time difference between the report submission time and the initial response time, and the processing duration is determined based on the time difference between the report submission time and the processing completion time. For the processing result of a single airport, the initial response time and processing time are compared with the corresponding preset maximum time limit. The initial response timeliness index and processing cycle timeliness index are determined based on the remaining proportion of each actual time within the corresponding preset maximum time limit. For the processing results of multiple airports within the statistical time window, the percentage of processing results whose first response time meets the preset response time limit and the percentage of processing results whose processing time meets the preset processing time limit are calculated separately. The timeliness of the first response and the timeliness of the processing cycle, or the two proportions, are combined according to a preset weight to obtain a processing timeliness score.

9. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The method of identifying duplicate feedback using question identifiers or text semantic vectors includes: Extract at least two items from the feedback text: airport identifier, operating area, facility object, problem phenomenon, triggering condition, and affected object, to generate structured problem features; The feedback text or the text sequence composed of the structured question features is encoded using a local text encoding model to obtain a question feature vector; From the subsequent feedback reports after the target feedback report is processed, candidate feedback reports that meet the preset time sequence conditions and airport association conditions are selected; Calculate the semantic similarity between the problem feature vectors of the target feedback report and the candidate feedback reports; Based on the matching results of the structured question features and the semantic similarity, duplicate feedback belonging to the same question is identified; When a feedback report has a pre-set problem identifier, duplicate feedback will be identified first based on the problem identifier.

10. The two-way quality assessment method for pilot feedback reports and airport processing results according to claim 1, characterized in that, The determination of the problem-solving effectiveness score and the airport processing result quality score includes: Normalize the pilot satisfaction score corresponding to the processing results of a single airport, or determine the pilot satisfaction index based on the percentage of satisfaction samples that meet the preset satisfaction conditions within the statistical time window. For a single airport's processing results, the duplicate feedback index is determined based on the number of times the same problem reappears between the completion of the processing result and the evaluation time point; for multiple airports' processing results within the statistical time window, the duplicate feedback index is determined based on the proportion of feedback reports that the same problem reappears. The repeated feedback index is processed in reverse and fused with the pilot satisfaction index according to a preset weight to obtain a problem-solving effectiveness score. The problem-solving effectiveness score and the processing timeliness score are combined according to preset weights to obtain the airport processing result quality score. The feedback report quality score and the airport processing result quality score are compared with the corresponding preset scoring thresholds, and information supplementation labels, low quality labels or airport processing abnormal tasks are generated based on the comparison results.

Images