An airport operation rule checking type credible retrieval method based on a mixed knowledge base

Abstract

This invention discloses a reliable retrieval method for airport operation rules verification based on a hybrid knowledge foundation. The method includes: converting an airport operation query request into a structured query object; performing vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching based on the same structured query object; fusing and reordering the three types of search results to generate structured search results and candidate answers; comparing the entities, airport operation parameters, and candidate conclusions in the candidate answers with the currently valid structured rule objects; outputting reliable retrieval results and generating audit records when verification passes; performing backtracking retrieval and regenerating or revising candidate answers when verification fails, and transferring the method to manual review after a preset number of attempts if verification still fails. This invention can jointly obtain textual evidence, relational evidence, and rule evidence, improving the reliability and auditability of the search results.

Description

Technical Field

[0001] This invention belongs to the field of civil aviation information technology, specifically relating to a reliable retrieval method for airport operation rule verification based on a hybrid knowledge foundation. Background Technology

[0002] Currently, airport operation knowledge retrieval is typically achieved through keyword retrieval, full-text retrieval, single vector retrieval, or retrieval enhancement to generate questions and answers. Operations personnel can search for regulations, operation manuals, support procedures, historical event reports, air-to-ground communication records, and operation logs by keywords, document names, clause numbers, or business tags, and the system returns relevant documents, paragraphs, or clauses.

[0003] Some systems segment documents into text fragments and write them into a vector database, then retrieve similar fragments based on vector similarity when the user inputs a natural language question; other systems input the retrieved fragments into a large language model, which then generates natural language answers, regulatory interpretations, or handling suggestions.

[0004] The above methods can improve question-answering efficiency, but the retrieval process mainly relies on keywords or semantic similarity, lacking joint judgment on airport operation entity relationships, currently valid structured rule objects, version status, conflict status, and source evidence. Although existing publicly available technologies involve knowledge graph-enhanced question answering, RAG, multi-path hybrid retrieval, and verifiable retrieval enhancement generation, they do not include item-by-item verification of airport operation rule objects as a necessary step before outputting candidate answers.

[0005] The existing technology has at least the following technical disadvantages:

[0006] 1. Keyword retrieval or single vector retrieval cannot simultaneously cover entity relationships, airport operation parameters, and rule conditions in airport operation queries, resulting in insufficient judgment of the association between gates, aircraft, permits, areas, and regulations.

[0007] 2. Large language models or general question-answering systems may directly output candidate answers generated from similar texts, lacking item-by-item verification of the currently valid structured rule objects, which poses a risk of non-compliance or illusion.

[0008] 3. Airport operation terminology has business ambiguities, such as release, takeoff permission, taxiing permission, docking at the bridge, and parking space occupancy. Simply relying on literal meaning or semantic similarity can easily lead to inapplicable information.

[0009] 4. Search results often lack source clauses, evidence fragments, knowledge paths, rule hit results, version status, and review status, making them difficult to interpret, verify, and audit.

[0010] 5. When candidate answers conflict with the hit rules, there are version conflicts among the hit rules, or the confidence of the results is insufficient, the existing system lacks mechanisms for backtracking retrieval, answer revision, manual review, and abnormal log recording. Summary of the Invention

[0011] To address the shortcomings of the existing technologies, the present invention aims to provide a reliable retrieval method based on a hybrid knowledge foundation for verifying airport operation rules. This method solves the problems in existing keyword retrieval, vector retrieval, knowledge graph-enhanced question answering, and retrieval enhancement generation schemes, such as the lack of item-by-item verification of airport operation rule objects in candidate answers, the possibility of directly outputting conflicting results, the possibility of accepting expired clauses, and incomplete audit links.

[0012] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0013] The present invention discloses a reliable retrieval method for airport operation rule verification based on a hybrid knowledge foundation, comprising the following steps:

[0014] 1) Receive airport operation query requests and convert the query requests into structured query objects;

[0015] 2) Perform vector semantic retrieval based on structured query objects;

[0016] 3) Perform knowledge graph path retrieval based on structured query objects;

[0017] 4) Perform structured rule object matching based on structured query objects;

[0018] 5) Perform fusion and reordering on the vector semantic retrieval results, knowledge graph path retrieval results, and rule object matching results to obtain a candidate evidence set;

[0019] 6) Generate structured search results based on the merged and reordered candidate evidence set;

[0020] 7) Generate candidate answers based on structured search results;

[0021] 8) Perform rule validation and conflict identification on candidate answers. If the rule validation is successful, output reliable search results and generate audit records. Otherwise, perform backtracking search, answer revision or manual review for candidate answers that fail the rule validation.

[0022] Among them, vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching all take the same structured query object as input, and the three are executed separately without any order restriction; after the three types of retrieval results are formed, they are merged and reordered.

[0023] Furthermore, the process of converting the query request into a structured query object specifically includes:

[0024] Receive query requests from natural language queries, external business systems, or interface calls, wherein the query request includes at least one of the following: query text, request source, and business scenario;

[0025] The system performs intent recognition, entity recognition, airport operation parameter extraction, and query type determination on query requests, and outputs recognition results. The recognition results include query intent, query entity, airport operation parameters, query type, and a success, pending confirmation, or failure flag for each recognition result.

[0026] The identification results, query text, query source, and rule verification requirements are encapsulated into a structured query object.

[0027] Furthermore, the process of performing intent recognition, entity recognition, airport operation parameter extraction, and query type determination on the query request specifically includes:

[0028] The query request is segmented, synonyms are normalized, and matched with a domain dictionary to identify the query intent. The query intent includes: rule query, applicable condition query, restriction condition query, operational status query, and handling suggestion query. Specifically, the query text is first segmented, and synonyms are normalized according to the airport operation domain dictionary. Then, the normalized terms are matched with rule trigger words, business scenario words, and query sentence structure. Finally, the query text is mapped to the query intent category.

[0029] The system identifies query entities based on an airport operation entity dictionary, a knowledge graph node alias table, and a rule object field table. These query entities include: airports, runways, taxiways, parking positions, flights, aircraft, operational phases, job types, and rule clause objects. Specifically, it performs entity code exact matching, entity standard name matching, and entity alias matching sequentially according to the airport operation entity dictionary, knowledge graph node alias table, and rule object field table. For each matched entity, it records the entity name, entity type, and standard code. If multiple candidate entities exist, it performs disambiguation based on the query intent and airport operation parameters. If disambiguation still fails, it retains the candidate entity list and marks it as pending confirmation.

[0030] Airport operation parameters are extracted, including: time, location, aircraft type, wingspan, flight number, direction of flight, operational status, permit type, weather conditions, and traffic restrictions. Specifically, this is achieved by: locating parameter fields based on an airport operation parameter template and field mapping table; extracting parameter values ​​using keyword triggering or regular expression rules; normalizing units for numerical parameters with units, while simultaneously retaining the original parameter value, normalized parameter value, and unit; and outputting the parameter field, parameter value, unit, and parameter source location.

[0031] The query type is determined based on the query intent, the query entity, and airport operation parameters. These query types include: text evidence retrieval, graph path retrieval, rule object matching, and rule verification. Specifically, the query is judged based on the query intent, the query entity, airport operation parameters, and trigger words indicating applicable scope, thresholds, permissions, restrictions, or prohibitions in the query text. A query primarily requiring the return of text evidence is marked as a text evidence retrieval; a query requiring the determination of entity relationships is marked as a graph path retrieval; a query requiring the location of applicable rules is marked as a rule object matching; and a query requiring the formation of permission, restriction, applicability, or compliance conclusions and needing to compare each rule object is marked as a rule verification. When multiple conditions are met simultaneously, the query requiring a rule-based conclusion is prioritized for marking as a rule verification. Rule verification requirements are only marked as "yes" in rule verification cases or when a rule-based conclusion is required; other types are marked as "no."

[0032] The query text, query intent, query entity, airport operation parameters, query type, query source, and rule validation requirements are encapsulated into a structured query object. During encapsulation, the query text, query intent, query entity, airport operation parameters, query type, query source, and rule validation requirements are written in a fixed field order. When a field is missing, a missing marker is retained, and an empty field is not used to identify the failure status.

[0033] Furthermore, the structured query object includes: query text, query intent, query entity, airport operation parameters, query type, query source, and rule validation requirements.

[0034] Furthermore, the step of performing vector semantic retrieval based on structured query objects specifically includes:

[0035] The query text and airport operation parameters are converted into query vectors. Specifically, this is achieved by: reading the query text and airport operation parameters from a structured query object; combining the query text, parameter names, and parameter values ​​into vectorized input text according to a preset field order; encoding the input text using the same vectorization processing method as for text block vectors to obtain the query vector; performing normalization processing on the query vector when the vector database uses cosine similarity; and submitting the query vector to the vector index of the vector database for similarity retrieval.

[0036] Based on the similarity between the query vector and semantically similar text block vectors, semantically similar text blocks are retrieved from the vector database of the hybrid knowledge foundation, and the corresponding source document, source document page number, chapter title, version status, and review status are read. Specifically, the similarity between the query vector and the text block vector is calculated according to the cosine similarity, inner product similarity, or normalized distance pre-configured in the vector database; text blocks within a preset candidate range are obtained by sorting by similarity; text blocks that do not meet the preset similarity conditions are filtered; the source document, source document page number, chapter title, clause number, version status, effective status, and review status corresponding to each retained text block are read based on the text block identifier; when no text block meets the preset conditions, an empty vector semantic retrieval result set is output and marked as low confidence.

[0037] A semantic retrieval result set is generated based on vector similarity scores, business category matching degree, source authority, version validity, and review status. Specifically, for each candidate text block, vector similarity, business category matching degree, source authority, version validity, and review status scores are recorded. Among them, vector similarity and other continuous scores are transformed to the 0 to 1 range according to a preset normalization method, and version validity and review status are transformed to the 0 to 1 range according to a preset status scoring rule. A semantic retrieval ranking score is generated according to preset weights, and each scoring item and its source are retained. The semantic retrieval result set is formed in descending order of semantic retrieval ranking scores.

[0038] The hybrid knowledge base includes a vector database, a knowledge graph, and a structured rule base. The vector database stores airport operation rule text blocks, regulation clause text blocks, operation manual text blocks, historical Q&A text blocks, and handling case text blocks, along with their vector representations. It also records the source document, page number, chapter title, clause number, version status, effective status, and review status for each text block. The knowledge graph stores entity nodes and relational edges between airports, runways, taxiways, parking positions, aircraft, flights, operation phases, work types, regulation clauses, and rule objects. The structured rule base stores rule objects, applicable entity types, scope of application, triggering conditions, constraints, threshold parameters, exceptions, effective versions, effective status, and rule priorities.

[0039] The hybrid knowledge base is pre-established before query execution, specifically including: segmenting airport operation rules, regulations, operation manuals, historical Q&As, and case studies into text blocks according to clause number, chapter title, and paragraph boundaries; associating each text block with a knowledge entry ID, source document, source type or source document level, source document page number, chapter title, clause number, version status, effective status, review status, and business tag; vectorizing the text blocks and writing the text block vectors and their identifiers into the vector index of the vector database; generating knowledge based on the name, alias, code, and entity type of the airport operation entity. The system identifies entity nodes in the graph, generates relationship edges based on existing entity relationships, and associates source clauses, version status, and review status with nodes and relationship edges. It encapsulates structured rule objects according to rule ID, applicable entity type, scope of application, triggering conditions, constraints, threshold parameters, comparison operators, exceptions, effective version, effective status, review status, and rule priority. It establishes associations between text blocks, graph nodes, relationship edges, and rule objects using knowledge entry ID, source clause, rule ID, or entity code, enabling graph path evidence and rule object evidence to be traced back to the corresponding source document level or source type and business tag.

[0040] The vector similarity score is derived from the similarity calculation result between the query vector and the semantically similar text block vector; the semantically similar text block vector is the vector representation of airport operation rule text blocks, regulation clause text blocks, operation manual text blocks, historical question and answer text blocks or handling case text blocks pre-stored in the vector database; the similarity calculation is obtained by converting cosine similarity, inner product similarity or normalized distance.

[0041] The source document, source document page number, chapter title, version status, and review status are all derived from the associated metadata of the retrieved semantically similar text blocks.

[0042] The text block vector is a vector representation formed by pre-vectorizing airport operation rule text blocks, regulation clause text blocks, operation manual text blocks, historical Q&A text blocks, or case handling text blocks.

[0043] Furthermore, the knowledge graph path retrieval based on structured query objects specifically includes:

[0044] Based on the name, alias, code, and entity type of the queried entity, the corresponding entity node is located in the knowledge graph of the hybrid knowledge foundation. Specifically, the matching is performed first by the entity code, then by the standard name and alias, and finally by the entity type. The matched entity node and its source clause, version status, and review status are output. When there are multiple candidate nodes, disambiguation is performed by combining the query intent and airport operation parameters.

[0045] Starting from the entity node, path expansion is performed according to the relation types defined in the airport operation knowledge ontology. Specifically, this is implemented as follows: a set of relation types allowed to participate in the expansion is loaded according to the query intent; a breadth-first traversal is performed starting from the entity node, with the maximum path depth and the maximum number of expansions per node determined by the retrieval configuration; during the traversal, a set of visited nodes is maintained, and duplicate nodes and paths forming loops are pruned; nodes and relation edges with invalid version status or abnormal review status are removed, while retaining the status and source of the removed objects; paths are sorted by path score and a preset number of paths are retained, and the number of paths retained and the reasons for not retaining them are output.

[0046] The path expansion range is limited according to the query intent, and a graph path retrieval result containing entity nodes, relation edges, path length, source terms, and version status is generated; the graph path retrieval result includes a list of entity nodes, a list of relation edges, path length, source terms, version status, and review status; when no reachable path is found, an empty path set is output, and the reason for the miss is retained;

[0047] The graph path score is calculated based on path length, relationship type matching degree, node source authority, version validity, and review status. Specifically, the number of relationship edges in the path is used as the path length, and the path length is converted into a path length score by 1 / (1+path length). The relationship type matching degree, node source authority, version validity, and review status are converted into scoring items from 0 to 1 according to a preset mapping table, and then weighted and summed according to preset weights to obtain the graph path score. The shorter the path, the more matching the relationship type, the more authoritative the source, and the more valid the version and review status, the higher the graph path score.

[0048] Specifically, entity nodes are matched in the knowledge graph based on the name, alias, code, and entity type of the queried entity, and when multiple candidate entity nodes exist, the corresponding entity node is determined by combining the query intent and airport operation parameters.

[0049] The generated graph path retrieval results also include the review status; the review status comes from the review metadata of entity nodes, relation edges and their associated source clauses.

[0050] The path length in the graph path score comes from the number of entity nodes and relation edges in the knowledge graph path retrieval results; the relation type matching degree comes from the correspondence between the query intent and the path relation type; and the node source authority, version validity, and review status come from the metadata of entity nodes, relation edges, and their associated terms.

[0051] Furthermore, the step of performing structured rule object matching based on structured query objects specifically includes:

[0052] Based on the query entity, airport operation parameters, and query intent, rule objects are matched in the structured rule base of the hybrid knowledge foundation to generate rule object matching results. The rule object matching results include: the matched rule objects, the currently valid structured rule objects obtained by filtering by effective version, effective status, and review status, the rule hit result, version status, review status, and rule hit score. Specifically, candidate rule objects are filtered according to applicable entity type, applicable scope, and triggering conditions; the effective version, effective status, and review status of the candidate rule objects are read; currently valid and reviewed rule objects are marked as currently valid structured rule objects; status marks are retained for historically valid, expired, or pending rule objects for status description and to reduce evidence weight, but they cannot be used as the sole basis for credible results.

[0053] The rule hit score is calculated based on the degree of matching between the rule object and the query entity, airport operation parameters, query intent, scope of application, triggering conditions, constraints, and rule priority. Specifically, the degree of matching between the rule object and the query entity, airport operation parameters, query intent, scope of application, triggering conditions, constraints, and rule priority is converted into a score item from 0 to 1 based on the field matching results. When the triggering conditions or constraints are clearly not met, the corresponding matching item is recorded as 0. The rule hit score is obtained by weighted summation according to preset weights, and each matching field, corresponding rule field, and reason for non-match are retained.

[0054] Increase the rule hit score when the current valid structured rule object is fully hit; decrease the rule hit score when no rule object is hit, only a historical valid rule object is hit, or a rule object pending review is hit.

[0055] When the query contains computable airport operation parameters, the computable airport operation parameters are pre-matched with the threshold parameters, comparison operators, and constraints in the rule object. When the rule object is in a historically valid, expired, or pending review state, the version status and review status are marked in the rule object matching result. Specifically, the normalized airport operation parameters are read from the structured query object; pre-matching is performed according to the field correspondence, comparison operators, threshold parameters, and constraints in the rule object; numerical comparison is only performed on parameters that have been normalized and whose fields correspond; the pre-matching status is output as consistent, inconsistent, or cannot be compared, and the parameter fields, actual parameter values, rule fields, and rule thresholds involved in the comparison are retained; the version status and review status of the rule object are synchronously written into the rule object matching result; the pre-matching result is used to filter rule objects and form candidate evidence, and does not replace the final rule verification performed on the candidate answer in step 8).

[0056] The query containing computable airport operation parameters refers to the structured query object containing computable airport operation parameters. The computable airport operation parameters are fields among aircraft type, wingspan, gate number, operation phase, permit type, time, location, weather conditions, or traffic restriction conditions that can be compared with the threshold parameters of the rule object. When the query lacks the parameter value, parameter unit, field correspondence, or corresponding threshold field required by the rule, the rule object matching result is marked as incomparable and is not considered as a pre-match match. The parameter value, parameter unit, field correspondence, or threshold field to be supplemented is recorded. This state proceeds with the candidate evidence to steps 5) to 8). In step 8), the corresponding candidate answer should not be marked as rule passed, but should proceed to backtracking or manual review.

[0057] Furthermore, the step of obtaining the candidate evidence set specifically includes:

[0058] Read the vector similarity score from the vector semantic retrieval results, the graph path score from the graph path retrieval results, and the rule hit score from the rule object matching results;

[0059] The fusion score is calculated by combining source authority score, version validity score, review status score, business category matching score, and query intent matching score. Specifically, each vector semantic retrieval result is converted into a vector evidence item, each graph path retrieval result is converted into a graph path evidence item, and each rule object matching result is converted into a rule object evidence item. The vector similarity score, graph path score, rule hit score, source authority score, version validity score, review status score, business category matching score, and query intent matching score of each evidence item are read. For vector evidence items, the graph path score and rule hit score are set to 0. For graph path evidence items, the vector similarity score and rule hit score are set to 0. For rule object evidence items, the vector similarity score and graph path score are set to 0. Normalization is performed only for consecutive score items with inconsistent dimensions or value ranges; version validity and review status are processed according to preset status scoring rules. When the maximum value of a score item requiring normalization in the same batch of candidate results equals the minimum value, the normalization value of that score item is set to 0.5. The fusion score is calculated using the following formula:

[0060] S = w1×Sv + w2×Sg + w3×Sr + w4×Sa + w5×S_ver + w6×Sh + w7×Sb +w8×Si – P;

[0061] Where Sv is the vector similarity score, Sg is the graph path score, Sr is the rule hit score, Sa is the source authority score, S_ver is the version validity score, Sh is the review status score, Sb is the business category matching score, Si is the query intent matching score, w1 to w8 are non-negative preset weights that sum to 1, and P is the penalty term; the fusion score is restricted to the range of 0 to 1 before being used for sorting.

[0062] The semantic retrieval results, graph path retrieval results, and rule object matching results are sorted according to the fusion score to obtain the sorted vector evidence, graph path evidence, and rule object evidence. The weight of results lacking rule support, lacking graph path support, having an expired source version, or having an abnormal review status is reduced. Specifically, for candidate evidence items lacking rule support, lacking graph path support, having an expired source version, or having an abnormal review status, corresponding penalty items are added, and the reason for the penalty is recorded. Among them, penalty items are set for the corresponding evidence missing only when the query type or rule verification requirements require rule support or graph path support. Candidate evidence items are sorted in descending order of fusion score. When the fusion scores are the same, they are sorted in the following order: rule hit score, version validity score, source authority score, and path length. Evidence items without path length are marked as inapplicable. A preset number of candidate evidence items are retained after sorting and marked as vector evidence, graph path evidence, or rule object evidence, respectively.

[0063] The scoring criteria are as follows: Source Authority Score (derived from source document level), Version Validity Score (derived from version status of source clauses or rule objects), Review Status Score (derived from review status of source clauses, text blocks, entity nodes, relation edges, or rule objects), Business Category Matching Score (derived from matching results between business tags of knowledge entries and query business scenarios), and Query Intent Matching Score (derived from matching results between knowledge entries, graph paths, or rule objects and query intent). Each scoring item is weighted and summed according to preset weights to obtain the fusion score. The preset weights, status scoring rules, and penalty items are determined by the retrieval configuration and recorded along with the fusion result. The Source Authority Score is determined based on the source document level or source type; the Business Category Matching Score is determined based on matching results between business tags and query business scenarios; and the Query Intent Matching Score is determined based on matching results between knowledge entries, graph paths, or rule objects and query intent. Each scoring item retains its source field; the scoring basis should not be lost simply by saving the final fusion score.

[0064] Among them, vector evidence comes from semantically similar text blocks and their associated metadata obtained by vector semantic retrieval; graph path evidence comes from entity nodes, relation edges, source terms and path scores generated in knowledge graph path retrieval; and rule object evidence comes from structured rule objects, rule hit results, version status and review status obtained by structured rule object matching.

[0065] The candidate evidence set consists of vector evidence items, graph path evidence items, and rule object evidence items that have been fused and reordered. Each evidence item retains its evidence type, fusion score, and source field, and is not directly judged as passing the rule because a single evidence item lacks other types of evidence.

[0066] Furthermore, the step of generating structured search results specifically includes:

[0067] The sorted vector evidence, graph path evidence, and rule object evidence are organized into structured search results. During organization, the knowledge entry ID, evidence type, evidence fragment, source clause, source document, source document page number, knowledge path, scoring field, version status, review status, and relevant airport operation parameters of each evidence item are written into the same structured record. Fields without corresponding content are marked as not applicable. Then, based on the source clause, rule ID, entity code, business category label, or version identifier, the associated knowledge entry IDs are recorded between mutually supporting vector evidence items, graph path evidence items, and rule object evidence items.

[0068] The structured search results include: knowledge entry ID, content summary, source terms, source document, source document page number, evidence fragment, knowledge path, vector similarity score, graph path score, rule hit result, version validity, review status, and relevant airport operation parameters;

[0069] When the query type does not require a natural language response, the structured search results are directly output to the external business system. When the query type requires a natural language response, the structured search results are used as the basis for generating candidate answers. When the rule verification requirement is marked as yes or the query conclusion needs to form a judgment on permission, restriction, applicability, or compliance, even if the query type does not require a natural language response, the rule verification should not be bypassed and the results should not be directly output as credible results. In this case, the structured search results are organized into candidate answers for rule verification according to the fixed field organization method in step 7) and marked as pending verification, and then the process proceeds to step 8) for item-by-item comparison. Only when the rule verification requirement is no, the query type is text evidence retrieval, and a natural language response is not required, are the structured search results with the source field directly output to the external business system, and the query request, structured query object, search results, and output status are recorded.

[0070] Furthermore, the generation of candidate answers based on structured search results specifically includes:

[0071] The source clauses, evidence fragments, knowledge paths, and rule-hitting results are extracted from the structured search results and combined with the query request or query text in the structured query object to assemble a candidate answer generation context. Specifically, the candidate answer generation context is organized in a fixed order of query text, query intent, query entity, airport operation parameters, source clauses, evidence fragments, knowledge paths, currently valid structured rule objects, version status, and review status, and this context is used as the input to the candidate answer generation module.

[0072] Based on the candidate answer generation context, candidate answers are generated. The candidate answers include: candidate conclusions, answer text, a list of cited sources, relevant airport operation parameters, and evidence fragments corresponding to the candidate conclusions. The candidate answer generation module obtains the query text and structured search results from the candidate answer generation context, outputs the candidate conclusions, answer text, list of cited sources, relevant airport operation parameters, evidence fragments corresponding to the candidate conclusions, and candidate answer status, and saves the correspondence between the candidate answers and the IDs of the cited knowledge entries.

[0073] Candidate answers are marked as pending verification and are prohibited from being output as trusted results before passing rule verification. When a candidate answer is in the pending verification state, it must not be output as a trusted search result.

[0074] The candidate answers are generated by a large language model, a template generation module, or a rule-based text generation module based on the candidate answer generation context; the generated results are only used as candidate answers before passing rule verification and are not output as reliable search results.

[0075] Furthermore, the specific steps of performing rule verification and conflict identification on candidate answers include:

[0076] Extract entities, airport operation parameters, and candidate conclusions from structured search results and candidate answers;

[0077] The entities, airport operation parameters, and candidate conclusions are compared item by item with the currently valid structured rule objects. This item-by-item comparison includes entity consistency comparison, airport operation parameter threshold comparison, and candidate conclusion compliance comparison. Specifically, in entity consistency comparison, the entity names, aliases, and codes in the candidate answers are normalized and compared item by item with the applicable entities in the currently valid structured rule objects. In airport operation parameter threshold comparison, parameter fields and values ​​are read from the candidate answers and their corresponding structured query objects, and the normalized parameter values ​​are compared with the threshold parameters using a comparison operator. In candidate conclusion compliance comparison, the rule conclusion is determined based on the constraints, exceptions, and rule hit results of the rule object, and it is judged whether the candidate conclusion is consistent with the rule conclusion. Each comparison records the actual value, rule value, comparison result, and reason for failure.

[0078] When a candidate answer satisfies all currently valid structured rule objects, it is marked as a rule-passed answer. A candidate answer is only marked as a rule-passed answer when all currently valid structured rule objects have been compared and the entity, parameters, and candidate conclusions are all passed. If no currently valid structured rule object is matched, it is marked as a rule-missing answer; if there are entities that cannot be disambiguated, it is marked as pending confirmation; if there are necessary parameters that cannot be compared, it is marked as incomparable. None of the above states are marked as rule-passed. If the rule validation requirement is yes or a rule-based conclusion is required, the process proceeds to backtracking or manual review. If the rule validation requirement is no and only textual evidence is requested, only structured search results with the source field are output.

[0079] When a candidate answer violates any currently valid structured rule object, or when the rule object referenced by the candidate answer fails the current validity screening, the candidate answer is marked as having a rule conflict or pending review status, and a conflict description is generated. The conflict description includes: conflict rule ID, conflict entity, conflict parameter, conflict field, actual parameter value, rule threshold, source clause, source document page number, and conflict reason. A rule conflict description is generated only when there is an inconsistency between the candidate answer and the currently valid structured rule object. The conflict entity and conflict parameter are derived from the comparison fields between the candidate answer and the rule object; the actual parameter value is derived from the structured query object or the candidate answer; the rule threshold and comparison operator are derived from the matched rule object; and the source clause and source document page number are derived from the corresponding evidence record. No rule match, pending confirmation, or incomparable statuses are recorded with their corresponding statuses and reasons, and no rule conflict description is applied.

[0080] Furthermore, the step of outputting reliable search results and generating audit records specifically includes:

[0081] Based on the rule validation results, regenerated or revised candidate answers, and manual review status, reliable search results are generated. For queries requiring rule validation, reliable search results and audit records are generated only after the candidate answer passes the rule validation, or after manual review clearly establishes a conclusion and writes the review conclusion and review basis into the results. Reliable search results are not output when the query is in the status of pending validation, rule conflict, no rule hit, pending confirmation, incomparable, pending review, or manual review without conclusion. For queries that do not require rule validation and only request textual evidence, structured search results with source fields can be output.

[0082] Record the query request, structured query object, search results, graph path, rule matching results, candidate answers, rule validation results, backtracking search process, manual review status, and final output results;

[0083] Generate audit information, which includes the abnormal retrieval log number, processing time, call source, hit rule ID, source clause, version status, review status, whether backtracking retrieval was performed, whether the answer was revised, and whether it entered manual review; the audit information also records the rule verification status, conflict type, number of backtracking attempts, whether the candidate answer was regenerated or revised, and whether the manual review reached a conclusion.

[0084] The credible search results include: final conclusion, confidence level, source terms, source document, source document page number, evidence fragment, knowledge path, hit rules, conflict description, whether it has been revised, and abnormal search log number.

[0085] Furthermore, the step of performing backtracking retrieval, answer revision, or manual review for candidate answers that fail the rule validation specifically includes:

[0086] Using conflict rule ID, conflict entity, conflict parameters, source clause, version status, and related knowledge path as search conditions, vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching are re-executed. Specifically, when a rule conflict occurs, the conflict rule ID, conflict entity, conflict parameters, conflict fields, source clause, version status, and related knowledge path are encapsulated as backtracking search conditions. When no rule is matched, the query entity, airport operation parameters, query intent, and reason for the match are encapsulated as backtracking search conditions. When confirmation is pending, the candidate entity list, query intent, and airport operation parameters are encapsulated as backtracking search conditions. When comparison is not possible, the missing parameter fields, units, field correspondences, or threshold fields are encapsulated as backtracking search conditions. Based on the corresponding conditions, the search scope of text blocks, entity nodes, relationship edges, and rule objects is restricted, and three-way retrieval, candidate evidence association, and rule object matching are re-executed, and supplementary context is generated.

[0087] Based on the supplementary context obtained from backtracking retrieval, the candidate answers are regenerated or revised to obtain the regenerated or revised candidate answers; after each regeneration or revision, the backtracking retrieval count is incremented by 1, and the new candidate answers are resubmitted to the rule validation; the backtracking ends when the rule validation is passed, and continues to execute the next backtracking if it fails, until the preset limit is reached;

[0088] Perform rule validation on the regenerated or revised candidate answers;

[0089] When the regenerated or revised candidate answer passes the rule validation, it is output as a reliable search result.

[0090] If the candidate answer after being regenerated or revised a preset number of times still fails the rule validation, or if there are version conflicts, source conflicts, or threshold conflicts among the currently valid structured rule objects that cannot be automatically resolved, the query request will be transferred to manual review. After the manual review is completed, the review conclusion, review basis, and review status will be recorded. If the manual review does not reach a clear conclusion, the status will remain pending review and no reliable search results will be output.

[0091] The relevant knowledge path is derived from the knowledge path field in the structured search results and is used to limit the scope of entity nodes, relation edges, and regulatory clauses during backtracking retrieval. During backtracking retrieval, the knowledge path associated with the query entity, candidate entity, conflicting entity, rule object, or source clause in the current backtracking conditions is used, and the retrieval scope is limited according to the entity nodes, relation edges, and regulatory clauses in the path.

[0092] The beneficial effects of this invention are:

[0093] 1. Improve the retrieval completeness of complex queries in airport operations: This invention can simultaneously obtain textual evidence, relational evidence, and rule evidence through structured query objects, vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching.

[0094] 2. Improve the credibility of candidate answers: This invention limits the answers generated by the large language model or other generation modules to candidate answers, and verifies the entities, airport operation parameters and candidate conclusions in the candidate answers item by item through the currently valid structured rule objects. Candidate answers that fail the verification shall not be output directly.

[0095] 3. Improve source credibility, relationship credibility, rule credibility, and status credibility: In this invention, the final result is bound to the source clause, source document, source document page number, and evidence fragment; when entity relationships are involved, a knowledge graph path is provided; when restrictions, thresholds, permissions, prohibitions, or conditional judgments are involved, a hit rule is provided; and version validity, rule conflict status, and manual review status are also considered.

[0096] 4. Improve the reliability of judgment in scenarios with ambiguous terms: This invention can distinguish between terms that are similar in wording but have different business meanings, such as release, takeoff permission, taxiing permission, docking operation, and stand occupancy, by query intent recognition, entity recognition, graph path retrieval and rule object matching.

[0097] 5. Improve the closed-loop processing capability of conflict results: In this invention, when a candidate answer conflicts with the hit rule, or when there are version conflicts, source conflicts, or threshold conflicts among the hit rules, backtracking retrieval, answer revision, or manual review are triggered, and abnormal retrieval logs are recorded.

[0098] 6. Improve the interpretability and auditability of external business system call results: The credible search results in this invention include the final conclusion, confidence level, source clause, evidence fragment, knowledge path, hit rules, conflict description and abnormal search log number, which facilitates review by operators, knowledge administrators and external systems. Attached Figure Description

[0099] Figure 1 This is a schematic diagram of the method of the present invention. Detailed Implementation

[0100] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0101] Reference Figure 1 As shown, the present invention provides a reliable retrieval method for airport operation rule verification based on a hybrid knowledge foundation, comprising the following steps:

[0102] 1) Receive airport operation query requests and convert the query requests into structured query objects, specifically including:

[0103] Receive query requests from natural language queries, external business systems, or interface calls, wherein the query request includes at least one of the following: query text, request source, and business scenario;

[0104] The system performs intent recognition, entity recognition, airport operation parameter extraction, and query type determination on the query request, and outputs the recognition results. The recognition results include at least the query intent, query entity, airport operation parameters, query type, and a success, pending confirmation, or failure flag for each recognition result.

[0105] The identification results, query text, query source, and rule verification requirements are encapsulated into a structured query object.

[0106] Specifically, the process of performing intent recognition, entity recognition, airport operation parameter extraction, and query type determination on the query request includes:

[0107] The query request is segmented, synonyms are normalized, and matched with a domain dictionary to identify the query intent. The query intent includes: rule query, applicable condition query, restriction condition query, operational status query, and handling suggestion query. Specifically, the query text is first segmented, and synonyms are normalized according to the airport operation domain dictionary. Then, the normalized terms are matched with rule trigger words, business scenario words, and query sentence patterns. Finally, the query text is mapped to the query intent category.

[0108] The system identifies query entities based on an airport operation entity dictionary, a knowledge graph node alias table, and a rule object field table. These query entities include: airports, runways, taxiways, parking positions, flights, aircraft, operational phases, job types, and rule clause objects. Specifically, it performs entity code exact matching, entity standard name matching, and entity alias matching sequentially according to the airport operation entity dictionary, knowledge graph node alias table, and rule object field table. For each matched entity, it records the entity name, entity type, and standard code. If multiple candidate entities exist, it performs disambiguation based on the query intent and airport operation parameters. If disambiguation still fails, it retains the candidate entity list and marks it as pending confirmation.

[0109] Airport operation parameters are extracted, including: time, location, aircraft type, wingspan, flight number, direction of flight, operational status, permit type, weather conditions, and traffic restrictions. Specifically, this is achieved by: locating parameter fields based on an airport operation parameter template and field mapping table; extracting parameter values ​​using keyword triggering or regular expression rules; normalizing units for numerical parameters with units, while simultaneously retaining the original parameter value, normalized parameter value, and unit; and outputting the parameter field, parameter value, unit, and parameter source location.

[0110] The query type is determined based on the query intent, query entity, and airport operation parameters. These query types include: text evidence retrieval, graph path retrieval, rule object matching, and rule verification. Specifically, the query is judged based on the query intent, query entity, airport operation parameters, and trigger words indicating applicable scope, thresholds, permissions, restrictions, or prohibitions in the query text. A query primarily requiring the return of text evidence is marked as a text evidence retrieval; a query requiring the determination of entity relationships is marked as a graph path retrieval; a query requiring the location of applicable rules is marked as a rule object matching; and a query requiring the formation of permission, restriction, applicability, or compliance conclusions and needing to compare each rule object item by item is marked as a rule verification. When multiple conditions are met simultaneously, those requiring the formation of rule-based conclusions are prioritized for marking as rule verification. Rule verification requirements are only marked as "yes" in rule verification cases or when rule-based conclusions are required; other types are marked as "no."

[0111] The query text, query intent, query entity, airport operation parameters, query type, query source, and rule validation requirements are encapsulated into a structured query object. During encapsulation, the query text, query intent, query entity, airport operation parameters, query type, query source, and rule validation requirements are written in a fixed field order. When a field is missing, a missing marker is retained, and an empty field is not used to identify the failure status.

[0112] The structured query objects include: query text, query intent, query entity, airport operation parameters, query type, query source, and rule validation requirements.

[0113] Among them, vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching all take the same structured query object as input, and the three are executed separately without any order restriction; after the three types of retrieval results are formed, they are merged and reordered.

[0114] 2) Perform vector semantic retrieval based on structured query objects, specifically including:

[0115] The query text and airport operation parameters are converted into query vectors. Specifically, this is achieved by: reading the query text and airport operation parameters from a structured query object; combining the query text, parameter names, and parameter values ​​into vectorized input text according to a preset field order; encoding the input text using the same vectorization processing method as for text block vectors to obtain the query vector; performing normalization processing on the query vector when the vector database uses cosine similarity; and submitting the query vector to the vector index of the vector database for similarity retrieval.

[0116] Based on the similarity between the query vector and semantically similar text block vectors, semantically similar text blocks are retrieved from the vector database of the hybrid knowledge foundation, and the corresponding source document, source document page number, chapter title, version status, and review status are read. Specifically, the similarity between the query vector and the text block vector is calculated according to the cosine similarity, inner product similarity, or normalized distance pre-configured in the vector database; text blocks within a preset candidate range are obtained by sorting by similarity; text blocks that do not meet the preset similarity conditions are filtered; the source document, source document page number, chapter title, clause number, version status, effective status, and review status corresponding to each retained text block are read based on the text block identifier; when no text block meets the preset conditions, an empty vector semantic retrieval result set is output and marked as low confidence.

[0117] A semantic retrieval result set is generated based on vector similarity scores, business category matching degree, source authority, version validity, and review status. Specifically, for each candidate text block, vector similarity, business category matching degree, source authority, version validity, and review status scores are recorded. Vector similarity and other continuous scores are converted to the 0-1 range using a preset normalization method, while version validity and review status are converted to the 0-1 range according to preset status scoring rules. A semantic retrieval ranking score is generated according to preset weights, and each scoring item and its source are retained. The semantic retrieval result set is then formed in descending order of the semantic retrieval ranking score.

[0118] Specifically, the hybrid knowledge base includes: a vector database, a knowledge graph, and a structured rule base; the vector database stores airport operation rule text blocks, regulation clause text blocks, operation manual text blocks, historical Q&A text blocks, and handling case text blocks and their vector representations, and records the source document, source document page number, chapter title, clause number, version status, effective status, and review status corresponding to each text block; the knowledge graph stores entity nodes and relationship edges between airports, runways, taxiways, parking positions, aircraft, flights, operation phases, operation types, regulation clauses, and rule objects; the structured rule base stores rule objects, applicable entity types, scope of application, triggering conditions, constraints, threshold parameters, exceptions, effective versions, effective status, and rule priorities.

[0119] The hybrid knowledge base is pre-established before query execution, specifically including: segmenting airport operation rules, regulations, operation manuals, historical Q&As, and case studies into text blocks according to clause number, chapter title, and paragraph boundaries; associating each text block with a knowledge entry ID, source document, source type or source document level, source document page number, chapter title, clause number, version status, effective status, review status, and business tag; vectorizing the text blocks and writing the text block vectors and their identifiers into the vector index of the vector database; generating knowledge based on the name, alias, code, and entity type of the airport operation entity. The system identifies graph entity nodes, generates relationship edges based on existing entity relationships, and associates source clauses, version status, and review status with nodes and relationship edges. It encapsulates structured rule objects according to rule ID, applicable entity type, scope of application, triggering conditions, constraints, threshold parameters, comparison operators, exceptions, effective version, effective status, review status, and rule priority. It establishes associations between text blocks, graph nodes, relationship edges, and rule objects using knowledge entry IDs, source clauses, rule IDs, or entity codes, enabling graph path evidence and rule object evidence to be traced back to the corresponding source document level or source type and business tag.

[0120] The vector similarity score is derived from the similarity calculation result between the query vector and the semantically similar text block vector; the semantically similar text block vector is the vector representation of airport operation rule text blocks, regulation clause text blocks, operation manual text blocks, historical question and answer text blocks or handling case text blocks pre-stored in the vector database; the similarity calculation is obtained by converting cosine similarity, inner product similarity or normalized distance.

[0121] The source document, source document page number, chapter title, version status, and review status are all derived from the associated metadata of the retrieved semantically similar text blocks.

[0122] The text block vector is a vector representation formed by pre-vectorizing airport operation rule text blocks, regulation clause text blocks, operation manual text blocks, historical Q&A text blocks, or case handling text blocks.

[0123] 3) Perform knowledge graph path retrieval based on structured query objects, specifically including:

[0124] Based on the name, alias, code, and entity type of the queried entity, the corresponding entity node is located in the knowledge graph of the hybrid knowledge foundation. Specifically, the matching is performed first by the entity code, then by the standard name and alias, and finally by the entity type. The matched entity node and its source clause, version status, and review status are output. When there are multiple candidate nodes, disambiguation is performed by combining the query intent and airport operation parameters.

[0125] Starting from the entity node, path expansion is performed according to the relation types defined in the airport operation knowledge ontology. Specifically, this is implemented as follows: a set of relation types allowed to participate in the expansion is loaded according to the query intent; a breadth-first traversal is performed starting from the entity node, with the maximum path depth and the maximum number of expansions per node determined by the retrieval configuration; during the traversal, a set of visited nodes is maintained, and duplicate nodes and paths forming loops are pruned; nodes and relation edges with invalid version status or abnormal review status are removed, while retaining the status and source of the removed objects; paths are sorted by path score and a preset number of paths are retained, and the number of paths retained and the reasons for not retaining them are output.

[0126] The path expansion range is limited according to the query intent, and a graph path retrieval result containing entity nodes, relation edges, path length, source terms, and version status is generated; the graph path retrieval result includes at least a list of entity nodes, a list of relation edges, path length, source terms, version status, and review status; when no reachable path is found, an empty path set is output, and the reason for the miss is retained;

[0127] The graph path score is calculated based on path length, relationship type matching degree, node source authority, version validity, and review status. Specifically, the number of relationship edges in the path is used as the path length, which is then converted into a path length score using the formula 1 / (1+path length). Relationship type matching degree, node source authority, version validity, and review status are each converted into scoring items from 0 to 1 according to a preset mapping table, and then weighted and summed according to preset weights to obtain the graph path score. The shorter the path, the more matched the relationship type, the more authoritative the source, and the more valid the version and review status, the higher the graph path score.

[0128] Specifically, entity nodes are matched in the knowledge graph based on the name, alias, code, and entity type of the queried entity, and when multiple candidate entity nodes exist, the corresponding entity node is determined by combining the query intent and airport operation parameters.

[0129] The generated graph path retrieval results also include the review status; the review status comes from the review metadata of entity nodes, relation edges and their associated source clauses.

[0130] The path length in the graph path score comes from the number of entity nodes and relation edges in the knowledge graph path retrieval results; the relation type matching degree comes from the correspondence between the query intent and the path relation type; and the node source authority, version validity, and review status come from the metadata of entity nodes, relation edges, and their associated terms.

[0131] 4) Perform structured rule object matching based on structured query objects, specifically including:

[0132] Based on the query entity, airport operation parameters, and query intent, rule objects are matched in the structured rule base of the hybrid knowledge foundation to generate rule object matching results. The rule object matching results include: the matched rule objects, the currently valid structured rule objects obtained by filtering by effective version, effective status, and review status, the rule hit result, version status, review status, and rule hit score. Specifically, candidate rule objects are filtered according to applicable entity type, applicable scope, and triggering conditions; the effective version, effective status, and review status of the candidate rule objects are read; currently valid and reviewed rule objects are marked as currently valid structured rule objects; status marks are retained for historically valid, expired, or pending rule objects for status description and to reduce evidence weight, but they cannot be used as the sole basis for credible results.

[0133] The rule hit score is calculated based on the degree of matching between the rule object and the query entity, airport operation parameters, query intent, scope of application, triggering conditions, constraints, and rule priority. Specifically, the degree of matching between the rule object and the query entity, airport operation parameters, query intent, scope of application, triggering conditions, constraints, and rule priority is converted into a score item from 0 to 1 based on the field matching results. When the triggering conditions or constraints are clearly not met, the corresponding matching item is recorded as 0. The rule hit score is obtained by weighted summation according to preset weights, and each matching field, corresponding rule field, and reason for non-match are retained.

[0134] Increase the rule hit score when the current valid structured rule object is fully hit; decrease the rule hit score when no rule object is hit, only a historical valid rule object is hit, or a rule object pending review is hit.

[0135] When the query contains computable airport operation parameters, the computable airport operation parameters are pre-matched with the threshold parameters, comparison operators, and constraints in the rule object. When the rule object is in a historically valid, expired, or pending review state, the version status and review status are marked in the rule object matching result. Specifically, the normalized airport operation parameters are read from the structured query object; pre-matching is performed according to the field correspondence, comparison operators, threshold parameters, and constraints in the rule object; numerical comparison is only performed on parameters that have been normalized and whose fields correspond; the pre-matching status is output as consistent, inconsistent, or cannot be compared, and the parameter fields, actual parameter values, rule fields, and rule thresholds involved in the comparison are retained; the version status and review status of the rule object are synchronously written into the rule object matching result; the pre-matching result is used to filter rule objects and form candidate evidence, and does not replace the final rule verification performed on the candidate answer in step 8).

[0136] The query containing computable airport operation parameters refers to the structured query object containing computable airport operation parameters. The computable airport operation parameters are fields among aircraft type, wingspan, gate number, operation phase, permit type, time, location, weather conditions, or traffic restriction conditions that can be compared with the threshold parameters of the rule object. When the query lacks the parameter value, parameter unit, field correspondence, or corresponding threshold field required by the rule, the rule object matching result is marked as incomparable and is not considered as a pre-match match. The parameter value, parameter unit, field correspondence, or threshold field to be supplemented is recorded. This state proceeds with the candidate evidence to steps 5) to 8). In step 8), the corresponding candidate answer should not be marked as rule passed, but should proceed to backtracking or manual review.

[0137] 5) Perform a fusion and reordering of the vector semantic retrieval results, knowledge graph path retrieval results, and rule object matching results to obtain a candidate evidence set, specifically including:

[0138] Read the vector similarity score from the vector semantic retrieval results, the graph path score from the graph path retrieval results, and the rule hit score from the rule object matching results;

[0139] The fusion score is calculated by combining source authority score, version validity score, review status score, business category matching score, and query intent matching score. Specifically, each vector semantic retrieval result is converted into a vector evidence item, each graph path retrieval result is converted into a graph path evidence item, and each rule object matching result is converted into a rule object evidence item. The vector similarity score, graph path score, rule hit score, source authority score, version validity score, review status score, business category matching score, and query intent matching score of each evidence item are read. For vector evidence items, the graph path score and rule hit score are set to 0. For graph path evidence items, the vector similarity score and rule hit score are set to 0. For rule object evidence items, the vector similarity score and graph path score are set to 0. Normalization is performed only for consecutive score items with inconsistent dimensions or value ranges; version validity and review status are processed according to preset status scoring rules. When the maximum value of a score item requiring normalization in the same batch of candidate results equals the minimum value, the normalization value of that score item is set to 0.5. The fusion score is calculated using the following formula:

[0140] S = w1×Sv + w2×Sg + w3×Sr + w4×Sa + w5×S_ver + w6×Sh + w7×Sb +w8×Si – P;

[0141] Where Sv is the vector similarity score, Sg is the graph path score, Sr is the rule hit score, Sa is the source authority score, S_ver is the version validity score, Sh is the review status score, Sb is the business category matching score, Si is the query intent matching score, w1 to w8 are non-negative preset weights that sum to 1, and P is the penalty term; the fusion score is restricted to the range of 0 to 1 before being used for sorting.

[0142] The semantic retrieval results, graph path retrieval results, and rule object matching results are sorted according to the fusion score to obtain the sorted vector evidence, graph path evidence, and rule object evidence. The weight of results lacking rule support, lacking graph path support, invalid source version, or abnormal review status is reduced. Specifically, for candidate evidence items lacking rule support, lacking graph path support, invalid source version, or abnormal review status, corresponding penalty items are added, and the reason for the penalty is recorded. Among them, penalty items are set for the corresponding evidence missing only when the query type or rule verification requirement requires rule support or graph path support. Candidate evidence items are sorted in descending order of fusion score. When the fusion scores are the same, they are sorted in the following order: rule hit score, version validity score, source authority score, and path length. Evidence items without path length are marked as inapplicable. A preset number of candidate evidence items are retained after sorting and marked as vector evidence, graph path evidence, or rule object evidence, respectively.

[0143] The scoring criteria are as follows: Source Authority Score (derived from source document level), Version Validity Score (derived from version status of source clauses or rule objects), Review Status Score (derived from review status of source clauses, text blocks, entity nodes, relation edges, or rule objects), Business Category Matching Score (derived from matching results between business tags of knowledge entries and query business scenarios), and Query Intent Matching Score (derived from matching results between knowledge entries, graph paths, or rule objects and query intent). Each scoring item is weighted and summed according to preset weights to obtain the fusion score. The preset weights, status scoring rules, and penalty items are determined by the retrieval configuration and recorded along with the fusion result. The Source Authority Score is determined based on the source document level or source type; the Business Category Matching Score is determined based on matching results between business tags and query business scenarios; and the Query Intent Matching Score is determined based on matching results between knowledge entries, graph paths, or rule objects and query intent. Each scoring item retains its source field; the scoring basis should not be lost simply by saving the final fusion score.

[0144] Among them, vector evidence comes from semantically similar text blocks and their associated metadata obtained by vector semantic retrieval; graph path evidence comes from entity nodes, relation edges, source terms and path scores generated in knowledge graph path retrieval; and rule object evidence comes from structured rule objects, rule hit results, version status and review status obtained by structured rule object matching.

[0145] The candidate evidence set consists of vector evidence items, graph path evidence items, and rule object evidence items that have been fused and reordered. Each evidence item retains its evidence type, fusion score, and source field, and is not directly judged as passing the rule because a single evidence item lacks other types of evidence.

[0146] 6) Generate structured search results based on the fused and reordered candidate evidence set, specifically including:

[0147] The sorted vector evidence, graph path evidence, and rule object evidence are organized into structured search results. During organization, the knowledge entry ID, evidence type, evidence fragment, source clause, source document, source document page number, knowledge path, scoring field, version status, review status, and relevant airport operation parameters of each evidence item are written into the same structured record. Fields without corresponding content are marked as not applicable. Then, based on the source clause, rule ID, entity code, business category label, or version identifier, the associated knowledge entry IDs are recorded between mutually supporting vector evidence items, graph path evidence items, and rule object evidence items.

[0148] The structured search results include: knowledge entry ID, content summary, source terms, source document, source document page number, evidence fragment, knowledge path, vector similarity score, graph path score, rule hit result, version validity, review status, and relevant airport operation parameters;

[0149] When the query type does not require a natural language response, the structured search results are directly output to the external business system. When the query type requires a natural language response, the structured search results are used as the basis for generating candidate answers. When the rule verification requirement is marked as yes or the query conclusion needs to form a judgment on permission, restriction, applicability, or compliance, even if the query type does not require a natural language response, the rule verification should not be bypassed and the results should not be directly output as credible results. In this case, the structured search results are organized into candidate answers for rule verification according to the fixed field organization method in step 7) and marked as pending verification, and then the process proceeds to step 8) for item-by-item comparison. Only when the rule verification requirement is no, the query type is text evidence retrieval, and a natural language response is not required, are the structured search results with source fields directly output to the external business system, and the query request, structured query object, search results, and output status are recorded.

[0150] 7) Generate candidate answers based on structured search results, specifically including:

[0151] The source clauses, evidence fragments, knowledge paths, and rule-hitting results are extracted from the structured search results and combined with the query request or query text in the structured query object to assemble a candidate answer generation context. Specifically, the candidate answer generation context is organized in a fixed order of query text, query intent, query entity, airport operation parameters, source clauses, evidence fragments, knowledge paths, currently valid structured rule objects, version status, and review status, and this context is used as the input to the candidate answer generation module.

[0152] Based on the candidate answer generation context, candidate answers are generated. The candidate answers include: candidate conclusions, answer text, a list of cited sources, relevant airport operation parameters, and evidence fragments corresponding to the candidate conclusions. The candidate answer generation module obtains the query text and structured search results from the candidate answer generation context, outputs the candidate conclusions, answer text, list of cited sources, relevant airport operation parameters, evidence fragments corresponding to the candidate conclusions, and candidate answer status, and saves the correspondence between the candidate answers and the IDs of the cited knowledge entries.

[0153] Candidate answers are marked as pending verification and are prohibited from being output as trusted results before passing rule verification; when a candidate answer is pending verification, it must not be output as a trusted search result.

[0154] The candidate answers are generated by a large language model, a template generation module, or a rule-based text generation module based on the candidate answer generation context; the generated results are only used as candidate answers before passing rule verification and are not output as reliable search results.

[0155] 8) Perform rule validation and conflict identification on candidate answers. If the rule validation is successful, output reliable search results and generate audit records. Otherwise, perform backtracking search, answer revision or manual review for candidate answers that fail the rule validation.

[0156] Specifically, the rule verification and conflict identification of candidate answers includes:

[0157] Extract entities, airport operation parameters, and candidate conclusions from structured search results and candidate answers;

[0158] The entities, airport operation parameters, and candidate conclusions are compared item by item with the currently valid structured rule objects. This item-by-item comparison includes entity consistency comparison, airport operation parameter threshold comparison, and candidate conclusion compliance comparison. Specifically, in entity consistency comparison, the entity names, aliases, and codes in the candidate answers are normalized and compared item by item with the applicable entities in the currently valid structured rule objects. In airport operation parameter threshold comparison, parameter fields and values ​​are read from the candidate answers and their corresponding structured query objects, and the normalized parameter values ​​are compared with the threshold parameters using a comparison operator. In candidate conclusion compliance comparison, the rule conclusion is determined based on the constraints, exceptions, and rule hit results of the rule object, and it is judged whether the candidate conclusion is consistent with the rule conclusion. Each comparison records the actual value, rule value, comparison result, and reason for failure.

[0159] When a candidate answer satisfies all currently valid structured rule objects, the candidate answer is marked as rule passed; a candidate answer is marked as rule passed only when all currently valid structured rule objects are compared and the entity, parameters, and candidate conclusions are all passed; if no currently valid structured rule objects are matched, it is marked as no rule matched; if there are entities that cannot be disambiguated, it is marked as pending confirmation; if there are necessary parameters that cannot be compared, it is marked as incomparable; none of the above states are marked as rule passed; when the rule verification requirement is yes or a rule-based conclusion needs to be formed, it is transferred to backtracking or manual review; when the rule verification requirement is no and only textual evidence is requested, only the structured search results with the source field are output;

[0160] When a candidate answer violates any currently valid structured rule object, or when the rule object referenced by the candidate answer fails the current validity screening, the candidate answer is marked as having a rule conflict or pending review status, and a conflict description is generated. The conflict description includes: conflict rule ID, conflict entity, conflict parameter, conflict field, actual parameter value, rule threshold, source clause, source document page number, and conflict reason. A rule conflict description is generated only when the comparison result between the candidate answer and the currently valid structured rule object is inconsistent. The conflict entity and conflict parameter are derived from the comparison fields between the candidate answer and the rule object; the actual parameter value is derived from the structured query object or the candidate answer; the rule threshold and comparison operator are derived from the matched rule object; and the source clause and source document page number are derived from the corresponding evidence record. No rule match, pending confirmation, or incomparable statuses are recorded with their corresponding statuses and reasons, and no rule conflict description is applied.

[0161] Specifically, the step of outputting reliable search results and generating audit records includes:

[0162] Based on the rule validation results, regenerated or revised candidate answers, and manual review status, reliable search results are generated. For queries requiring rule validation, reliable search results and audit records are generated only after the candidate answer passes the rule validation, or after manual review clearly establishes a conclusion and writes the review conclusion and review basis into the results. Reliable search results are not output when the query is in the status of pending validation, rule conflict, no rule hit, pending confirmation, incomparable, pending review, or manual review without conclusion. For queries that do not require rule validation and only request textual evidence, structured search results with source fields can be output.

[0163] The credible search results include: final conclusion, confidence level, source terms, source document, source document page number, evidence fragment, knowledge path, hit rules, conflict explanation, whether it has been revised, and abnormal search log number;

[0164] Record the query request, structured query object, search results, graph path, rule matching results, candidate answers, rule validation results, backtracking search process, manual review status, and final output results;

[0165] Generate audit information, which includes the abnormal retrieval log number, processing time, call source, hit rule ID, source clause, version status, review status, whether backtracking retrieval was performed, whether the answer was revised, and whether it entered manual review; the audit information also records the rule verification status, conflict type, number of backtracking attempts, whether the candidate answer was regenerated or revised, and whether the manual review reached a conclusion.

[0166] Specifically, the step of performing backtracking retrieval, answer revision, or manual review for candidate answers that fail the rule validation includes:

[0167] Using conflict rule ID, conflict entity, conflict parameters, source clause, version status, and related knowledge path as search conditions, vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching are re-executed. Specifically, when a rule conflict occurs, the conflict rule ID, conflict entity, conflict parameters, conflict fields, source clause, version status, and related knowledge path are encapsulated as backtracking search conditions. When no rule is matched, the query entity, airport operation parameters, query intent, and reason for the match are encapsulated as backtracking search conditions. When confirmation is pending, the candidate entity list, query intent, and airport operation parameters are encapsulated as backtracking search conditions. When comparison is not possible, the missing parameter fields, units, field correspondences, or threshold fields are encapsulated as backtracking search conditions. Based on the corresponding conditions, the search scope of text blocks, entity nodes, relationship edges, and rule objects is restricted, and three-way retrieval, candidate evidence association, and rule object matching are re-executed, and supplementary context is generated.

[0168] The relevant knowledge path is derived from the knowledge path field in the structured search results and is used to limit the scope of entity nodes, relation edges, and regulatory clauses during backtracking retrieval. During backtracking retrieval, the knowledge path associated with the query entity, candidate entity, conflicting entity, rule object, or source clause in the current backtracking conditions is used, and the retrieval scope is limited according to the entity nodes, relation edges, and regulatory clauses in the path.

[0169] Based on the supplementary context obtained from backtracking retrieval, the candidate answers are regenerated or revised to obtain the regenerated or revised candidate answers; after each regeneration or revision, the backtracking retrieval count is incremented by 1, and the new candidate answers are resubmitted to the rule validation; the backtracking ends when the rule validation is passed, and continues to execute the next backtracking if it fails, until the preset limit is reached;

[0170] Perform rule validation on the regenerated or revised candidate answers;

[0171] When the regenerated or revised candidate answer passes the rule validation, it is output as a reliable search result.

[0172] If the candidate answer after being regenerated or revised a preset number of times still fails the rule validation, or if there are version conflicts, source conflicts, or threshold conflicts among the currently valid structured rule objects that cannot be automatically resolved, the query request will be transferred to manual review. After the manual review is completed, the review conclusion, review basis, and review status will be recorded. If the manual review does not reach a clear conclusion, the status will remain pending review and no reliable search results will be output.

[0173] The number of candidate vectors, similarity filtering criteria, maximum depth of the graph path, maximum expansion number per node, number of candidate evidence items retained after fusion, and upper limit of backtracking retries are all written into the retrieval configuration before query execution. The number of candidates, path depth, expansion number, and retention number are positive integers, the similarity filtering criteria are within the similarity value range, and the upper limit of backtracking retries is a non-negative integer. The same retrieval configuration is read during each step, and the configuration used is recorded in the audit information. The text vector dimension is determined by the text block vectorization processing method used; the query vector uses the same dimension, and no different dimensions are set separately.

[0174] Anomaly and boundary handling include: when vector retrieval, graph path retrieval, and rule object matching are all empty, an empty evidence set is output and marked with low confidence, without outputting a credible conclusion; when no currently valid structured rule object is hit, the "no rule hit" status is retained, and backtracking or manual review is only initiated when the rule verification requirement is "yes" or a rule-based conclusion needs to be formed; when multiple entities cannot be disambiguated, a candidate entity list is retained and marked as pending confirmation, without directly forming a credible conclusion; when necessary parameters, units, field correspondences, or threshold fields are missing, an incomparable status is output; when parameters equal to thresholds, the comparison operator in the rule object is used for judgment, and the boundary value is judged as satisfied by the comparison operator containing the equal sign; when the candidate answer is inconsistent with the currently valid structured rule object, a rule conflict description is generated, and when there are conflicts between sources, versions, or thresholds, the conflicting objects and sources are recorded respectively; when candidate answer generation fails or citation evidence is missing, manual review is initiated; when the backtracking count reaches the upper limit and still fails to pass the verification, automatic backtracking is stopped and manual review is initiated; when manual review does not form a clear conclusion, the pending review status is maintained and no credible search results are output.

[0175] Example 1: Implementation of trusted retrieval for the machine location adaptation problem;

[0176] Step 1): Receive a query request for "whether the specified aircraft type can perform bridge docking operations at the specified stand"; identify the aircraft type, stand, wingspan, and bridge docking operation terminology based on the airport operations dictionary to obtain the query intent as a stand compatibility judgment; extract the aircraft type, stand, wingspan parameters, bridge docking operation type, and query source from the query text or external business system fields, and encapsulate them into a structured query object.

[0177] Steps 2) through 4): Perform vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching respectively using the same structured query object. Vector semantic retrieval obtains textual evidence T1 containing the aircraft stand wingspan limit and its source metadata; knowledge graph path retrieval obtains path evidence G1 containing the aircraft stand, aircraft, wingspan parameters, and regulatory clause nodes and relational edges; structured rule object matching obtains currently valid rule evidence R1 that matches the aircraft stand, aircraft type, and docking operation type. R1 includes at least the applicable entity, threshold field, comparison operator, threshold parameter, source clause, effective status, and review status. If no currently valid R1 is obtained, it is marked as a no-match rule and does not proceed to the rule approval branch.

[0178] Step 5): Convert T1, G1, and R1 into vector evidence items, graph path evidence items, and rule object evidence items, respectively, and calculate the fusion score for each. As a scoring example used only to illustrate the calculation process, the values ​​of the three pieces of evidence are Sa=0.90, S_ver=1, Sh=1, Sb=0.95, and Si=0.95; the values ​​of T1 are Sv=0.82, Sg=0, and Sr=0; the values ​​of G1 are Sv=0, Sg=0.78, and Sr=0; and the values ​​of R1 are Sv=0, Sg=0, and Sr=0.95; the corresponding weights are 0.25, 0.15, 0.25, 0.10, 0.10, 0.05, 0.05, and 0.05, respectively. If the combined penalty for T1 lacking both map path support and rule support is 0.10, the penalty for G1 lacking rule support is 0.05, and the penalty for R1 lacking map path support is 0.05, then the fusion scores for the three pieces of evidence are 0.44, 0.402, and 0.5225, respectively, and the order is R1, T1, G1. The above penalties are only set because the aircraft position adaptation judgment in this embodiment requires both map path and rule support; the scores, weights, and penalty values ​​are only for demonstrating fusion calculations and do not represent the actual airport scoring configuration.

[0179] Steps 6) and 7): Extract source clauses, evidence fragments, knowledge paths, rule hit results, version status, review status, and airport operation parameters from the sorted and retained R1, T1, and G1. Record the association between the three pieces of evidence based on the source clause, entity code, and rule ID to form a structured search result. Then, combine the structured search result with the query text in a fixed field order to form a candidate answer generation context, generate candidate answers, and mark them as pending verification. To illustrate the comparison process, the wingspan value in the query can be denoted as Vq, the wingspan threshold in the rule object can be denoted as Vr, and the candidate answer can be recorded as "the specified aircraft type meets the docking conditions of the specified stand"; Vq and Vr are only variables and do not represent the actual rule values ​​of a specific airport.

[0180] Step 8): First, compare whether the aircraft type and location in the candidate answers are consistent with the applicable entity of R1. Then, compare Vq and Vr according to the comparison operator recorded in R1. Finally, determine the rule conclusion based on the constraints, exceptions, and rule hit results of R1, and determine whether the candidate conclusion is consistent with the rule conclusion. When the entity, parameters, and candidate conclusion all pass, mark the candidate answer as rule passed and output a reliable search result containing the source clause, evidence fragment, knowledge path, hit rule, and abnormal search log number. When Vq does not meet the comparison operator, generate a rule conflict explanation. When R1 fails the current validity or review status screening, mark the corresponding status. When R1 is not hit, Vq is missing, or the unit cannot be normalized, mark it as no rule hit or cannot be compared, and enter the backtracking search or manual review.

[0181] Step 8) Backtracking after failure: Re-execute the three-way search using the conflict rule ID, permission type, call time, knowledge item ID, and related knowledge path to generate supplementary context and regenerate the candidate answer "Existing evidence indicates that the flight has obtained taxiing permission, but there is no evidence to prove that it has obtained takeoff permission"; the new candidate answer is then sent to the entity, permission type, and candidate conclusion for comparison again. Upon success, the rule-verified permission status, evidence fragment, applicable source clause, confidence level, and abnormal search log number are output; if a clear conclusion cannot be reached even after the backtracking limit is reached, it is transferred to manual review. If no conclusion is reached during manual review, the system remains in a pending review state and no credible search results are output. This invention transforms natural language queries or external system requests into structured query objects and simultaneously performs vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching based on a hybrid knowledge foundation, generating structured search results containing source clauses, evidence fragments, knowledge paths, rule hit results, version status, and review status.

[0182] In this process, vector semantic retrieval, knowledge graph path retrieval, and structured rule object matching are performed separately using the same structured query object. The three types of results are then converted into vector evidence items, knowledge graph path evidence items, and rule object evidence items, respectively, before being fused and reordered. When forming structured search results, the associations between each evidence item are recorded based on the source clause, rule ID, entity code, business category tag, or version identifier. When a natural language answer is required, this invention only uses the generated result as a candidate answer, and before outputting, compares the entities, airport operation parameters, and candidate conclusions in the candidate answer with the currently valid structured rule object item by item. If a candidate answer is not matched with a currently valid structured rule object, the candidate conclusion is inconsistent with the rule conclusion, the cited source version is invalid or pending review, there are unresolved conflicts between the matched rules, or the source clause or evidence fragment is untraceable, the candidate answer must not be output as a reliable search result. For queries that require rule validation or rule-based conclusions, candidate answers should not be output as credible search results when no valid structured rule object is found, required parameters cannot be compared, entities cannot be disambiguated, the source of the reference is untraceable, or no clear conclusion has been reached through manual review. For queries that do not require rule validation and only request textual evidence, structured search results with source fields can be output.

[0183] This invention has many specific applications. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. A reliable retrieval method for airport operation rule verification based on a hybrid knowledge foundation, characterized in that, The steps are as follows: Receive airport operation query requests and convert the query requests into structured query objects; Perform vector semantic retrieval based on structured query objects; Knowledge graph path retrieval is performed based on structured query objects; Perform structured rule object matching based on structured query objects; The vector semantic retrieval results, knowledge graph path retrieval results, and rule object matching results are fused and reordered to obtain a candidate evidence set. Structured search results are generated based on the merged and reordered set of candidate evidence; Candidate answers are generated based on structured search results; Perform rule validation and conflict identification on candidate answers. If the rule validation passes, output reliable search results and generate audit records. Otherwise, perform backtracking search, answer revision or circulation on candidate answers that fail the rule validation.

2. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation as described in claim 1, characterized in that, The process of converting a query request into a structured query object specifically includes: Receive query requests from natural language queries, external business systems, or interface calls, wherein the query request includes at least one of the following: query text, request source, and business scenario; Perform intent recognition, entity recognition, airport operation parameter extraction, and query type determination on the query request, and output the recognition results; The identification results, query text, query source, and rule verification requirements are encapsulated into a structured query object.

3. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation as described in claim 2, characterized in that, The vector semantic retrieval based on structured query objects specifically includes: Convert query text and airport operation parameters into query vectors; Based on the similarity between the query vector and the semantically similar text block vector, semantically similar text blocks are retrieved from the vector database of the hybrid knowledge base, and the corresponding source document, source document page number, chapter title, version status, and review status are read. A set of semantic search results is generated based on vector similarity score, business category matching degree, source authority, version validity, and review status.

4. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation as described in claim 3, characterized in that, The hybrid knowledge base includes a vector database, a knowledge graph, and a structured rule base. The vector database stores text blocks of airport operation rules, regulations, operation manuals, historical Q&A, and case handling, along with their vector representations. It also records the source document, page number, chapter title, clause number, version status, effective status, and review status for each text block. The knowledge graph stores entity nodes and relational edges between airports, runways, taxiways, parking positions, aircraft, flights, operational phases, job types, regulations, and rule objects. The structured rule base stores rule objects, applicable entity types, scope of application, triggering conditions, constraints, threshold parameters, exceptions, effective versions, effective status, and rule priorities.

5. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation according to claim 4, characterized in that, The knowledge graph path retrieval based on structured query objects specifically includes: Based on the name, alias, code, and entity type of the queried entity, locate the corresponding entity node in the knowledge graph of the hybrid knowledge base; Starting from the entity node, path expansion is performed according to the relation types defined in the airport operation knowledge ontology; The path expansion range is limited according to the query intent, and graph path retrieval results containing entity nodes, relationship edges, path length, source terms, and version status are generated. The graph path score is calculated based on path length, degree of matching of relationship type, authority of node source, version validity, and review status.

6. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation as described in claim 5, characterized in that, The process of performing structured rule object matching based on structured query objects specifically includes: Based on the query entity, airport operation parameters, and query intent, rule objects are matched in the structured rule base of the hybrid knowledge foundation to generate rule object matching results. The rule object matching results include: the matched rule objects, the currently valid structured rule objects that have been filtered by effective version, effective status, and review status, the rule matching results, version status, review status, and rule matching score. The rule hit score is calculated based on the degree of matching between the rule object and the query entity, airport operating parameters, query intent, scope of application, triggering conditions, constraints, and rule priority. Increase the rule hit score when the current valid structured rule object is fully hit; decrease the rule hit score when no rule object is hit, only a historical valid rule object is hit, or a rule object pending review is hit. When the query contains computable field operation parameters, the computable field operation parameters are pre-matched with the threshold parameters, comparison operators and constraints in the rule object; when the rule object is in a historically valid, expired or pending review state, the version status and review status are marked in the rule object matching results.

7. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation as described in claim 6, characterized in that, The steps for obtaining the candidate evidence set specifically include: Read the vector similarity score from the vector semantic retrieval results, the graph path score from the graph path retrieval results, and the rule hit score from the rule object matching results; The fusion score is calculated by combining the source authority score, version validity score, review status score, business category matching score, and query intent matching score. The semantic retrieval results, graph path retrieval results, and rule object matching results are sorted according to the fusion score to obtain the sorted vector evidence, graph path evidence, and rule object evidence. The weight of results lacking rule support, lacking graph path support, having an expired source version, or having an abnormal review status is reduced.

8. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation as described in claim 7, characterized in that, The steps for generating structured search results specifically include: The sorted vector evidence, graph path evidence, and rule object evidence are organized into structured search results; The structured search results include: knowledge entry ID, content summary, source terms, source document, source document page number, evidence fragment, knowledge path, vector similarity score, graph path score, rule hit result, version validity, review status, and relevant airport operation parameters; When the query type does not require a natural language response, the structured search results are directly output to the external business system; when the query type requires a natural language response, the structured search results are used as the basis for generating candidate answers.

9. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation as described in claim 8, characterized in that, The generation of candidate answers based on structured search results specifically includes: Extract source terms, evidence fragments, knowledge paths, and rule-hitting results from the structured search results, and combine them with the query requests or query text in the structured query objects to assemble the candidate answer generation context. Based on the candidate answer generation context, candidate answers are generated, which include: candidate conclusions, answer text, a list of cited sources, relevant airport operation parameters, and evidence fragments corresponding to the candidate conclusions; The candidate answer is marked as pending verification and is prohibited from being output as a reliable result before it passes the rule verification.

10. The airport operation rule verification-based trusted retrieval method based on a hybrid knowledge foundation according to claim 9, characterized in that, The specific steps of performing rule verification and conflict identification on candidate answers include: Extract entities, airport operation parameters, and candidate conclusions from structured search results and candidate answers; The entities, airport operation parameters, and candidate conclusions are compared item by item with the currently valid structured rule objects that have been matched; the item-by-item comparison includes entity consistency comparison, airport operation parameter threshold comparison, and candidate conclusion compliance comparison; When a candidate answer satisfies all currently valid structured rules, the candidate answer is marked as passed by the rule. When a candidate answer violates any currently valid structured rule object, or when the rule object referenced by the candidate answer fails the current validity screening, the candidate answer is marked as a rule conflict or pending review status, and a conflict description is generated.

Images