Protocol standard document syntax tree error correction method and system based on large language model

Abstract

The application relates to a large language model-based protocol standard document syntax tree error correction method and system, which comprises the following steps: obtaining a target CCSDS protocol standard document, and extracting a protocol full-text; performing protocol recognition based on a large language model, and extracting a protocol context; creating a protocol syntax tree template based on the protocol context, and performing staged semantic extraction on the protocol full-text to construct an initial protocol syntax tree; extracting a set of entities to be verified from the initial protocol syntax tree; generating a set of verification questions according to the protocol context and the set of entities to be verified; generating a reference answer through a source standard document retrieval channel for each verification question; generating an answer to be evaluated through a protocol syntax tree question and answer channel; performing consistency evaluation on the reference answer and the answer to be evaluated, and generating protocol syntax tree correction data based on the reference answer according to the evaluation result. Compared with the prior art, the application effectively improves the efficiency, accuracy and verifiability of syntax tree generation.

Description

Technical Field

[0001] This invention relates to the field of document conversion technology, and in particular to a method and system for error correction of protocol standard document syntax trees based on large language models. Background Technology

[0002] The CCSDS (Consultative Committee on Space Data Systems) protocol standard documents serve as crucial technical guidelines for space data systems, satellite telemetry and control, space link communication, and space mission data transmission. These standards typically express protocol rules using natural language, field definitions, message structures, parameter descriptions, constraints, and protocol behavior specifications. For space network protocol analysis and implementation verification, it is necessary to convert these protocol rules into machine-readable, searchable, and verifiable structured protocol knowledge.

[0003] Existing protocol document processing methods typically focus on text retrieval, semantic question answering, or single-shot knowledge extraction. While these methods can extract some field or parameter information from protocol documents, they often suffer from the following shortcomings: First, the structured protocol results lack item-by-item consistency verification with the source standard document, making it difficult to determine whether the extracted results faithfully reflect the original protocol specifications. Second, the generated protocol knowledge is prone to omissions, type errors, missing constraints, or misleading content. Third, low-quality knowledge items usually rely on manual review and correction, lacking an automated closed-loop error correction mechanism. Fourth, there is a lack of a unified process between protocol knowledge generation, question answering verification, error correction optimization, and evaluation report output.

[0004] As large language models are increasingly used in document understanding, semantic extraction, and question answering generation, utilizing them to assist in parsing CCSDS protocol standard documents, constructing protocol syntax trees, and performing fact consistency verification has become a feasible approach. However, relying solely on large language models to generate protocol knowledge in a single step may still lead to structural drift, content omissions, and illusion errors.

[0005] Therefore, a method for efficient generation and error correction of CCSDS protocol syntax trees based on a large language model is needed. After the protocol syntax tree is generated, the source standard document is used to perform reverse verification of the generated result and to perform closed-loop correction of low consistency content. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology, such as low efficiency in generating protocol syntax trees, difficulty in verifying the generated results, and difficulty in locating and correcting low-consistency knowledge items, in the process of processing CCSDS protocol standard documents, and to provide a protocol standard document syntax tree error correction method and system based on a large language model.

[0007] The objective of this invention can be achieved through the following technical solutions: A method for error correction of protocol standard document syntax trees based on large language models includes the following steps: Obtain the target CCSDS protocol standard document, and perform document loading, format cleaning and full-text extraction on the target CCSDS protocol standard document to obtain the full-text protocol text. The protocol is identified based on a large language model, and the protocol context is extracted. A protocol syntax tree template is constructed based on the protocol context, and the full text of the protocol is subjected to phased semantic extraction to obtain basic protocol information, message type, parameter list and parameter constraints, so as to construct an initial protocol syntax tree. Extract entities to be verified from the initial protocol syntax tree, sample the entities to be verified based on their importance to obtain a set of entities to be verified; generate a set of verification questions based on the protocol context and the set of entities to be verified. For each verification question in the set of verification questions, the full text of the protocol is retrieved for relevant context through the source standard document retrieval channel to generate a reference answer; and an answer to be evaluated is generated through the protocol syntax tree question-and-answer channel based on the verification question, the corresponding entity to be verified, and the structured content of the initial protocol syntax tree. A consistency evaluation is performed on the reference answer and the answer to be evaluated. If the preset consistency conditions are not met, protocol syntax tree correction data is generated based on the reference answer to update the initial protocol syntax tree.

[0008] Furthermore, the document loading and format cleaning include: Load the PDF, HTML, TXT, XML, Markdown, or Word text content from the target CCSDS protocol standard document, and remove formatting noise, invalid whitespace, and non-text content to obtain the full text of the protocol.

[0009] Furthermore, the protocol identification includes: Based on the document title, protocol terminology, message description, parameter description, field definition, constraint statement and context semantics in the full text of the protocol, the protocol name, protocol version, protocol type, transmission characteristics, key protocol characteristics and protocol context are identified, thereby extracting the protocol context.

[0010] Furthermore, the phased semantic extraction includes: Extract basic protocol information nodes from the full text of the protocol, then identify and enumerate various message types, methods, commands and message formats, supplement descriptive information and transmission direction for each message type, and extract the parameter list corresponding to each message and the parameter constraints of all parameters. The parameter constraints include one or more of the following: parameter name, data type, encoding method, length information, value range, default value, constraint conditions, and parameter description; The process of constructing the initial protocol syntax tree includes: The extracted protocol basic information nodes; various message types, methods, commands and message formats; description information and transmission direction of each message type; parameter list corresponding to each message and parameter constraints of all parameters are filled into the protocol syntax tree template to construct the initial protocol syntax tree.

[0011] Furthermore, the entity to be verified includes one or more of the following: protocol message type, protocol parameter, field constraint, parameter type, parameter value range, encoding method, and length limit; The entity importance is determined based on one or more of the following: term frequency inverse document frequency feature, entity quality score, entity importance, or entity diversity. Based on entity importance, entities to be verified are sampled to obtain a set of entities to be verified, including: Calculate the entity weight of each entity to be verified based on entity importance, and then sample according to the entity weight to construct a set of entities to be verified.

[0012] Furthermore, the verification problem includes one or more of the following: value-based problems, type-based problems, constraint-based problems, and behavior-based problems; The verification question is generated based on the protocol context, the entity to be verified, and the corresponding preset dynamic question template.

[0013] Furthermore, the processing procedure for the source standard document retrieval channel includes: The full text of the protocol is divided into blocks to obtain multiple blocks, and the blocks are vectorized to form a similarity retrieval index for each block's vector. Based on the verification question to be processed, the relevant contextual block text is retrieved based on the similarity retrieval index, and a reference answer is generated based on the retrieval results; The processing procedure of the protocol syntax tree question-and-answer channel includes: Based on the verification question, the corresponding entity to be verified, and the initial protocol syntax tree, the relevant message types, parameter nodes, and constraint content are located, and the answer to be evaluated is generated based on the structured content of the initial protocol syntax tree.

[0014] Furthermore, the consistency assessment includes the calculation of basic indicators and the comparison of protocol-aware semantics of the large language model; The basic metrics include one or more of the following: exact match, F1 score, and semantic similarity. The large language model protocol-aware semantic comparison is used to determine the degree of consistency between the reference answer and the answer to be evaluated in terms of protocol rules, field meanings, constraint relationships, and technical accuracy. The method generates a final consistency score based on the results of basic index calculation and large language model protocol-aware semantic comparison, and divides the consistency levels into high consistency, medium consistency and low consistency. If the consistency level is classified as high consistency, then the consistency condition is met; otherwise, the consistency condition is not met.

[0015] Furthermore, the protocol syntax tree correction data includes the field to be corrected, the content before correction, the content after correction, and the basis for correction; the basis for correction includes at least the reference answer generated by the source standard document retrieval channel and relevant CCSDS protocol document fragments; The method further includes: regenerating the answer to be evaluated from the updated initial protocol syntax tree, and performing a consistency evaluation again with the corresponding reference answer to determine whether the consistency condition is met, so as to determine whether the modification of the protocol syntax tree improves consistency.

[0016] This invention also provides a protocol standard document syntax tree error correction system that implements the above-described large language model-based protocol standard document syntax tree error correction method, comprising: The document loading and cleaning module is used to obtain the target CCSDS protocol standard document, and to perform document loading, format cleaning and full-text extraction on the target CCSDS protocol standard document to obtain the full-text protocol text. The protocol recognition module is used to perform protocol recognition on the full text of the protocol based on a large language model and extract the protocol context. The protocol syntax tree extraction module is used to construct a protocol syntax tree template based on the protocol context, and to perform phased semantic extraction on the full text of the protocol to obtain basic protocol information, message types, parameter lists and parameter constraints, so as to construct an initial protocol syntax tree. The entity extraction and sampling module is used to extract entities to be verified from the initial protocol syntax tree, sample the entities to be verified based on their importance, and obtain a set of entities to be verified. The verification question generation module is used to generate a set of verification questions based on the protocol context and the set of entities to be verified. The source standard document answer generation module is used to generate a reference answer for each verification question in the set of verification questions by searching the full text of the protocol for relevant context through the source standard document retrieval channel. The protocol syntax tree answer generation module is used to generate an answer to be evaluated based on the verification question, the corresponding entity to be verified, and the structured content of the initial protocol syntax tree through the protocol syntax tree question and answer channel; The consistency assessment module is used to assess the consistency between the reference answer and the answer to be assessed. The low consistency error correction module is used to generate protocol syntax tree correction data based on the reference answer when the preset consistency conditions are not met, so as to update the initial protocol syntax tree.

[0017] Compared with the prior art, the present invention has the following advantages: (1) This invention improves the efficiency of structured parsing of protocol standard documents by loading and cleaning the target CCSDS protocol standard document path, and then performing CCSDS protocol identification and phased extraction of protocol syntax tree based on a large language model. The initial protocol syntax tree obtained by phased extraction is used to extract entities to be verified. Through entity importance sampling and dynamic verification question generation, the content of the protocol syntax tree is transformed into a verifiable question-and-answer task, which can realize automated consistency verification of the protocol syntax tree. By generating dual-channel answers through the source standard document retrieval channel and the protocol syntax tree question-and-answer channel, the answer in the source standard document is used as a reference and the answer in the protocol syntax tree is used as the object to be evaluated, thereby improving the interpretability of the consistency evaluation. Overall, it effectively improves the efficiency, accuracy, and verifiability of CCSDS protocol syntax tree generation.

[0018] (2) The present invention adopts a flat protocol syntax tree template and a phased semantic extraction method to organize the basic protocol information, message type, parameter list and parameter constraints into structured protocol knowledge, thereby reducing information omission and structural drift caused by one-time extraction.

[0019] (3) In the process of entity importance sampling, the present invention calculates entity weights based on word frequency-inverse document frequency features, entity quality scores, entity importance and entity diversity, and performs intelligent sampling based on entity weights to obtain a set of entities to be verified. In this way, key knowledge items in the protocol syntax tree can be covered first while controlling verification costs.

[0020] (4) The present invention forms a closed-loop error correction process through low consistency sample location, error correction target construction, protocol syntax tree correction data generation, protocol syntax tree update and re-evaluation, which can continuously improve the consistency between the protocol syntax tree and the source standard document.

[0021] (5) The present invention outputs the initial protocol syntax tree, the revised protocol syntax tree, the consistency assessment report and the comparison results before and after error correction, which facilitates the tracking, review and continuous optimization of the CCSDS protocol structure results. Attached Figure Description

[0022] Figure 1This is a flowchart illustrating a protocol standard document syntax tree error correction method based on a large language model provided in an embodiment of the present invention; Figure 2 This is a flowchart of the protocol syntax tree construction stage provided in an embodiment of the present invention; Figure 3 This is a flowchart of a dual-channel answer generation and consistency evaluation process provided in an embodiment of the present invention; Figure 4 This is a flowchart of a low-consistency closed-loop error correction method provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the module composition of a protocol standard document syntax tree error correction system provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] Example 1 like Figure 1 As shown, this embodiment provides a method for error correction of protocol standard document syntax trees based on large language models, including the following steps: S1: Obtain the target CCSDS protocol standard document, load the target CCSDS protocol standard document, clean the format and extract the full text of the protocol to obtain the full text of the protocol. Document loading and format cleaning include: Load the CCSDS protocol standard document in PDF, HTML, TXT, XML, Markdown, Word or other parsable text formats from the target CCSDS protocol standard document, remove formatting noise, invalid whitespace and non-text content, and retain the protocol text, chapter titles, paragraph content, table field descriptions and contextual text to obtain the full text of the protocol.

[0027] S2: Based on the large language model, perform protocol recognition on the full text of the protocol and extract the protocol context; based on the protocol context, construct a protocol syntax tree template and perform phased semantic extraction on the full text of the protocol to obtain basic protocol information, message type, parameter list and parameter constraints in order to construct the initial protocol syntax tree; Protocol identification includes: Based on the document title, protocol terminology, message description, parameter description, field definition, constraint statement, and contextual semantics in the full text of the protocol, the protocol name, protocol version, protocol type, transmission characteristics, key protocol characteristics, and protocol context are identified, and the protocol context is used for subsequent protocol syntax tree generation, verification question generation, answer generation, and consistency assessment.

[0028] The protocol syntax tree template is a flat protocol syntax tree template.

[0029] Phased semantic extraction includes: Extract basic protocol information nodes from the full text of the protocol, then identify and enumerate various message types, methods, commands and message formats, supplement descriptive information and transmission direction for each message type, and extract the parameter list corresponding to each message and the parameter constraints of all parameters. Parameter constraints include one or more of the following: parameter name, data type, encoding method, length information, value range, default value, constraint condition, and parameter description. The process of constructing the initial protocol syntax tree includes: The extracted basic protocol information nodes; various message types, methods, commands and message formats; description information and transmission direction of each message type; parameter list corresponding to each message and parameter constraints of all parameters are filled into the protocol syntax tree template to construct the initial protocol syntax tree.

[0030] The initial protocol syntax tree is stored in the form of structured data, including at least basic protocol information, message type information, parameter information, parameter constraint information, and metadata information.

[0031] S3: Extract entities to be verified from the initial protocol syntax tree, sample entities to be verified based on entity importance to obtain a set of entities to be verified; generate a set of verification questions based on the protocol context and the set of entities to be verified. The entity to be verified includes one or more of the following: protocol message type, protocol parameter, field constraint, parameter type, parameter value range, encoding method, and length limit; Entity importance is determined based on one or more of the following: term frequency inverse document frequency features, entity quality score, entity importance, or entity diversity. Based on entity importance, entities to be verified are sampled to obtain a set of entities to be verified, including: Calculate the entity weight of each entity to be verified based on entity importance, and then sample according to the entity weight to construct a set of entities to be verified.

[0032] Validation problems include one or more of the following: value-based problems, type-based problems, constraint-based problems, and behavior-based problems. Verification questions are generated based on the protocol context, the entity to be verified, and the corresponding preset dynamic question templates.

[0033] S4: For each verification question in the verification question set, retrieve the relevant context of the full text of the protocol through the source standard document retrieval channel to generate a reference answer; generate the answer to be evaluated through the protocol syntax tree question-and-answer channel based on the verification question, the corresponding entity to be verified, and the structured content of the initial protocol syntax tree. Specifically, the processing steps for the source standard document retrieval channel include: The full text of the protocol is divided into blocks to obtain multiple blocks, and the blocks are vectorized to create a similarity retrieval index for each block's vector. Based on the verification question to be processed, the relevant context blocks are retrieved using the similarity retrieval index, and a reference answer is generated based on the retrieval results. The processing steps of the protocol syntax tree question-and-answer channel include: Based on the verification question, the corresponding entity to be verified, and the initial protocol syntax tree, the relevant message types, parameter nodes, and constraint content are located, and the answer to be evaluated is generated based on the structured content of the initial protocol syntax tree.

[0034] S5: Perform a consistency evaluation on the reference answer and the answer to be evaluated. If the preset consistency conditions are not met, generate protocol syntax tree correction data based on the reference answer to update the initial protocol syntax tree.

[0035] Consistency assessment includes the calculation of basic indicators and the comparison of protocol-aware semantics of large language models; The basic metrics include one or more of the following: exact match, F1 score, and semantic similarity. Large language model protocol-aware semantic comparison is used to determine the degree of consistency between the reference answer and the answer to be evaluated in terms of protocol rules, field meanings, constraint relationships, and technical accuracy; The method generates a final consistency score based on the results of basic index calculation and large language model protocol-aware semantic comparison, and divides the consistency levels into high consistency, medium consistency and low consistency. When the consistency level is not high consistency and the consistency score is lower than the preset threshold, the corresponding verification question, target entity, reference answer and answer to be evaluated are marked as low consistency samples, that is, they do not meet the consistency conditions.

[0036] The protocol syntax tree correction data includes the field to be corrected, the content before correction, the content after correction, and the basis for correction; the basis for correction includes at least the reference answer generated by the source standard document retrieval channel and relevant CCSDS protocol document fragments; S6: Regenerate the answer to be evaluated from the updated initial protocol syntax tree, and perform a consistency evaluation again with the corresponding reference answer to determine whether the consistency conditions are met, so as to determine whether the correction of the protocol syntax tree improves consistency. Output the corrected protocol syntax tree, the consistency evaluation report, and the comparison results before and after the error correction.

[0037] Below is an example of a specific implementation process for the above solution: This example solution mainly includes loading and cleaning the CCSDS protocol standard document, protocol identification and context generation, initial protocol syntax tree generation, dual-channel answer generation, consistency evaluation, low-consistency closed-loop error correction, and result output. The phased construction process of the protocol syntax tree is as follows: Figure 2 As shown, the dual-channel answer generation and consistency evaluation process is as follows: Figure 3 As shown, the error correction and re-evaluation process for low-consistency samples is as follows: Figure 4 As shown, the system modules are composed as follows: Figure 5 As shown.

[0038] Specifically, the following steps are included: Step 1) Document loading and cleaning.

[0039] The system receives the path to the target CCSDS protocol standard document and calls the document loading module to read the protocol document content. For PDF, HTML, TXT, XML, Markdown, Word, or other parsable text formats, the system performs format cleaning, invalid character removal, text extraction, and full-text merging to obtain the full-text protocol. This full-text protocol serves as the basic input for subsequent protocol recognition, protocol syntax tree extraction, and source standard document retrieval.

[0040] Step 2) Protocol identification and context generation.

[0041] The system uses a large language model to identify the protocol in the full text, obtaining the protocol name, version, type, transmission characteristics, key protocol features, and protocol context. The protocol context is then passed to the question generation module, answer generation module, and consistency evaluation module to ensure that subsequent generated content remains consistent with the semantics of the current CCSDS protocol.

[0042] Step 3) Dynamic extraction of the initial protocol syntax tree.

[0043] Based on the aforementioned protocol identification results, the system of this invention combines the full-text protocol and its context to construct a flattened protocol syntax tree template. It then employs a phased, progressive semantic extraction method to extract protocol knowledge: first, it populates the basic protocol information nodes; then, it identifies and enumerates the various message types, methods, commands, and message formats included in the protocol, supplementing each message type with descriptive information and transmission direction; subsequently, it extracts the parameter list corresponding to each message; and finally, it extracts the technical constraints of all parameters. Parameter constraints cover dimensions such as parameter name, data type, encoding rules, length definition, value range, default value, conditional constraints, and parameter description. After the above extraction process is completed, the system automatically generates an initial protocol syntax tree and persistently stores it in a structured data format. This structured data format includes at least basic protocol information, message type information, parameter information, parameter constraint information, and metadata information.

[0044] Step 4) Entity extraction and importance sampling.

[0045] The system extracts entities to be verified from the initial protocol syntax tree. These entities include protocol message type, protocol parameters, field constraints, parameter types, parameter value ranges, encoding methods, and length limits. The system calculates entity weights based on term frequency-inverse document frequency features, entity quality scores, entity importance, and entity diversity, and performs intelligent sampling based on these weights to obtain the set of entities to be verified. This approach allows for priority coverage of key knowledge items in the protocol syntax tree while controlling verification costs.

[0046] Step 5) Verify the generated question.

[0047] The system generates verification questions based on the protocol context and the set of entities to be verified. Verification questions include value-based questions, type-based questions, constraint-based questions, and behavioral questions. Value-based questions are used to verify the range of parameter values, default values, or valid values; type-based questions are used to verify parameter types, encoding methods, or length information; constraint-based questions are used to verify parameter constraints, conditional relationships, or limiting rules; and behavioral questions are used to verify protocol behavior, processing logic, or state descriptions.

[0048] Step 6) Dual-channel answer generation.

[0049] For each verification question, the system generates both a source standard document answer and a protocol syntax tree answer. The source standard document retrieval channel segments the full-text protocol into multiple text blocks, vectorizes these blocks, and establishes a similarity retrieval index. The system retrieves relevant context based on the verification question and generates the source standard document answer based on that context. The protocol syntax tree question-answering channel generates the protocol syntax tree answer based on the verification question, the target entity, and the structured content of the protocol syntax tree.

[0050] In this embodiment, the answer from the source standard document serves as the reference answer, while the answer from the protocol syntax tree serves as the answer to be evaluated. Through dual-channel answer generation, the system can compare the consistency between the source standard document's basis and the protocol syntax tree's structured expression under the same verification question.

[0051] Step 7) Consistency assessment.

[0052] The system performs a consistency assessment on the reference answer and the answer to be evaluated. The consistency assessment includes the calculation of basic indicators and a protocol-aware semantic comparison using a large language model. The basic indicators include exact matching, F1 score, and semantic similarity; the protocol-aware semantic comparison using the large language model is used to determine whether the reference answer and the answer to be evaluated are consistent in terms of field meaning, protocol rules, constraint relationships, and technical accuracy. Based on the results of the basic indicators and the protocol-aware semantic comparison using the large language model, the system generates a final consistency score and assigns a consistency level.

[0053] When the consistency level is high, it means that the corresponding knowledge in the protocol syntax tree is basically consistent with the source standard document; when the consistency level is medium or low, the system records the difference description. If the consistency level is not high and the final consistency score is lower than the preset threshold, the sample is marked as a low consistency sample and enters the closed-loop error correction process.

[0054] Step 8) Low consistency closed-loop error correction.

[0055] For samples with low consistency, the system extracts the entities that need correction and constructs correction targets. The correction targets include the validation question, target entity, reference answer, answer to be evaluated, consistency evaluation result, and relevant source standard document context. The system uses the source standard document answer as the authoritative reference to generate protocol syntax tree correction data. The protocol syntax tree correction data includes the field to be corrected, the content before correction, the content after correction, and the basis for correction.

[0056] The system applies the corrected protocol syntax tree data to the original protocol syntax tree to obtain the corrected protocol syntax tree. Subsequently, the system performs a re-evaluation of the corrected protocol syntax tree, that is, it regenerates the protocol syntax tree answer based on the corrected protocol syntax tree and compares it again with the answer in the source standard document. If the re-evaluation result meets preset conditions, the corrected protocol syntax tree is output; otherwise, error correction continues within a preset number of rounds.

[0057] Step 9) Output the results.

[0058] The system ultimately outputs an initial protocol syntax tree, a revised protocol syntax tree, a conformance assessment report, error correction analysis results, and detailed intermediate results. The detailed intermediate results include a set of generated questions, a set of question-and-answer results, and a set of conformance assessment results. Furthermore, the system can save error diagnosis results, correction strategies, reassessment results, and before-and-after comparison information for subsequent result traceability and manual review.

[0059] Example 2 This embodiment provides a protocol standard document syntax tree correction system that implements the protocol standard document syntax tree correction method based on a large language model as described in Embodiment 1, including: The document loading and cleaning module is used to obtain the target CCSDS protocol standard document, load the target CCSDS protocol standard document, clean the format and extract the full text of the protocol to obtain the full text of the protocol. The protocol recognition module is used to identify the protocol in the full text of the protocol based on a large language model and extract the protocol context. The protocol syntax tree extraction module is used to build a protocol syntax tree template based on the protocol context, and to perform phased semantic extraction on the full text of the protocol to obtain basic protocol information, message types, parameter lists and parameter constraints in order to build an initial protocol syntax tree. The entity extraction and sampling module is used to extract entities to be verified from the initial protocol syntax tree, sample the entities to be verified based on their importance, and obtain a set of entities to be verified. The verification question generation module is used to generate a set of verification questions based on the protocol context and the set of entities to be verified. The source standard document answer generation module is used to generate a reference answer for each verification question in the verification question set by searching the full text of the protocol for relevant context through the source standard document retrieval channel. The protocol syntax tree answer generation module is used to generate an answer to be evaluated based on the verification question, the corresponding entity to be verified, and the structured content of the initial protocol syntax tree through the protocol syntax tree question and answer channel; The consistency assessment module is used to assess the consistency between the reference answer and the answer to be assessed. The low consistency error correction module is used to generate a protocol syntax tree correction data based on the reference answer when the preset consistency conditions are not met, so as to update the initial protocol syntax tree. The re-evaluation module is used to regenerate the answer to be evaluated from the updated initial protocol syntax tree and perform a consistency evaluation again with the corresponding reference answer to determine whether the consistency conditions are met, so as to determine whether the modification of the protocol syntax tree improves consistency. The results output module is used to output the initial protocol syntax tree, the revised protocol syntax tree, the conformance assessment report, the error correction analysis results, and detailed intermediate results.

[0060] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for error correction of protocol standard document syntax trees based on large language models, characterized in that, Includes the following steps: Obtain the target CCSDS protocol standard document, and perform document loading, format cleaning and full-text extraction on the target CCSDS protocol standard document to obtain the full-text protocol text. The protocol is identified based on a large language model, and the protocol context is extracted. A protocol syntax tree template is constructed based on the protocol context, and the full text of the protocol is subjected to phased semantic extraction to obtain basic protocol information, message type, parameter list and parameter constraints, so as to construct an initial protocol syntax tree. Extract entities to be verified from the initial protocol syntax tree, sample the entities to be verified based on their importance to obtain a set of entities to be verified; generate a set of verification questions based on the protocol context and the set of entities to be verified. For each verification question in the set of verification questions, the full text of the protocol is retrieved for relevant context through the source standard document retrieval channel to generate a reference answer; The protocol syntax tree question-and-answer channel generates an answer to be evaluated based on the verification question, the corresponding entity to be verified, and the structured content of the initial protocol syntax tree. A consistency evaluation is performed on the reference answer and the answer to be evaluated. If the preset consistency conditions are not met, protocol syntax tree correction data is generated based on the reference answer to update the initial protocol syntax tree.

2. The method for error correction of protocol standard document syntax trees based on a large language model according to claim 1, characterized in that, The document loading and format cleaning include: Load the PDF, HTML, TXT, XML, Markdown, or Word text content from the target CCSDS protocol standard document, and remove formatting noise, invalid whitespace, and non-text content to obtain the full text of the protocol.

3. The method for error correction of protocol standard document syntax trees based on a large language model according to claim 1, characterized in that, The protocol identification includes: Based on the document title, protocol terminology, message description, parameter description, field definition, constraint statement and context semantics in the full text of the protocol, the protocol name, protocol version, protocol type, transmission characteristics, key protocol characteristics and protocol context are identified, thereby extracting the protocol context.

4. The method for error correction of protocol standard document syntax trees based on a large language model according to claim 1, characterized in that, The phased semantic extraction includes: Extract basic protocol information nodes from the full text of the protocol, then identify and enumerate various message types, methods, commands and message formats, supplement descriptive information and transmission direction for each message type, and extract the parameter list corresponding to each message and the parameter constraints of all parameters. The parameter constraints include one or more of the following: parameter name, data type, encoding method, length information, value range, default value, constraint conditions, and parameter description; The process of constructing the initial protocol syntax tree includes: The extracted protocol basic information nodes; various message types, methods, commands and message formats; description information and transmission direction of each message type; parameter list corresponding to each message and parameter constraints of all parameters are filled into the protocol syntax tree template to construct the initial protocol syntax tree.

5. The method for error correction of protocol standard document syntax trees based on a large language model according to claim 1, characterized in that, The entity to be verified includes one or more of the following: protocol message type, protocol parameter, field constraint, parameter type, parameter value range, encoding method, and length limit; The entity importance is determined based on one or more of the following: term frequency inverse document frequency feature, entity quality score, entity importance, or entity diversity. Based on entity importance, entities to be verified are sampled to obtain a set of entities to be verified, including: Calculate the entity weight of each entity to be verified based on entity importance, and then sample according to the entity weight to construct a set of entities to be verified.

6. The method for error correction of protocol standard document syntax trees based on a large language model according to claim 1, characterized in that, The verification problems include one or more of the following: value-based problems, type-based problems, constraint-based problems, and behavior-based problems. The verification question is generated based on the protocol context, the entity to be verified, and the corresponding preset dynamic question template.

7. The method for error correction of protocol standard document syntax trees based on a large language model according to claim 1, characterized in that, The processing steps of the source standard document retrieval channel include: The full text of the protocol is divided into blocks to obtain multiple blocks, and the blocks are vectorized to form a similarity retrieval index for each block's vector. Based on the verification question to be processed, the relevant contextual block text is retrieved based on the similarity retrieval index, and a reference answer is generated based on the retrieval results; The processing procedure of the protocol syntax tree question-and-answer channel includes: Based on the verification question, the corresponding entity to be verified, and the initial protocol syntax tree, the relevant message types, parameter nodes, and constraint content are located, and the answer to be evaluated is generated based on the structured content of the initial protocol syntax tree.

8. The method for error correction of protocol standard document syntax trees based on a large language model according to claim 1, characterized in that, The consistency assessment includes the calculation of basic indicators and the comparison of protocol-aware semantics of the large language model. The basic metrics include one or more of the following: exact match, F1 score, and semantic similarity. The large language model protocol-aware semantic comparison is used to determine the degree of consistency between the reference answer and the answer to be evaluated in terms of protocol rules, field meanings, constraint relationships, and technical accuracy. The method generates a final consistency score based on the results of basic index calculation and large language model protocol-aware semantic comparison, and divides the consistency levels into high consistency, medium consistency and low consistency. If the consistency level is classified as high consistency, then the consistency condition is met; otherwise, the consistency condition is not met.

9. The method for error correction of protocol standard document syntax trees based on a large language model according to claim 1, characterized in that, The protocol syntax tree correction data includes the field to be corrected, the content before correction, the content after correction, and the basis for correction; The basis for the correction includes at least the reference answers generated by the source standard document retrieval channel and relevant CCSDS protocol document fragments; The method further includes: regenerating the answer to be evaluated from the updated initial protocol syntax tree, and performing a consistency evaluation again with the corresponding reference answer to determine whether the consistency condition is met, so as to determine whether the modification of the protocol syntax tree improves consistency.

10. A protocol standard document syntax tree error correction system that implements the protocol standard document syntax tree error correction method based on a large language model as described in any one of claims 1-9, characterized in that, include: The document loading and cleaning module is used to acquire the target CCSDS protocol standard document, load the target CCSDS protocol standard document, clean the format and extract the full text of the protocol to obtain the full text of the protocol. The protocol recognition module is used to perform protocol recognition on the full text of the protocol based on a large language model and extract the protocol context. The protocol syntax tree extraction module is used to construct and create a protocol syntax tree template based on the protocol context, and to perform phased semantic extraction on the full text of the protocol to obtain basic protocol information, message types, parameter lists and parameter constraints, so as to construct an initial protocol syntax tree. The entity extraction and sampling module is used to extract entities to be verified from the initial protocol syntax tree, sample the entities to be verified based on their importance, and obtain a set of entities to be verified. The verification question generation module is used to generate a set of verification questions based on the protocol context and the set of entities to be verified. The source standard document answer generation module is used to generate a reference answer for each verification question in the set of verification questions by searching the full text of the protocol for relevant context through the source standard document retrieval channel. The protocol syntax tree answer generation module is used to generate an answer to be evaluated based on the verification question, the corresponding entity to be verified, and the structured content of the initial protocol syntax tree through the protocol syntax tree question and answer channel; The consistency assessment module is used to assess the consistency between the reference answer and the answer to be assessed. The low consistency error correction module is used to generate protocol syntax tree correction data based on the reference answer when the preset consistency conditions are not met, so as to update the initial protocol syntax tree.

Images