Network protocol intelligent extraction method based on large language model and application

Abstract

The invention relates to a network protocol intelligent extraction method and application based on a large language model, and the method comprises the steps: taking a network protocol specification text as an analysis object, and initializing each candidate prompt word; a large language model is constructed, scoring consideration of three indexes including accuracy, format compliance and coverage rate under network protocol specification text extraction application is achieved according to the candidate cue words, cue word variants under a preset cue word variant mechanism are matched under loop iteration, all target cue words are obtained in an optimized mode, and then the target cue words are obtained on the basis of all the target cue words; analysis and extraction of chart description are deepened, efficient and accurate acquisition of a target network protocol text about a protocol message format and a protocol interaction process is achieved by applying a large language model, and compared with a traditional method, the design scheme has obvious advantages in the aspects of accuracy, generalization ability, information utilization, complex protocol processing and the like. And a more efficient and reliable way is provided for extraction of protocol specifications.

Description

Technical Field

[0001] The present invention relates to a network protocol intelligent extraction method based on a large language model and its application, belonging to the technical field of network protocol text extraction. Background Art

[0002] Industrial control protocols are a key component of networked collaborative manufacturing. While significantly improving the efficiency of organizing and controlling the interactions between numerous devices within manufacturing systems, they also face increasing security risks. Industrial control systems rely on proprietary industrial control protocols to enable communication between components, thereby controlling various industrial control devices within the industrial control network. Their security is crucial to the security of the entire industrial control system. Network protocols are the foundation for communication between devices and systems, but analysis and implementation are complex, and the complexity and diversity of network protocols make traditional manual analysis and implementation time-consuming and error-prone.

[0003] In addition to relying on manual analysis, existing technologies for extracting protocol message formats from protocol specification texts (such as RFC documents) can also utilize automated methods based on rule engines, natural language processing technologies (such as lexical and syntactic analysis, named entity recognition), formal methods, and deep learning (such as convolutional neural networks and recurrent neural networks) to extract protocol message formats from protocol specification texts.

[0004] The rule engine-based approach matches and extracts protocol specification text by formulating a series of rules related to the protocol message format. Specifically, rules such as message header structure, message body structure, and field types can be defined based on the protocol's standard documents or common format patterns. When processing the protocol specification text, the rule engine scans and parses the text according to these rules to identify the message format parts that meet the rules. The advantage of this method is that the extraction process has clear logic and explainability, and it can efficiently and accurately extract message formats for protocols with clear rules. The disadvantage is that it requires a lot of effort to formulate comprehensive rules in advance, and it has poor adaptability to changes in the protocol. Once the protocol changes, the rules may need to be readjusted.

[0005] Natural language processing (NLP)-based methods include lexical analysis and syntactic analysis. Lexical analysis breaks down protocol specification text into individual words or symbols, identifying key terms and vocabulary. Syntactic analysis parses the grammatical structure of sentences and understands the relationships between their components. By analyzing these structures, information related to the message format, such as how message fields are combined, can be inferred.

[0006] Named Entity Recognition (NER) is specifically used to identify entities with specific meanings in text, such as message types, field names, and data types in protocols. Within protocol specification text, NER can accurately locate these key entities, providing a foundation for subsequent extraction of the complete message format. Combined with other NLP techniques, it can more comprehensively extract protocol message formats. The advantage of NLP lies in its powerful processing capabilities for natural language text and its adaptability to certain text variations. However, for highly specialized and complex text like protocol specifications, achieving ideal extraction results may require extensive training data and sophisticated model adjustments.

[0007] Deep learning-based methods, such as convolutional neural networks (CNNs), have achieved tremendous success in image processing and can also be applied to text processing. By converting protocol specification text into a format suitable for CNN processing, CNNs can automatically extract local features from the text through convolutional layers. These features may correspond to key components of the protocol message format, such as identifying specific recurring structural patterns in the text and inferring the message format. CNNs have the advantage of automatically learning features and strong processing capabilities for large-scale data. However, training CNNs requires a large amount of labeled data, and the model's interpretability is relatively poor. Regarding recurrent neural networks (RNNs) and their variants (such as LSTMs and GRUs), RNNs can process text data with sequential characteristics and consider contextual relationships within the text. In protocol specification text, message formats are often context-dependent. RNNs and their variants can better capture these relationships through memory mechanisms, thereby accurately extracting the protocol message format. However, RNNs may encounter gradient vanishing or gradient exploding issues during training, requiring careful parameter tuning and optimization. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a network protocol intelligent extraction method based on a large language model, optimize the prompt words in the application of the large language model, and efficiently complete the extraction of network protocol text.

[0009] To solve the above technical problems, the present invention adopts the following technical solution: the present invention designs a network protocol intelligent extraction method based on a large language model, performing the following steps A to E to obtain target prompt words extracted from the network protocol text corresponding to the large language model;

[0010] Step A. Initialize the number of iterations Based on the preset number of network protocol specification texts, the large language model is initialized to extract the corresponding prompt words of the protocol message format and protocol interaction process in each network protocol specification text, forming the first Each candidate prompt word in the iteration, and then enter step B;

[0011] Step B. Construct a scoring function based on accuracy, format compliance, and coverage to obtain the first The scores of each candidate prompt word in the iteration are respectively corresponding to each candidate prompt word, and the candidate prompt words are sorted from large to small based on the scores. A preset proportion of candidate prompt words are selected before the order is filtered as the first Each scoring filter prompt word in the iteration, and then enter step C;

[0012] Step C. Judgment Is it greater than or equal to the preset number of evaluation iterations? If yes, go to step D; otherwise go to step E;

[0013] Step D. Obtain Iterations to The highest score in each iteration in the order of the iterations is used to determine whether the increase in the highest score is less than the preset stable increase threshold. If so, the Each scored and filtered prompt word in the iteration constitutes each target prompt word extracted by the large language model corresponding to the network protocol text; otherwise, proceed to step E;

[0014] Step E. According to the preset prompt word variation mechanism, In the iteration, each scoring filter prompt word is processed by the large language module to obtain the Each hint word variant under the iteration, combined with the Each scoring filter prompt word in the iteration is updated to form the +1 iteration for each candidate prompt word, and then for The value of is updated by adding 1 and returning to step B.

[0015] As a preferred technical solution of the present invention: in step A, the large language model is initialized to extract the prompt words of the protocol message format and the protocol interaction process in the specification text of each network protocol in the following manner;

[0016] Method a: Zero-shot prompting, in the order of coarse-grained extraction to fine-grained extraction, directly requires the large language model to generate various prompt words for extracting the protocol message format from the network protocol specification text;

[0017] Method b. Few-sample prompts: This is constructed based on examples in the network protocol specification texts. The large language model is required to extract the prompt words of the protocol message format from the network protocol specification texts.

[0018] Method c. Task decomposition and combination: First, parse the network protocol specification text, extract all state names, and based on each state name, list all transition events to form each subtask. Then, build a large language model to extract prompts for each subtask from the network protocol specification text and combine them to form prompt words.

[0019] Step d. Hierarchical message extraction: if the network protocol specification text has a nested or hierarchical message structure, a large language model is constructed to recursively extract the various prompt words of the nested fields in the protocol message format from the network protocol specification text;

[0020] Step e. Applying a preset large language model to extract each prompt word corresponding to the network protocol specification text.

[0021] As a preferred technical solution of the present invention: in the step B, respectively For each candidate prompt word in the iteration, follow steps B1 to B2 to obtain the score corresponding to the candidate prompt word;

[0022] Step B1. Regarding accuracy, the large language model first extracts each field from each network protocol specification text based on the candidate prompt words, constructs a first set, and obtains the number of fields in the first set. At the same time, based on the manually annotated fields of the candidate prompt words corresponding to the network protocol specification text, a second set is constructed, and the number of fields in the second set is obtained. ; Then determine the number of fields in the intersection of the first set and the second set , and calculate the accuracy , recall rate ; Finally press , get the accuracy score corresponding to the candidate prompt word ;

[0023] Regarding format compliance, the large language model uses candidate prompts to extract each field from each network protocol specification text and determines whether the format of each field is legal. If so, the format compliance score corresponding to the candidate prompt is 1; otherwise, the format compliance score corresponding to the candidate prompt is 0.

[0024] Regarding coverage, recall The coverage score corresponding to the candidate prompt words;

[0025] Step B2. Calculate the preset weights for accuracy, format compliance, and coverage. 、 、 , the accuracy scores corresponding to the candidate prompt words , format compliance score, coverage score, perform weighted operations to obtain the scores corresponding to the candidate prompt words, where .

[0026] As a preferred technical solution of the present invention, the preset prompt word variation mechanism in step E includes at least one of the following:

[0027] Prompt word variant mechanism a. Genetic algorithm crossover, which performs crossover and recombination on the components in the scored and screened prompt words to generate prompt word variants;

[0028] b. Online learning mechanism: Based on the manually annotated fields of scoring and filtering prompts corresponding to the standard text of each network protocol, a large language model is applied to generate constraints corresponding to the scoring and filtering prompts, and then combined with the scoring and filtering prompts to form prompt variants;

[0029] Prompt word variant mechanism c. Multi-protocol adaptation: Based on the protocol message formats of different protocols, preset protocol identification instructions including protocol type and corresponding operation extraction words are added to the scoring and screening prompt words to form prompt word variants. Preset grammar-aware instructions are also added to the scoring and screening prompt words to adjust field parsing rules to form prompt word variants.

[0030] Prompt word variant mechanism d. Multimodal prompt fusion. If there are diagrams in the network protocol specification text, then generate diagram description instructions and text and diagram cross-validation instructions to form each prompt word variant;

[0031] Prompt word variant mechanism e. Adversarial optimization, introducing noise or fuzzy instructions for scoring and filtering prompt words to form prompt word variants.

[0032] Corresponding to the above, the technical problem that the present invention also needs to solve is to provide an application of a network protocol intelligent extraction method based on a large language model to optimize the obtained prompt words, further deepen the application of the large language model, and improve the extraction efficiency of the network protocol text.

[0033] In order to solve the above technical problems, the present invention adopts the following technical solutions: The present invention designs an application of a network protocol intelligent extraction method based on a large language model, performing the following steps To step , realize the extraction of target network protocol text;

[0034] step . Pre-process and update the target network protocol text, and then enter step ;

[0035] step Input the target network protocol text and each target prompt word into the large language model, and the large language model will parse the target network protocol text according to each target prompt word to extract the protocol message format and protocol interaction process, and then enter step ;

[0036] step Locate the location and scope of the graph description in the target network protocol text, apply the large language model to perform structured analysis and extraction on the graph description, obtain the graph extraction result, and then enter step ;

[0037] step . Supplement the chart extraction results into the protocol message format and protocol interaction process, and integrate them into the target output format for output to complete the extraction of the target network protocol text.

[0038] As a preferred technical solution of the present invention, the steps The pre-processing and updating of the target network protocol text includes: firstly removing irrelevant header information, directory, appendix, and redundant characters in the target network protocol text to realize the cleaning and updating of the target network protocol text; then dividing the target network protocol text according to the preset text block size to obtain the text blocks corresponding to the target network protocol text;

[0039] The steps In the process, for each text block corresponding to the target network protocol text, the text block and each target prompt word are input into the large language model, and the large language model parses and extracts the text block according to each target prompt word to obtain the extraction result, and then obtains the extraction result corresponding to each text block in the target network protocol text, forming the protocol message format and protocol interaction process in the target network protocol text;

[0040] The steps In the process, for each text block corresponding to the target network protocol text, the position and range of the chart description are located, and the large language model is applied to perform structured analysis and extraction on the chart description to obtain the chart extraction result.

[0041] As a preferred technical solution of the present invention: the steps In the process, the chart description position and range are first located based on heuristic rules, and the field offset is matched by regular expressions. Then, a large language model is applied to perform structured parsing and extraction on the chart description to obtain the chart extraction result.

[0042] The steps In the process of supplementing the chart extraction results to the protocol message format and protocol interaction process, if the same field is described in both the text and the chart, the large language model is applied to deduplicate and complete the missing information to achieve enhanced understanding of the corresponding content.

[0043] As a preferred technical solution of the present invention: the steps After adding the chart extraction results to the protocol message format and protocol interaction process, error handling including conflict detection and processing, self-consistency checking, and default value completion is performed on the protocol message format and protocol interaction process, and finally integrated into the target output format for output to complete the extraction of the target network protocol text.

[0044] Corresponding to the above, the present invention also designs an electronic device, including a memory, a processor and a computer program stored in the memory, wherein when the processor executes the program, steps A to E in the prompt word optimization method are implemented, and steps To step .

[0045] The method and application of intelligent network protocol extraction based on a large language model described in the present invention, using the above technical solution, has the following technical effects compared with the existing technology:

[0046] The present invention designs a network protocol intelligent extraction method and application based on a large language model. Taking the network protocol specification text as the analysis object, each candidate prompt word is initialized, and a large language model is constructed to implement scoring considerations of three indicators: accuracy, format compliance, and coverage in the network protocol specification text extraction application based on the candidate prompt words. By using prompt word variants under a preset prompt word variant mechanism in a cyclic iteration, each target prompt word is optimized and obtained. Then, based on each target prompt word, the analysis and extraction of the chart description is deepened. The large language model is applied to achieve efficient and accurate acquisition of the protocol message format and protocol interaction process of the target network protocol text. Compared with traditional methods, the design scheme has obvious advantages in accuracy, generalization ability, information utilization, and processing of complex protocols, providing a more efficient and reliable approach for extracting protocol specifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a schematic diagram of the architecture of a network protocol intelligent extraction method based on a large language model and its application designed by the present invention;

[0048] Figure 2 This is an example of the message format presented in ASCII chart in the RFC of the embodiment. DETAILED DESCRIPTION

[0049] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0050] The large language model itself has strong semantic understanding capabilities, performs well in natural language understanding, and can extract structured information from complex texts. Therefore, without the need for other tools, the large language model can be used through sophisticated prompt engineering to parse communication protocol specifications and extract the protocol's message format and interaction process.

[0051] Specifically, the present invention designs a network protocol intelligent extraction method and application based on a large language model. In practical applications, such as Figure 1 As shown, first perform the following steps A to E to implement the prompt word optimization method to obtain the target prompt words extracted from the network protocol text corresponding to the large language model.

[0052] Step A. Initialize the number of iterations Based on the preset number of network protocol specification texts, the large language model is initialized to extract the corresponding prompt words of the protocol message format and protocol interaction process in each network protocol specification text, forming the first Iterate each candidate prompt word and then go to step B.

[0053] Regarding the design of initializing each prompt word in step A above, in the specific design, a large language model (LLM) is used to directly generate or optimize multiple prompt variants to generate a variety of initial candidate prompt words to cover possible extraction requirements, such as:

[0054] Variant 1: List all message fields and their bit lengths;

[0055] Variant 2: Returns the field name, offset, and description in a tabular format.

[0056] Variant 3: Generates JSON containing the field name, bits, and constraints.

[0057] In actual applications, based on a preset number of network protocol specification texts, the large language model is initialized in the following ways to extract the corresponding prompt words of the protocol message format and protocol interaction process in each network protocol specification text.

[0058] Method a. Zero-sample prompting, in the order of coarse-grained extraction to fine-grained extraction, directly requires the large language model to generate various prompt words for extracting the protocol message format from the network protocol specification text.

[0059] In the application, coarse-grained extraction is first performed to identify the macro structure of the message, such as listing all components of the message (such as Header, Payload, and Trailer); then fine-grained extraction is performed to extract each part field by field, such as parsing the fields in the Header part, including the field name, offset (bits / bytes), and whether it is optional.

[0060] Method b: Few-sample prompts: This method uses examples from the network protocol specification text as a reference and requires the large language model to extract the prompt words of the protocol message format from the network protocol specification text. In other words, it guides the large language model with examples to avoid parsing difficulties caused by free text. In practical applications, for example, if the current protocol is TCP, the following example is shown.

[0061] "Refer to the following TCP field example:

[0062] name:"ACK Number", bits:32, description:"Confirmation Number"

[0063] Please extract the fields of the current protocol similarly."

[0064] Method c. Task decomposition and combination: First, parse the specification text of each network protocol, extract all state names, and based on each state name, list all conversion events to constitute each subtask. Then, build a large language model to extract prompts for each subtask from the specification text of each network protocol, and combine them to form prompt words.

[0065] Step d. Hierarchical message extraction: If the network protocol specification text contains nested or hierarchical message structures, a large language model is constructed to recursively extract the corresponding prompt words for the nested fields in the protocol message format from the network protocol specification text. For example, in the example: "Recursively extract all nested fields in the message until the bottom-level basic types (such as int, string)."

[0066] Step e. Apply the preset large language model to extract each prompt word corresponding to the network protocol specification text, such as the example: name: "QNAME", bits: variable length, description: "DNS query domain name", please extract in the same style.

[0067] In practical applications, take RFC documents as an example. RFC (Request for Comments) is a standardized document that describes network protocols. Its content includes natural language text, diagrams, state machine descriptions, etc. The protocol message format in RFC usually includes the following:

[0068] Data packet header / tail structure: field name, length, value (such as the sequence number and flag bit of the TCP header).

[0069] Field semantics: For example, SYN=1 indicates a connection request.

[0070] Encoding rules: such as Big-Endian and variable-length fields.

[0071] Sample data packet: An example in hexadecimal or ASCII form (such as 0x00 0x01 0x02).

[0072] The protocol interaction process in RFC usually includes the following:

[0073] States: such as TCP's SYN_SENT and ESTABLISHED.

[0074] Events: such as receiving SYN and sending ACK.

[0075] Transitions: For example, SYN_SENT → (SYN+ACK received) → ESTABLISHED.

[0076] Step B. Construct a scoring function based on accuracy, format compliance, and coverage to obtain the first The scores of each candidate prompt word in the iteration are respectively corresponding to each candidate prompt word, and the candidate prompt words are sorted from large to small based on the scores. A preset proportion of candidate prompt words are selected before the order is filtered as the first Each scoring filter prompt word is iterated, and then enters step C.

[0077] In actual implementation, the above step B is respectively for For each candidate prompt word in the iteration, the specific design is to perform the following steps B1 to B2 to obtain the score corresponding to the candidate prompt word.

[0078] Step B1. Regarding accuracy, the large language model first extracts each field from each network protocol specification text based on the candidate prompt words, constructs a first set, and obtains the number of fields in the first set. At the same time, based on the manually annotated fields of the candidate prompt words corresponding to the network protocol specification text, a second set is constructed, and the number of fields in the second set is obtained. ; Then determine the number of fields in the intersection of the first set and the second set , and calculate the accuracy , recall rate ; Finally press , get the accuracy score corresponding to the candidate prompt word .

[0079] Regarding format compliance, the large language model uses candidate prompt words to extract each field from each network protocol specification text, and determines whether the format of each field is legal. In practical applications, for example, if the target field format is JSON or CSV, it is determined whether both field formats are legal. If so, the format compliance score corresponding to the candidate prompt word is 1; otherwise, the format compliance score corresponding to the candidate prompt word is 0.

[0080] Regarding coverage, recall The coverage score corresponding to the candidate prompt word.

[0081] Step B2. Calculate the preset weights for accuracy, format compliance, and coverage. 、 、 , the accuracy scores corresponding to the candidate prompt words , format compliance score, coverage score, perform weighted operations to obtain the scores corresponding to the candidate prompt words, where , in practical applications, such as design weights 、 、 =0.2.

[0082] Regarding the selection of scoring and screening prompt words in the above step B, in actual application, considering diversity protection, we can first remove duplicates for each candidate prompt word by semantic similarity (such as BERT embedding cosine distance), update the ranking of each candidate prompt word, ensure that each candidate prompt word covers different strategies, and then preset a proportion of candidate prompt words before screening the order as the first Each scoring filter prompt word in the iteration.

[0083] Before filtering, a preset number of candidate prompt words are used as the first Each scoring filter prompt word in the iteration,

[0084] Step C. Judgment Is it greater than or equal to the preset number of evaluation iterations? If yes, go to step D; otherwise, go to step E.

[0085] Step D. Obtain Iterations to The highest score in each iteration in the order of the iterations is used to determine whether the increase in the highest score is less than the preset stable increase threshold. If so, the In the iteration, each scored and filtered prompt word constitutes the target prompt word extracted by the large language model corresponding to the network protocol text; otherwise, go to step E.

[0086] Step E. According to at least one of the following preset prompt word variation mechanisms, In the iteration, each scoring filter prompt word is processed by the large language module to obtain the Each hint word variant under the iteration, combined with the Each scoring filter prompt word in the iteration is updated to form the +1 iteration for each candidate prompt word, and then for The value of is updated by adding 1 and returning to step B.

[0087] Prompt word variant mechanism a. Genetic algorithm crossover, which performs crossover and recombination on the components (such as format description and examples) in the scoring and screening prompt words to generate prompt word variants.

[0088] Prompt word variation mechanism b. Online learning mechanism: Based on the manually annotated fields of scoring and filtering prompt words corresponding to each network protocol specification text, a large language model is applied to generate constraints corresponding to the scoring and filtering prompt words. These constraints are then combined to form prompt word variations. In actual implementation, examples of the constraints generated here are as follows:

[0089] Add constraints: such as "ignore appendices and references";

[0090] Explicit format: such as "output must be a Python list";

[0091] In context: For example, "If the text description is unclear, refer to the chart first."

[0092] Prompt word variant mechanism c. Multi-protocol adaptation: Based on the protocol message formats of different protocols, new preset protocol identification instructions are added to the scoring and filtering prompt words, including the protocol type and its corresponding operation extraction words, to form prompt word variants; and new preset grammar-aware instructions are added to the scoring and filtering prompt words to adjust the field parsing rules to form prompt word variants.

[0093] In practical applications of multi-protocol adaptation, examples of protocol identification instructions include: "If the current RFC describes the HTTP protocol, extract the Header field; if it is the CoAP protocol, extract the Options field"; examples of preset syntax-aware instructions include: "Note: Binary protocol fields are parsed in big-endian order, and text protocol fields are split by line."

[0094] Prompt word variant mechanism d. Multimodal prompt fusion. If there are diagrams in the network protocol specification text, diagram description instructions and text and diagram cross-validation instructions are generated to form each prompt word variant.

[0095] In the actual implementation of multimodal prompt fusion, examples of chart description instructions include: "First describe the content of the chart, and then extract the state machine based on the description." Examples of text and chart cross-validation instructions include: "If the text and chart conflict, the chart shall prevail and the differences shall be noted."

[0096] Prompt word variation mechanism e. Adversarial optimization introduces noise or fuzzy instructions for scoring and filtering prompt words to form prompt word variations. Examples of noise introduced here include: "Extract the state, even if the text description is incomplete" and examples of fuzzy instructions include: "Try to guess possible state transitions" to detect whether large language models are over-associative.

[0097] By executing the above steps A to E, the prompt word optimization method is implemented to obtain the target prompt words corresponding to the network protocol text extraction of the large language model, and then in practical applications, the target network protocol text can be extracted, and in implementation, such as Figure 1 As shown, the specific design performs the following steps To step , realize the extraction of target network protocol text;

[0098] step . Pre-process and update the target network protocol text, including first removing irrelevant header information, directory, appendix, and redundant characters in the target network protocol text to achieve the target network protocol text cleaning and updating; then segment the target network protocol text according to the preset text block size, obtain the text blocks corresponding to the target network protocol text, and enter step .

[0099] The above steps In practical applications, irrelevant header information in the target network protocol text is removed, including the RFC header containing the author's name, page number, publication year, and tracking information, as it does not contribute to the protocol definition; regarding the removal of the table of contents, it is removed because experiments have found that the table of contents will cause LLM to misjudge; in addition, other content is cleaned up, specifically removing references, appendices, and unnecessary line breaks and whitespace characters.

[0100] The above steps The target network protocol text is segmented in the process because according to the context length limit of the large language model, the cleaned target network protocol text is segmented into 500 lines per text to adapt to the maximum context length limit of the large language model. In addition, the text block contains both plain text and text graphs for subsequent finite state machine extraction.

[0101] step . For each text block corresponding to the target network protocol text, the text block and each target prompt word obtained according to the above steps A to E are input into the large language model. The large language model parses and extracts the text block according to each target prompt word to obtain the extraction result, and then obtains the extraction result corresponding to each text block in the target network protocol text, forming the protocol message format and protocol interaction process in the target network protocol text, and then enters step .

[0102] In actual application, since the message format usually appears in the RFC text such as the Packet Format section, the Message Structure section, the Header Fields section, and the section containing the chart (ASCII or table), the above steps In actual applications, the large language model parses and extracts the above-mentioned chapters of the text block based on the target prompt words, and in the operation of the large language model here, examples of questions asked to the large language model are shown below.

[0103] Please extract the detailed definition of the protocol message format from the following RFC text and output it in a structured manner according to the following requirements:

[0104] 1. Field extraction:

[0105] -List all message header or message body field names (such as "Source Port", "Sequence Number").

[0106] - Label the data type of each field (such as uint8, uint16, string, flags).

[0107] -Specify the offset and length of the field in bits or bytes.

[0108] - If the field length is variable, indicate the dependent conditions (e.g. "The value of the Length field determines the length of the Data").

[0109] 2. Format specification:

[0110] -The output is in JSON format and contains the following keys.

[0111] -field_name:Field name (string).

[0112] -type: data type (string).

[0113] -offset: offset (number + unit, such as "4 bits").

[0114] -length: length (number + unit or "variable").

[0115] -description: A brief description of the field's function (optional).

[0116] - Ignore any reserved or padding fields.

[0117] 3. Context processing:

[0118] -If field definitions are scattered across multiple sections, merge all definitions and remove duplicates.

[0119] - If there is a conflict between the text description and a figure, the figure shall prevail and be marked "Conflict Resolution: Based on Figure X".

[0120] Corresponding to the above example, the actual output example from the large language model is as follows.

[0121] {

[0122] "field_name":"Source Port",

[0123] "type":"uint16",

[0124] "offset":"0 bytes",

[0125] "length":"2 bytes",

[0126] "description": "Source port number"

[0127] }

[0128] {

[0129] "field_name":"Content-Type",

[0130] "type":"string",

[0131] "offset":"variable",

[0132] "length":"variable",

[0133] "description": "Media type identifier"

[0134] }”

[0135] Examples of constraints involved in applications include: "a SYN flag bit being 1 indicates a connection request."

[0136] In practical applications such as RFC, message formats are often presented in ASCII graphics, such as Figure 2 shown.

[0137] Further introduce the following steps design and application.

[0138] step For each text block corresponding to the target network protocol text, first locate the position and range of the chart description based on heuristic rules (such as +---+, |, etc.), match the field offset through regular expressions, and then apply the large language model to perform structured parsing and extraction on the chart description to obtain the chart extraction result, and then enter step .

[0139] In practical applications, for example, a large language model can be asked a question, parsed, and returned in JSON format by field name, starting bit, and length (in bits). The following is an example of the large language model output.

[0140] "{

[0141] "header_format":[

[0142] {"name":"Source Port","start_bit":8,"length":16},

[0143] {"name":"Destination Port","start_bit":16,"length":16} ]

[0145] }”

[0146] step The chart extraction results are supplemented into the protocol message format and protocol interaction process. If the same field is described in both the text and the chart, the large language model is applied to remove duplicates and complete missing information to achieve enhanced understanding of the corresponding content. For example, if the text description is: "The sequence number field occupies 32 bits and is used for packet sorting" and the chart description is: |Sequence Number (32 bits)|, the output example after processing by the large language model is as follows.

[0147] "{

[0148] "name":"Sequence Number",

[0149] "bits":32,

[0150] "description": "Used for packet sorting"

[0151] }”

[0152] Then, error handling including conflict detection and processing, self-consistency checking, and default value completion is performed for the protocol message format and protocol interaction process. Finally, the protocol message format and protocol interaction process are integrated into the target output format for output, completing the extraction of the target network protocol text.

[0153] In practical applications, conflict detection and processing can be done for example: if a diagram conflicts with the text description (e.g., length inconsistency), the conflict is noted and the most likely correct value is selected based on the context; or if the description of the same field in the text and diagram is inconsistent (e.g., length conflict), the Large Language Model (LLM) can flag the issue as shown in the following example.

[0154] "Conflict detected:

[0155] -Text description: "The Flags field occupies 8 bits"

[0156] -Chart description: "The Flags field occupies 6 bits"

[0157] Please confirm the correct value."

[0158] Regarding self-consistency verification, that is, letting the large language model self-verify the extraction results, such as the following example.

[0159] "Check that the following state machine is self-consistent:

[0160] Is there isolation?

[0161] -Do all transitions have clear triggering events?"

[0162] Regarding default value completion, that is, for fields that are not explicitly described (such as reserved bits), the Large Language Model (LLM) can infer default values, such as the example: "Reserved fields should be 0 unless otherwise specified."

[0163] Finally, regarding "applying a large language model to perform structured parsing and extraction on the chart description to obtain the chart extraction result", an example is shown below.

[0164] (1) Output result field alignment requirements:

[0165] Explicitly specify the output format (such as JSON) to avoid free text that may cause the output results to be too different and difficult to integrate. The example is as follows.

[0166] "Return field information strictly in the following JSON format:

[0167] {

[0168] "fields":[

[0169] {"name":"field name","bits":length,"description":"semantics"} ]

[0171] }”

[0172] (2) Results integration:

[0173] The JSON results extracted from each RFC block are integrated together in sequence. For convenience and intuitiveness, the large model can output the message format in a tabular form. The example is as follows.

[0174] RFC 3550, titled "RTP: Real-Time Application Transport Protocol," defines the Real-time Transport Protocol (RTP), which is widely used to transmit audio and video over IP networks. RTP provides end-to-end network transport capabilities suitable for applications that transmit real-time data, such as interactive audio and video conferencing.

[0175] The key components of RTP include:

[0176] 1.RTP header format: defines the structure of each RTP data packet.

[0177] 2. RTCP (RTP Control Protocol): Provides feedback about the quality of data distribution and helps synchronize media streams.

[0178] 3. Packetization: defines how to encapsulate media into RTP packets.

[0179] 4. Timing and synchronization: Ensure the correct playback time of the media stream.

[0180] An example of breaking down the important formats, including fields, field lengths, and their meanings, follows.

[0181] RTP header format,The RTP header format is defined in Section 5.1 of RFC 3550. The RTP header has a fixed part, which can be followed by an optional CSRC (Contributing Source) identifier.

[0182] The RTP fixed header fields are shown in Table 1 below.

[0183] Table 1 RTP fixed header fields

[0184]

[0185] An example of the message interaction process of the simultaneous output protocol is as follows.

[0186] Common RTCP packet types are shown in Table 2 below.

[0187] Table 2 Common RTCP packet types

[0188]

[0189] The message interaction process is as follows.

[0190] 1. Session initialization:

[0191] Participants join an RTP session and exchange initial RTCP packets (usually SR or RR).

[0192] 2. Data transmission:

[0193] RTP packets containing media payload are transmitted periodically.

[0194] Each participant sends RTCP reports (SR or RR) periodically to provide feedback on the quality of the session.

[0195] 3. Source Identification:

[0196] SDES packets are exchanged to provide a textual description of the source.

[0197] 4. End of participation:

[0198] When a participant leaves a session, it sends a BYE packet to notify other participants.

[0199] 5. Error Monitoring:

[0200] The receiver uses the SR / RR packets to calculate the packet loss rate and jitter and adjust the playout buffer accordingly.

[0201] In practical applications of the above design scheme, an electronic device is further designed, including a memory, a processor and a computer program stored in the memory. When the processor executes the program, steps A to E in the prompt word optimization method are implemented, as well as steps in the application. To step .

[0202] The method and application of optimizing prompt words for network protocol extraction based on a large language model described in the present invention have the following technical effects compared with the prior art:

[0203] The present invention designs a network protocol intelligent extraction method and application based on a large language model. Taking the network protocol specification text as the analysis object, each candidate prompt word is initialized, and a large language model is constructed to implement scoring considerations of three indicators: accuracy, format compliance, and coverage in the network protocol specification text extraction application based on the candidate prompt words. By using prompt word variants under a preset prompt word variant mechanism in a cyclic iteration, each target prompt word is optimized and obtained. Then, based on each target prompt word, the analysis and extraction of the chart description is deepened. The large language model is applied to achieve efficient and accurate acquisition of the protocol message format and protocol interaction process of the target network protocol text. Compared with traditional methods, the design scheme has obvious advantages in accuracy, generalization ability, information utilization, and processing of complex protocols, providing a more efficient and reliable approach for extracting protocol specifications.

[0204] Higher Accuracy: Traditional methods, such as those based on semantic parsing and network tracking message analysis, are susceptible to factors such as text ambiguity and network environment complexity, resulting in problems such as partial extraction and inaccurate probabilities in the extracted protocol FSM. This invention utilizes a large model combined with text and graphs for extraction, enabling more accurate extraction of protocol state and transition information.

[0205] Stronger generalization: Most traditional methods are highly targeted, relying on RFC documents in specific formats, predefined FSM grammars, or specific programming environments, and are less adaptable to different types of protocols. The method proposed in this paper does not rely on predefined state machine grammars. The large language model has been pre-trained on text from various domains and can understand most RFC formats. It can better generalize to a variety of different RFC documents and has a wider range of applicability.

[0206] Low requirement for manually annotated data: The proposed method requires less manually annotated pre-training data. Traditional deep learning-based information extraction methods rely on large amounts of high-quality annotated data, which is difficult and expensive to obtain in the network protocol field. This method uses prompt engineering technology to strategically optimize prompts rather than relying on large amounts of annotated data to train the model. Therefore, compared with traditional methods, the proposed method significantly reduces the need for manually annotated pre-training data when extracting protocol specifications.

[0207] Reducing human intervention: The design of this invention reduces the reliance on human intervention to a certain extent through a large language model and automated prompt engineering, but has made significant progress in reducing human intervention compared to traditional methods.

[0208] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by ordinary technicians in this field without departing from the spirit of the present invention.

Claims

1. A network protocol intelligent extraction method based on a large language model, characterized in that: Execute the following steps A to E to obtain target prompt words extracted from the network protocol text by the large language model; Step A. Initialize the number of iterations Based on the preset number of network protocol specification texts, the large language model is initialized to extract the corresponding prompt words of the protocol message format and protocol interaction process in each network protocol specification text, forming the first Each candidate prompt word in the iteration, and then enter step B; Step B. Construct a scoring function based on accuracy, format compliance, and coverage to obtain the first The scores of each candidate prompt word in the iteration are respectively corresponding to each candidate prompt word, and the candidate prompt words are sorted from large to small based on the scores. A preset proportion of candidate prompt words are selected before the order is filtered as the first Each scoring filter prompt word in the iteration, and then enter step C; Step C. Judgment Is it greater than or equal to the preset number of evaluation iterations? If yes, go to step D; otherwise go to step E; Step D. Obtain Iterations to The highest score in each iteration in the order of the iterations is used to determine whether the increase in the highest score is less than the preset stable increase threshold. If so, the Each scored and filtered prompt word in the iteration constitutes the target prompt words extracted by the large language model corresponding to the network protocol text; Otherwise, go to step E; Step E. According to the preset prompt word variation mechanism, In the iteration, each scoring filter prompt word is processed by the large language module to obtain the Each prompt word variant under the iteration, combined with the Each scoring filter prompt word in the iteration is updated to form the +1 iteration for each candidate prompt word, and then for The value of is updated by adding 1 and returning to step B.

2. The method for intelligently extracting network protocols based on a large language model according to claim 1, characterized in that: In step A, the large language model is initialized to extract the prompt words of the protocol message format and the protocol interaction process in the specification text of each network protocol in the following manners; Method a: Zero-shot prompting, in the order of coarse-grained extraction to fine-grained extraction, directly requires the large language model to generate various prompt words for extracting the protocol message format from the network protocol specification text; Method b. Few-sample prompts: This is constructed based on examples in the network protocol specification texts. The large language model is required to extract the prompt words of the protocol message format from the network protocol specification texts. Method c. Task decomposition and combination: First, parse the network protocol specification text, extract all state names, and based on each state name, list all transition events to form each subtask. Then, build a large language model to extract prompts for each subtask from the network protocol specification text and combine them to form prompt words. Step d. Hierarchical message extraction: if the network protocol specification text has a nested or hierarchical message structure, a large language model is constructed to recursively extract the various prompt words of the nested fields in the protocol message format from the network protocol specification text; Step e. Applying a preset large language model to extract each prompt word corresponding to the network protocol specification text.

3. The method for intelligently extracting network protocols based on a large language model according to claim 1, characterized in that: In the step B, respectively For each candidate prompt word in the iteration, follow steps B1 to B2 to obtain the score corresponding to the candidate prompt word; Step B1. Regarding accuracy, the large language model first extracts each field from each network protocol specification text based on the candidate prompt words, constructs a first set, and obtains the number of fields in the first set. At the same time, based on the manually annotated fields of the candidate prompt words corresponding to the network protocol specification text, a second set is constructed, and the number of fields in the second set is obtained ; Then determine the number of fields in the intersection of the first set and the second set , and calculate the accuracy , recall rate ; Finally press , get the accuracy score corresponding to the candidate prompt word ; Regarding format compliance, the large language model uses candidate prompts to extract each field from each network protocol specification text and determines whether the format of each field is legal. If so, the format compliance score corresponding to the candidate prompt is 1; otherwise, the format compliance score corresponding to the candidate prompt is 0. Regarding coverage, recall The coverage score corresponding to the candidate prompt words; Step B2. Calculate the preset weights for accuracy, format compliance, and coverage. 、 、 , the accuracy scores corresponding to the candidate prompt words , format compliance score, coverage score, perform weighted operations to obtain the scores corresponding to the candidate prompt words, where .

4. The method for intelligently extracting network protocols based on a large language model according to claim 1, characterized in that: The preset prompt word variation mechanism in step E includes at least one of the following: Prompt word variant mechanism a. Genetic algorithm crossover, which performs crossover and recombination on the components in the scored and screened prompt words to generate prompt word variants; b. Online learning mechanism: Based on the manually annotated fields of scoring and filtering prompts corresponding to the standard text of each network protocol, a large language model is applied to generate constraints corresponding to the scoring and filtering prompts, and then combined with the scoring and filtering prompts to form prompt variants; Prompt word variant mechanism c. Multi-protocol adaptation: Based on the protocol message formats of different protocols, preset protocol identification instructions including protocol type and corresponding operation extraction words are added to the scoring and screening prompt words to form prompt word variants. Preset grammar-aware instructions are also added to the scoring and screening prompt words to adjust field parsing rules to form prompt word variants. Prompt word variant mechanism d. Multimodal prompt fusion. If there are diagrams in the network protocol specification text, then generate diagram description instructions and text and diagram cross-validation instructions to form each prompt word variant; Prompt word variant mechanism e. Adversarial optimization, introducing noise or fuzzy instructions for scoring and filtering prompt words to form prompt word variants.

5. Application of the network protocol intelligent extraction method based on a large language model according to any one of claims 1 to 4, characterized in that: Perform the following steps To step , realize the extraction of target network protocol text; step . Pre-process and update the target network protocol text, and then enter step ; step Input the target network protocol text and each target prompt word into the large language model, and the large language model will parse the target network protocol text according to each target prompt word to extract the protocol message format and protocol interaction process, and then enter step ; step Locate the location and scope of the chart description in the target network protocol text, apply the large language model to perform structured analysis and extraction on the chart description, obtain the chart extraction result, and then enter step ; step . Supplement the chart extraction results into the protocol message format and protocol interaction process, and integrate them into the target output format for output to complete the extraction of the target network protocol text.

6. The application of the network protocol intelligent extraction method based on a large language model according to claim 5, characterized in that: The steps The pre-processing and updating of the target network protocol text includes: firstly removing irrelevant header information, directory, appendix, and redundant characters in the target network protocol text to realize the cleaning and updating of the target network protocol text; then dividing the target network protocol text according to the preset text block size to obtain the text blocks corresponding to the target network protocol text; The steps In the process, for each text block corresponding to the target network protocol text, the text block and each target prompt word are input into the large language model, and the large language model parses and extracts the text block according to each target prompt word to obtain the extraction result, and then obtains the extraction result corresponding to each text block in the target network protocol text, forming the protocol message format and protocol interaction process in the target network protocol text; The steps In the process, for each text block corresponding to the target network protocol text, the position and range of the chart description are located, and the large language model is applied to perform structured analysis and extraction on the chart description to obtain the chart extraction result.

7. The application of the network protocol intelligent extraction method based on a large language model according to claim 5, characterized in that: The steps In the process, the chart description position and range are first located based on heuristic rules, and the field offset is matched by regular expressions. Then, a large language model is applied to perform structured parsing and extraction on the chart description to obtain the chart extraction result. The steps In the process of supplementing the chart extraction results to the protocol message format and protocol interaction process, if the same field is described in both the text and the chart, the large language model is applied to deduplicate and complete the missing information to achieve enhanced understanding of the corresponding content.

8. The application of the network protocol intelligent extraction method based on a large language model according to claim 7, characterized in that: The steps After adding the chart extraction results to the protocol message format and protocol interaction process, error handling including conflict detection and processing, self-consistency checking, and default value completion is performed on the protocol message format and protocol interaction process, and finally integrated into the target output format for output to complete the extraction of the target network protocol text.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the processor executes the program, steps A to E of the prompt word optimization method according to any one of claims 1 to 4 are implemented, and steps To step .

Images