Large language model assisted security protocol symbol model generation method and device

Abstract

The application relates to a large language model assisted security protocol symbol model generation method and device, which comprises the following steps: a natural language document is parsed into a parsed result represented by a domain specific language through a semantic parser, and the parsed result is taken as an intermediate representation, wherein the semantic parser is constructed based on a large language model driven combined category grammar; problems existing in the intermediate representation are repaired through static analysis and user interaction to normalize the intermediate representation in form; the normalized intermediate representation is converted into a Sapic+ specification through a converter; the normalized Sapic+ process language is compiled into a symbol model directly accepted by a verification tool through a compiler; and transformation correctness between different models is proved. The application proposes a method capable of extracting a protocol symbol model from a natural language description, and a user-friendly synthesis tool is realized based on the method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of security protocol, and particularly relates to a large language model assisted security protocol symbol model generation method and device. BACKGROUND

[0002] Security protocols are programs designed to protect secure network (e.g. Internet) communication through cryptographic primitives. Security protocols are ubiquitous, being used in security-critical scenarios such as electronic commerce (e.g. the TLS protocol used in https: / / URLs), bank transactions, mobile communications, WiFi networks (e.g. the WPA2 encryption protocol), etc. However, security protocols are prone to design errors (e.g. the famous Needham-Schroeder public-key protocol), and these security errors can have serious consequences, so it is crucial to guarantee the security and correctness of protocol design.

[0003] Common software testing methods often cannot detect security errors, because they only appear in the presence of malicious attacks. For example, protocol fuzzing techniques can often find implementation errors in memory, but it is very difficult to find logical errors at the design level. Formal verification is a powerful tool to solve this problem, especially protocol conformance model analysis, which is very important to guarantee the correctness of protocol design. It can rigorously prove that this class of protocols is immune to specific attacks at the design level, and can also be used to find potential hidden attack vectors. For decades, the verification of symbolic models of security protocols has been a hot research topic, and various automated tools (ProVerif, Tamarin Prover, etc.) have emerged to help protocol developers verify protocols. Among them, the automated tools ProVerif and Tamarin Prover have achieved great success in practical cases. Taking the AKA authentication protocol of 5G communication as an example, Basin et al. extracted precise requirements from the 3GPP standard and identified missing security goals. Using the security protocol verification tool Tamarin, they conducted a comprehensive and systematic security evaluation of the model for the 5G security goals. Through automated analysis, the minimum security assumptions required to achieve each security goal were determined, and it was found that some key security goals could not be achieved.

[0004] The formal verification of protocols generally has the following steps:

[0005] 1. The protocol developer or expert manually models the specific implementation of the protocol and the security properties to be verified by reading the protocol document (such as IETF Request For Comments, RFC).

[0006] 2. The established model is described using formal languages of verification tools (such as Tamarin Prover and ProVerif), and the model is converted into an input form that the verification tool can receive.

[0007] 3. The security properties (such as privacy, data integrity, authentication) that the protocol needs to satisfy are modeled using logical formulas (such as First-Order Logic, FOL).

[0008] 4. The verification tool (semi-)automatically constructs a formal proof that the protocol model satisfies the security properties. It can be seen that protocol verification relies heavily on manual specification extraction and proof construction.

[0009] Different protocol verification tools use different input languages. Although each tool has its own advantages and disadvantages, they have one thing in common from the perspective of usability: the input language is difficult for non-expert users. In symbolic models, cryptographic primitives are considered perfect and are modeled by function symbols in term algebra (such as the function senc representing symmetric encryption), whose semantics are given by a series of equations. Messages are terms based on these cryptographic primitives, and an attacker can only symbolically compute new terms using these primitives (and the equations modeling them). In particular, an attacker can only decrypt encrypted terms if he has the corresponding key. The default attack model is the Dolev-Yao model, in which the attacker is assumed to have complete control over the network: he can listen to, delete, replace, duplicate, and delay messages sent by both parties, and he can insert his own selected messages on the public channel.

[0010] In addition to the complexity of using the tool itself, security protocols involve many fields (such as access control, encryption authentication, etc.), and a practical security protocol is often specific to a particular field, i.e., different security protocols involve different domain knowledge and require different modeling choices. These problems all increase the cost of protocol verification, making the threshold of protocol verification high. For the analysis of protocols using tools, reading protocol documents and modeling are the starting point of the entire security protocol analysis (as shown in Figure 1 Therefore, if the entire modeling and verification process can be automated from the protocol text, it will greatly reduce the threshold for using protocol analysis tools and will have a very important significance for the verification and deployment of security protocols.

[0011] There have been many works in the field of software engineering that extract specifications or even generate code from natural language texts. For example, Jdoctor, C2S, and the like use a word alignment method to process Javadoc documents to generate corresponding executable specifications, thereby guiding the generation of program test cases. Tellina, TreeGen, and the like use an end-to-end machine translation model to generate bash commands or Python code from natural language descriptions. In recent years, the emergence and rapid development of large language models have made these tasks easier. Because of the rich source of training data, large language models have achieved good performance on these tasks and can generate high-quality programs with obvious structure and not very complex semantics. However, for protocols, the input documents and corresponding verifiable symbolic models are difficult to align, which leads to a lack of high-quality data to train or fine-tune large language models. In addition, since the synthesized results are used for formal verification, it is also crucial to ensure or explain the correctness of the synthesized results. Therefore, simply modeling the protocol symbolic model synthesis task as an end-to-end code generation problem is not suitable. In terms of effect, the quality of the synthesized results cannot be guaranteed, and in terms of correctness, the black-box working mechanism of the large language model cannot provide a guarantee for the reliability. SUMMARY

[0012] To solve the above technical problems, the present application provides a large language model assisted secure protocol symbolic model generation method and device.

[0013] The purpose of the present application is achieved by the following technical solutions:

[0014] In a first aspect, a large language model assisted secure protocol symbolic model generation method is provided, which includes the following steps:

[0015] A semantic parser is used to parse a natural language document based on a secure protocol into a parsed result represented in a domain-specific language, and the parsed result is used as an intermediate representation, wherein the semantic parser is based on a large language model driven combined category grammar construction;

[0016] Problems existing in the intermediate representation are repaired through static analysis and user interaction to formalize the intermediate representation;

[0017] A converter is used to convert the formalized intermediate representation into a Sapic+ specification;

[0018] A compiler is used to compile the standardized Sapic+ process language into a symbolic model directly accepted by verification tools Tamarin Prover, DeepSec and ProVerif;

[0019] Prove the correctness of the transformation between different models.

[0020] In some embodiments, the domain-specific language is based on instantiating parameters of key predicates by concrete terms in the natural language document, and is simplified by the form of lambda expressions and defined with explicit syntax rules to constitute a protocol document.

[0021] The syntax rules include combinations of event sequences, conditional control structures, and termination representations of the protocol.

[0022] In some embodiments, the semantic parser includes a parsing algorithm part for parsing the natural language document and a prompt word part for guiding the parsing algorithm to parse the natural language document.

[0023] In some embodiments, the parsing algorithm part includes: cutting the protocol document as input into multiple sub-blocks, moving a sliding window on the sub-blocks in turn, and using adjacent sub-blocks and corresponding parsing results as context to help the parsing of the next word block.

[0024] In some embodiments, the prompt word part includes a first part, a second part and a third part, wherein the first part is to specify the role for the large language model, and set the role as a parsing expert of a combined category syntax; the second part is to explain the designed domain-specific language in natural language, and inform the large language model of the syntax rules that need to be met; the third part is to add actual sample examples.

[0025] In some embodiments, the problems existing in the intermediate representation are repaired through static analysis and user interaction to formalize the intermediate representation, including:

[0026] First, the large language model verifies the consistency of the parsing result with the specification of the protocol document, and formats the unreadable message into a readable form; then, the unbound variables in the parsing result are analyzed, and a graphical message sequence chart of the protocol is generated; then, the user is allowed to correct in an interactive environment in combination with the unbound variable set and the graphical message sequence chart until there are no unbound variables in the parsing result, and finally a readable domain-specific language expression conforming to the protocol specification is generated, that is, a domain-specific language program, which is the intermediate representation.

[0027] In some embodiments, the formalized intermediate representation is converted into a Sapic+ specification by a converter, including:

[0028] First, a set of conversion rules are determined for converting the repaired domain-specific language program into local processes of each role in the protocol, and the conversion rules process each type of event in the domain-specific language program by recursive manner;

[0029] Then based on the signature of the local process, combined with the prompt of the large language model, through few-shot learning, the large language model is used to generate the top-level process of the protocol.

[0030] Finally, the local process and the top-level process are combined to form a complete Sapic+ specification.

[0031] In some embodiments, the correctness of the transformation between different models is proved, including:

[0032] First, the trajectory inclusion relationship between different models is established.

[0033] Then it is proved that any security property that passes the verification on the verification tool is also applicable to the domain-specific language model and the Sapic+ model.

[0034] The second aspect provides a large language model assisted security protocol symbolic model generation device, which comprises a memory and one or more processors, and the memory stores executable code, characterized in that when the processor executes the executable code, it is used to realize the large language model assisted security protocol symbolic model generation method as any one of the first aspect.

[0035] The beneficial effects of the present application are:

[0036] 1. The complexity of the protocol document may make the large language model lacking domain-specific knowledge more prone to errors, and the protocol document is often long (hundreds or even thousands of word tokens), which will affect the accuracy of the output of the large language model, and the existing large language model often pays less attention to the relevant information in the middle part of the input, resulting in insufficient use of the provided context. The semantic parser based on the large language model driven combination category grammar constructed by the present application can better process the protocol document input.

[0037] 2. The semantic characteristics of the intermediate representation of the existing work are not rich enough (such as only being able to represent the semantics of a single sentence, and being unable to represent the context information of the sentence), and the existing intermediate representation language is expanded or constructed, aiming to improve the accuracy of the extraction model and reduce the difficulty of subsequent conversion to the modeling language of the verification tool.

[0038] 3. The process from the intermediate representation to the modeling language of ProVerif, Tamarin Prover, etc. cannot be simply regarded as a language translation model, and an analysis-based conversion tool is needed to ensure that the semantics of the extracted protocol model before and after conversion do not change, and are equivalent and consistent. The present application proves the path inclusion relationship of the protocol model before and after conversion through the construction of manual proof.

[0039] 4. The application proposes a method for synthesizing a symbolic model from a protocol natural language document, which can automatically and correctly synthesize symbolic models of some classic security protocols and verify the success. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor under the premise of the drawings.

[0041] Figure 1 is a general flowchart of the large language model assisted security protocol symbolic model generation method provided by the present application in an embodiment;

[0042] Figure 2 is a structural schematic diagram of the large language model assisted security protocol symbolic model generation device provided by the present application in an embodiment. DETAILED DESCRIPTION

[0043] In order to better understand the technical solutions of the present application, the embodiments of the present application will be described in detail below with reference to the drawings.

[0044] As Figure 1 shown, in an embodiment, a large language model assisted security protocol symbolic model generation method is provided, which includes the following steps:

[0045] The natural language document based on the security protocol is parsed into a parsed result represented by a domain specific language by a semantic parser, and the parsed result is taken as an intermediate representation, wherein the semantic parser is based on a large language model driven combination category grammar construction;

[0046] In some embodiments, the present invention introduces a domain-specific language (DSL) for specifying security protocols in an intuitive way, making it easier to be generated by large language models (LLMs). Essentially, the designed DSL is a special kind of lambda expression. From the perspective of lambda calculus, it can be observed that a sentence in a protocol document is usually an application, i.e., instantiating the parameters of a key predicate by concrete terms (e.g., role names, explicit messages). Intuitively, a lambda expression describes some operations performed by a certain agent from a global perspective at a certain point in time. To simplify, an application sxy(λx, y.e(x, y) at) is abbreviated as e(a, t), which is called an event E. The syntax of the DSL is shown below, where e is the name of an event E, f and a are the names of a function and a subject, respectively, and t is a term in T: ⊥ represents a termination statement. P represents a program block, E; P represents a sequential expression defined recursively, condαP else Q represents a branching statement, where cond and else are keywords, α represents a judgment condition, P is a program executed when the α judgment is satisfied, and Q is a program executed when the α judgment is not satisfied. E represents time, ⊥ represents a termination statement, where A = {t∈T | t: agent} is a set of protocol entity names. e represents a symbol set, including gen (generate a random number), send (send a message), recv (receive a message), knows (initialize), and op (local operation).

[0047] P: := E; P | condαP else Q

[0048] E: := e(a, t) | ⊥, a∈A, t∈T

[0049] e: := gen | send | recv | knows | op

[0050] A type msg is defined for each term, i.e., for all terms t∈T, it has a type msg, further, two subtypes of msg are defined: terms representing protocol entity names are classified as agent, denoted as t: agent; terms representing newly generated random numbers are classified as nonnce, denoted as t: agent. The basic event set is represented as Ev g= E(A, T(N)), where A = {t e T | t : agent} is the set of protocol entity names. In this invention, only the symbol sets given in the above syntax are considered, which can be extended to other event symbols. The event gen(a, n) means that the protocol entity a generates a new random number n. The interaction events send(a, m) and recv(a, m) mean that the agent a sends and receives a message m over a public channel, respectively. The event know(a, t) means that the agent a knows the term t initially, i.e., t is in the initial knowledge set of a. The event op(a, t) is a special event, which means that the agent a performs some local operations, such as binding a term to a variable or declaring the secrecy of a locally generated random number.

[0051] In some embodiments, the semantic parser (LCCG) parses a natural language document into a designed domain-specific language.

[0052] CCG: Combinatory Categorial Grammar (CCG) is a rule system that combines syntax and semantics, which can take natural language sentences as input and output corresponding lambda expressions. Here, a large language model is used as a CCG parser, and since it has been pre-trained on a large amount of language data, including various protocol-related technical documents, the model itself already has the ability to understand the documents without manually constructing the semantics and syntax rules of the CCG system. In this part, the main design includes two parts: 1) parsing algorithm and 2) prompt design for parsing documents.

[0053] The invention constructs a parsing algorithm for slices. First, the input protocol document is divided into multiple sub-blocks, and a sliding window is used to move over the sub-blocks one by one. The adjacent sub-blocks and the corresponding parsing results are used as context to help the parsing of the next word block. In the prompt design, the design method of few-shot context learning is combined.

[0054] The prompt is divided into three parts: (1) specify the role for the large language model and set it as a CCG parsing expert; (2) explain the designed DSL in natural language and tell the large language model the syntax rules that need to be met; (3) add actual sample examples (few-shot) to take the parsing process of a small actual protocol as a case. In this way, the parsing process of the initial text input is completed, and the DSL program fragment that follows the syntax defined above is obtained.

[0055] Problems existing in the intermediate representation are fixed through static analysis and user interaction to formalize the intermediate representation;

[0056] In some embodiments, the model generated by the LCCG needs to be canonicalized and consistent with the natural language document before being converted to the Sapic+ specification.

[0057] However, the initial model often has problems for various reasons, the main problems include:

[0058] 1) Inconsistency: Due to the hallucination and randomness of large language models (LLMs), the initial model can be inconsistent with the natural language description. For example, in the elaboration of a certain protocol document, the server should respond to the client's request using the hash value hash(k) of the key, but the large language model sometimes generates incorrect messages. Similarly, messages that should be sent by role A may be wrongly attributed to role B by the LLM;

[0059] 2) Ambiguity: Since ambiguity is common in natural language documents, it is essential to adopt methods to reduce errors and ensure clarity. According to observations, large language models often map unclear natural language descriptions to incorrect expressions with unbound variables, i.e., variables are used but not declared;

[0060] 3) Unreadability: Natural language describes the behavior and messages of the protocol from a global perspective. However, the message in the "receive" event is presented from a local perspective, specifically showing what the receiver should expect. In the symbolic model of the protocol, the form of the incoming message should be readable to a specific role, i.e., variables should be accessible. In the protocol verifier, pattern matching is based on the form of the message. Although the verifier can still accept incoming messages containing inaccessible variables, this unexpected pattern matching may disrupt semantics.

[0061] To solve the above problems, a repair algorithm is proposed, as described in Algorithm 1 below. The input includes the natural language description N of the protocol and the parsing result P. To solve the inconsistency problem due to the hallucination and randomness of large language models, the invention introduces a self-verification method that guides the large language model to carefully review the specification of the given protocol description without introducing additional information, finding and correcting errors produced in the parsing stage. To solve the unreadability of the received message, a set of examples is encoded using the diff format;

[0062] Algorithm 1 Repairing parsing result:

[0063] Input: protocol document N, parsing result P

[0064] Output: repaired DSL expression P

[0065] 1, prompt the large language model (LLM) to verify the parsing result: P←Validate(P, N)

[0066] 2. LLM modifies the received message into readable form: P <- Readable(P)

[0067] 3. WHILE true DO

[0068] 4. Analyze the program to obtain the set of unbound variables: V <- Analyze(P, N)

[0069] 5. If the set of variables is empty:

[0070] 6. Return the set of variables: Return V

[0071] 7. Extract the message sequence chart of the protocol for interactive modification: msc <- Graphviz(P), where msc represents the message sequence chart and Graphviz represents the extraction process

[0072] 8. Allow the user to directly modify: P <- Interact(msc, P), where interact represents the interaction process;

[0073] 9. Finally, return the repaired program: Return P

[0074] The large language model is demonstrated step by step on how to transform the message into the correct form in the role specification step (line 2). When implementing the framework as a tool, a semi-automatic repair method is used (lines 4-9 of Algorithm 1). In this case, the user can directly modify the program P. The modification hints provided to the user include the graphical message sequence chart (MSC, see line 8) of the protocol and the unbound variable results from the analysis.

[0075] Convert the formally normalized intermediate representation to Sapic+ specification through a translator;

[0076] Sapic+ is a protocol verification platform that allows users to efficiently verify security protocols using three major protocol verification tools: ProVerif, Tamarin, and DeepSec. The core features of this platform include:

[0077] Unified input language: Sapic+ designs a common input language that can serve as input for multiple tools, including ProVerif, Tamarin, and DeepSec, thereby eliminating the need to choose a specific tool, making these tools more accessible and easier to use.

[0078] Automatic translation: Sapic+ can automatically translate protocol specifications into the input format of each tool, allowing users to verify the security properties of a protocol through a single Sapic+ file.

[0079] Correctness proof: Sapic+ proves the correctness of the translation process, meaning that the results proven using one tool can be used for another, allowing for the utilization of the respective strengths of each tool.

[0080] The goal of Sapic+ is to leverage the strengths of each tool to improve the efficiency and accessibility of protocol verification while maintaining consistency and correctness in the verification results.

[0081] Presented by Cheval et al. at the top security conference Usenix.

[0082] Cheval V, Jacomme C, Kremer S, et al. {SAPIC+}: protocol verifiers of the world, unite! [C] / / 31st USENIX Security Symposium (USENIX Security 22). 2022:3935-3952.

[0083] Link address: https: / / www.usenix.org / system / files / sec22-cheval.pdf

[0084] In some embodiments, a complete Sapic+ specification can be represented as: U r P r ∪P t , where r is the protocol entity name, P r represents its corresponding local process, and P t represents the top process, which is related to the initialization of the protocol (such as key configuration). In this invention, a set of transformation rules is first defined to convert the repaired DSL program P into a local process. The rules are defined as:

[0085]

[0086] The three parameters of T represent the program P, the role r, and the corresponding knowledge set M, respectively. Rule (1) represents rewriting the repaired program P into the Sapic+ process algebra, and the conversion result of the DSL program P is the mutually disjoint set of all role specifications extracted from P. Rule (2) represents rewriting the branch structure condαP else Q into ifαthen P in the process. The symbol

[0087] Rules (3)-(8) describe the process of deriving local processes from role specifications by recursively applying the transformation rules T. Specifically, rule (3) transforms knows events into the signature let r(t) = P of a local process, where r denotes the role name; rule (4) indicates that the event gen(r, n) statement in the program is rewritten into the new n statement in the process, where event gen(r, m) means that the protocol entity r generates a new random number; rule (5) indicates that the send(r, n) statement in the program is rewritten into the out(n) statement in the process, where event send(r, m) means that the principal r sends a message m through a public channel; rule (6) indicates that the recv(r, m) statement in the program is rewritten into the in(m) statement in the process, where recv(r, m) means that the principal r receives a message m through a public channel; rule (7) indicates that the op(r, f(t)) statement in the program is rewritten into the signal statement event f(t) in the process, where op(r, f(t)) is a special event, indicating that the principal r performs some local operation f(t), and f is the operation name.

[0088] Rules (5)-(6) define the message input / output in the process; rules (7)-(8) transform op events into corresponding let bindings and signal events. The rewriting rules are defined under the default assumptions. Specifically, the assumptions are as follows: (i) the threat model is the classic Dolev-Yao model, and the attacker can inject, modify, and intercept messages in the network; (ii) message communication between different roles is transmitted through a public channel in an asynchronous manner; (iii) the initial knowledge set of the role is empty. In order to obtain the complete Sapic+ reduction, it is also necessary to further synthesize the top-level process P t . First, collect the signatures of all local processes ∪ r P r , and then use the few-shot learning design to prompt the large language model to generate the top-level specification. Once the complete process U r P r ∪ P t is obtained, it can be compiled into the corresponding symbolic models of Tamarin Prover, DeepSec, and ProVerif using the compiler.

[0089] Compiling the standardized Sapic+ process language into symbolic models directly accepted by the verification tools Tamarin Prover, DeepSec, and ProVerif through the compiler;

[0090] Sapic+ is designed as a general-purpose security protocol modeling specification language, which is integrated in Tamarin Prover. Users can provide a protocol specification, which is then automatically translated to three different verifiers by the Sapic+ compiler. The correctness of the compilation process has been guaranteed by constructing a manual proof. In this invention, this part of the proof is utilized as a part of the correctness proof of the whole framework.

[0091] Prove the correctness of the transformation between different models.

[0092] In this invention, the main concern is the trace property of the protocol, thus the correctness mentioned here mainly refers to: the security properties that are verified by the verifier still hold on the generated DSL protocol model. The proof idea is: first establish the trace inclusion relationship between different models. Then prove that any security property that is verified by the verifier also holds on the DSL model and the Sapic+ model defined in this invention. The proof method is to establish the refinement relationship on the two models before and after the transformation (including T and the compilation process) respectively, and prove that the trace set of the former model is included in the latter model. The use of the following two lemmas shows the correctness of the transformation from the DSL model to the verifiable symbolic model.

[0093] Lemma 1. (The correctness of the transformation rule T). For a DSL protocol model and the SAPIC+ specification obtained by T If the symbolic model on the SAPIC+ specification satisfies the security property that is then it can also be obtained that that is, there is an inclusion relationship between the two trace sets: where denotes the rewriting operation T on denotes that the model satisfies the property trace set.

[0094] Lemma 2. (The correctness of the compilation). For a symbolic model expressed by the Sapic+ specification the symbolic model after compilation is then for the security property satisfied on it still holds on that is and vice versa. In summary, there is an equivalence relationship where the symbol denotes if and only if the relationship.​

[0095] By the above two lemmas, the path set on the DSL model is contained in the generated rewrite rules.

[0096] Tool implementation. In this invention, the above method is implemented as a tool with a Web front-end, written using HTML, GoJS JavaScript library and JQuery. In order to obtain and process intermediate results at different stages, a backend server is built using Flask. An editor is embedded in the html, allowing users to modify the generated results.

[0097] The complexity of the protocol document may make the large language model without domain-specific knowledge more prone to errors, while the protocol document is often long (hundreds or even thousands of word tokens), which will affect the accuracy of the output of the large language model, and the existing large language model often pays less attention to the relevant information in the middle part of the input, resulting in insufficient use of the provided context. The semantic parser constructed based on the large language model driven by the combined category grammar can better process the protocol document input.

[0098] The intermediate representation semantic characteristics of the existing work are not rich enough (such as only representing the semantics of a single sentence, unable to represent the context information of the sentence), and the existing intermediate representation language is expanded or constructed, aiming to improve the accuracy of the extraction model, while reducing the difficulty of subsequent conversion to the modeling language of the verification tool.

[0099] The process from the intermediate representation to the modeling language such as ProVerif and Tamarin Prover cannot be simply regarded as a language translation model, and an analysis-based conversion tool is needed to ensure that the semantics of the extracted protocol model does not change before and after conversion, and is equivalent and consistent. The invention proves the path inclusion relationship of the protocol model before and after conversion by constructing a manual proof.

[0100] The invention proposes a method for synthesizing symbolic models from protocol natural language documents, which can automatically and correctly synthesize symbolic models of some classic security protocols, and can verify successfully.

[0101] Benchmark dataset and large language model selection

[0102] Before conducting case tests, a dataset containing natural language descriptions of protocols and their corresponding symbolic models is needed. Although there are some examples of symbolic models in the GitHub repository of TAMARIN, it is difficult to find the corresponding natural language descriptions. Therefore, a standardized benchmark is first constructed to evaluate the effectiveness of the symbolic model generation method. The classic authentication protocol, such as the Needham-Schroeder protocol, and the modern key exchange protocol, such as SSH, are selected, and the size of these protocols is controllable, without including large-scale protocols such as 5GAKA. The original documents of these protocols, including IETF RFCs, research papers, and online informal texts, are carefully read, and the core parts corresponding to the model are extracted and reorganized. The dataset D is a set of tuples, i.e., D = {(N, P, R)}, where N is the natural language description, P is the Sapic+ specification, and R is the MSR rule verified by TAMARIN. The large language models used for experiments include GPT-3.5-turbo, GPT-4, and Google Gemini-pro, and the temperature parameters for semantic parsing and automatic repair are set to 0.4. To compile and verify the generated symbolic models, the Tamarin Prover version 1.8.0 is used, which includes the Sapic+ platform.

[0103] (2) Comparative experiment setup

[0104] Since there is no related work on synthesizing protocol symbolic models from natural language before, the invention will use the few-shot prompting generation method of large language models as a comparative experiment, that is, given a small number of examples, prompt the large language model to directly synthesize symbolic models from natural language text. For each experimental case, 5 times are synthesized under each method in turn, and the generated results are evaluated from the perspectives of syntax correctness and semantic correctness. Among them, 1) syntax correctness refers to the final synthesized symbolic model that can be directly accepted by the verifier, and 2) semantic correctness is a more stringent requirement, which requires the symbolic model not only to be syntax correct and directly verifiable, but also to be consistent with the reference model manually constructed in the dataset in terms of semantics, correctly modeling the behavior of the protocol.

[0105] (3) Experimental results and analysis

[0106] Example: Evaluate protocol model synthesis methods using constructed dataset.

[0107] The method of the invention is evaluated using the benchmark constructed above, and the detailed information is shown in Table V. The size of the input text ranges from 51 to 639 tokens, and the number of code lines (Sapic+ code lines and Tamarin rewrite rules) ranges from 26 (37) lines to 134 (707) lines

[0108] Table 1 Dataset and synthesis results

[0109]

[0110] A series of basic security properties, including confidentiality and authentication, were verified. The number of properties is listed in column 4 of Table 1. In this experiment, no user interaction was introduced, and the number of running iterations for each protocol was set to 5. The results show that 10 correct models can be automatically obtained in 18 cases, with a success rate of 2 / 5 to 5 / 5. As a comparison, the method has a significant effect compared to a simple few-shot learning method. When using the GPT-4 model, the few-shot learning (3 samples) can at most synthesize 4 semantically correct symbolic models, while the method can synthesize 10 symbolic models.

[0111] Referring to Figure 2 The embodiment of the present application provides a large language model assisted security protocol symbolic model generation method, including a memory and one or more processors, the memory stores executable code, and the processor executes the executable code to realize the large language model assisted security protocol symbolic model generation method in the above embodiment.

[0112] The embodiment of the large language model assisted security protocol symbolic model generation method can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for running by the processor of the device with data processing capability. From the hardware level, as shown in Figure 2 The embodiment of the large language model assisted security protocol symbolic model generation method can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for running by the processor of the device with data processing capability. From the hardware level, as shown in Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown in

[0113] The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0114] For the apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be seen from the parts of the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the modules can be selected to achieve the purposes of the present application according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0115] The embodiment of the present application also provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the method for predicting the reaction of a molecule and an olfactory entity based on a multi-modal deep learning network in the above embodiment.

[0116] The computer readable storage medium can be an internal storage unit of any data processing capable device of any of the preceding embodiments, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device of any data processing capable device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both an internal storage unit and an external storage device of any data processing capable device. The computer readable storage medium is used to store computer programs and other programs and data required by any data processing capable device, and can also be used to temporarily store data that has been output or will be output.

[0117] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that processes, methods, articles, or devices that comprise a list of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

[0118] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired result. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0119] The above-described embodiments of the application have several aspects, no single one of which is solely responsible for the application's desirable attributes. Without limiting the scope of this application, other aspects of the application will become apparent from consideration of the drawings and following detailed description, it being understood that such changes in the state of the art can be made without departing from the spirit and scope of the application.

[0120] The above descriptions are only the preferred embodiment of one or more embodiments of the application, and are not intended to limit one or more embodiments of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the application shall be included in the protection scope of one or more embodiments of the application.

Claims

1. A method for generating a security protocol symbol model assisted by a large language model, characterized in that: The following steps are involved: Parsing a natural language document based on a security protocol into a parsing result represented in a domain-specific language by a semantic parser, and using the parsing result as an intermediate representation, wherein the semantic parser is constructed based on a combinatorial categorical grammar driven by a large language model; Fix problems in the intermediate representation by static analysis and allowing user interaction to formally normalize the intermediate representation; Convert the formally normalized intermediate representation into a universal process specification called Sapic+ through a converter, which can be verified by multiple tools at the same time and can combine the advantages of different verification tools; Compile the standardized Sapic+ process language into a symbolic model directly accepted by verification tools Tamarin Prover, DeepSec, and ProVerif; Prove the correctness of transformations between different models.

2. The method for generating a security protocol symbol model assisted by a large language model according to claim 1, characterized in that: The domain-specific language instantiates the parameters of key predicates through specific items based on the natural language document, and is simplified in the form of lambda expressions and defines clear grammatical rules; The grammatical rules include a combination of event sequences, conditional control structures, and termination representations of protocols.

3. The method for generating a security protocol symbol model assisted by a large language model according to claim 1, characterized in that: The semantic parser includes: a parsing algorithm part for parsing natural language documents and a prompt word part for guiding the parsing algorithm to parse the natural language documents.

4. The method for generating a security protocol symbol model assisted by a large language model according to claim 3, characterized in that: The parsing algorithm includes: dividing the input protocol document into multiple sub-blocks, using a sliding window to move on the sub-blocks in sequence, and using adjacent sub-blocks and corresponding parsing results as context to help parse the next word block.

5. The method for generating a security protocol symbol model assisted by a large language model according to claim 3, characterized in that: The prompt word part includes: the first part, the second part and the third part, among which the first part is to specify a role for the large language model, setting the role as a parsing expert of the combined category grammar; the second part is to interpret the designed domain-specific language in natural language to inform the large language model of the grammatical rules that the output needs to meet; the third part is to add actual sample examples.

6. The method for generating a security protocol symbol model assisted by a large language model according to claim 3, characterized in that: Fix problems in the intermediate representation through static analysis and allowing user interaction to formally normalize the intermediate representation, including: First, a large language model is used to verify the parsing results' consistency with the protocol document's specifications, and unreadable messages are formatted into a readable form. Next, the unbound variables in the parsing results are analyzed, and a graphical message sequence chart of the protocol is generated. The user is then allowed to make corrections in an interactive environment by combining the unbound variable set and the graphical message sequence chart until the parsing result has no unbound variables, ultimately generating a readable domain-specific language expression that conforms to the protocol specifications, namely a domain-specific language program, or intermediate representation.

7. The method for generating a security protocol symbol model assisted by a large language model according to claim 1, characterized in that: The formally normalized intermediate representation is converted to the Sapic+ specification by a converter, including: First, a set of transformation rules is determined to transform the repaired DSL program into local processes of each role in the protocol. The transformation rules recursively process various events in the DSL program. Then, based on the signature of the local process and combined with the hints of the large language model, the top-level process of the protocol is generated using the large language model through few-shot learning; Finally, the local process and the top-level process are combined to form a complete Sapic+ specification.

8. The method for generating a security protocol symbol model assisted by a large language model according to claim 1, characterized in that: Prove the correctness of transformations between different models, including: First, establish the trajectory inclusion relationship between different models; We then prove that any security property that passes verification on the verification tool also applies to the domain-specific language model and the Sapic+ model.

9. A device for generating a security protocol symbol model assisted by a large language model, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, it is used to implement the large language model-assisted security protocol symbol model generation method according to any one of claims 1 to 8.

Images