Cryptography logic vulnerability detection method and system based on large language model

Abstract

The invention relates to the technical field of computer science, and discloses a cryptographic logic vulnerability detection method and system based on a large language model. The method comprises the following steps: firstly, constructing a diversified cryptographic vector knowledge base, and organizing and managing through text vectorization; based on cryptographic algorithm standards such as FIPS, performing logic compliance check on the target code; in combination with a small number of examples and thinking chain prompts, guiding a large language model to identify potential logic vulnerabilities; summarizing a compliance check and reasoning process result, and retrieving related knowledge in a knowledge base based on cosine similarity to enhance model judgment; and finally, integrating knowledge enhancement and logic analysis results, and outputting a structured vulnerability detection report. According to the method, a universal cryptographic vulnerability detection framework which does not need to run codes and is suitable for multiple programming languages is constructed, automatic recognition and detection of complex logic defects in cryptographic codes are achieved, and the accuracy and reasoning ability of a large language model in a cryptographic vulnerability detection task are improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer science and technology and is the application of artificial intelligence in cryptographic analysis. Specifically, it is a cryptographic logic vulnerability detection method and system that combines knowledge retrieval enhancement with large language model reasoning capabilities. Background Art

[0002] Cryptographic algorithms and protocols are fundamental to computer system security and are widely used to ensure data confidentiality, integrity, and authentication. However, correctly implementing cryptographic theory into practical systems is a complex and error-prone task. Developers' mistakes in algorithm implementation, input processing, and parameter configuration often lead to serious security vulnerabilities, especially when they lack professional cryptographic knowledge. In recent years, large language models have been used to assist in code generation, but the resulting cryptographic implementations can introduce new security risks.

[0003] Because cryptographic code is often packaged as libraries and widely reused, a single vulnerability can affect a large number of downstream projects, causing serious consequences. For example, an Elliptic Curve Digital Signature Algorithm (ECDSA) bypass vulnerability discovered in a Java core development library in 2022 allowed attackers to forge digital signatures, bypass authentication mechanisms, and conduct unauthorized access and identity impersonation attacks, resulting in a major security incident.

[0004] While some automated cryptographic vulnerability detection technologies have been studied, most methods focus on detecting errors in cryptographic API (Application Programming Interface) calls. This type of "API misuse" typically refers to developers improperly calling cryptographic library interfaces, such as incorrectly setting the encryption mode or missing random number generation. In contrast, cryptographic logic vulnerabilities stem more deeply from design and reasoning errors in the algorithm implementation itself, and are more hidden and specialized. Currently, discovering these logic vulnerabilities still relies primarily on manual analysis and expert knowledge, a cumbersome and inefficient process that fails to meet the practical needs of efficient security review. Although large language models have shown potential in automated analysis and vulnerability discovery, there is currently no systematic approach to applying them to cryptographic logic vulnerability detection, and related research remains underdeveloped. Summary of the Invention

[0005] To address the above issues, the present invention aims to provide a cryptographic logic vulnerability detection method and system based on a large language model. This method simulates the manual vulnerability analysis process and improves the ability to detect complex logical flaws in cryptographic code. It also integrates chained reasoning and search-enhanced generation technologies to enhance the model's detection accuracy and reasoning capabilities. The technical solution is as follows:

[0006] A cryptographic logic vulnerability detection method based on a large language model includes the following steps:

[0007] Step 1: Collect cryptography-related unstructured text data, use prompt words to guide the basic large language model to extract cryptography knowledge text, and build a local diversified cryptography vector knowledge base through text segmentation and vectorization processing;

[0008] Step 2: Perform semantic analysis on the cryptographic code to be detected (vulnerability detection target) and guide the basic large language model to generate a semantic summary including cryptographic logic, parameter structure and mathematical formulas;

[0009] Step 3: Guide the vulnerability detection model to perform pre-tests on the cryptographic code to be tested, including compliance testing of cryptographic algorithms and chain-thinking reasoning, to obtain pre-test results and identify potential cryptographic logic flaws;

[0010] Step 4: vectorize the semantic summary of the cryptographic code to be detected obtained in step 2 and the pre-detection result obtained in step 3, and then perform similarity search in the diversified cryptographic vector knowledge base to obtain text knowledge blocks for knowledge enhancement;

[0011] Step 5: Input the text knowledge block and the pre-detection results into the large language model to generate a structured vulnerability detection report.

[0012] A cryptographic logic vulnerability detection system based on a large language model, characterized by comprising:

[0013] Knowledge base construction module: used to extract, segment and vectorize cryptographic knowledge text to generate a diversified cryptographic vector knowledge base;

[0014] Semantic understanding module: used to generate semantic summaries for the cryptographic code to be detected;

[0015] Pre-detection module: This module uses a vulnerability detection model to conduct compliance analysis and cryptographic logic flaw reasoning to obtain pre-detection results. The vulnerability detection model is constructed by calling the Application Programming Interface (API) and incorporating technologies such as Chain-of-Thought (CoT) and Retrieval-augmented Generation (RAG).

[0016] Retrieval module: used to perform vector retrieval on the knowledge base and obtain text knowledge blocks for knowledge enhancement;

[0017] Vulnerability analysis module: used to generate vulnerability detection reports by combining semantic summaries, pre-detection results and retrieved text knowledge blocks.

[0018] The beneficial effects of the present invention are:

[0019] 1) This paper integrates Chain-of-Thought (CoT) and Retrieval-augmented Generation (RAG) technologies to simulate the manual vulnerability analysis process, enabling automated identification and detection of complex logical flaws in cryptographic code. This improves the accuracy and reasoning capabilities of large language models in cryptographic vulnerability detection tasks.

[0020] 2) This paper constructs a universal cryptographic vulnerability detection framework that does not require running code and is applicable to multiple programming languages. Its accuracy and robustness are verified through comparative experiments on multiple open source and closed source large language models.

[0021] 3) The present invention discovered undisclosed vulnerabilities in multiple public cryptography code libraries, verifying the effectiveness and practicality of the present invention in automated vulnerability detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is the overall architecture diagram of the cryptographic logic vulnerability detection method system based on the large language model.

[0023] Figure 2 This is a schematic diagram of the prompt word structure for chain thinking reasoning based on small samples. DETAILED DESCRIPTION

[0024] In order to better understand the technical solution of the present invention, the following further describes in detail the implementation of a cryptographic logic vulnerability detection method and system based on a large language model of the present invention in conjunction with the accompanying drawings.

[0025] like Figure 1As shown, the system proposed by the present invention mainly includes the following five steps:

[0026] Step 1: Build a diversified cryptography knowledge vector library: Collect cryptography-related unstructured text data, use prompt words to guide the basic large language model to extract cryptography knowledge text, and build a local diversified cryptography vector knowledge library through text segmentation and vectorization processing.

[0027] Unstructured text data includes: solutions to CTF (Capture the Flag) competitions, cryptography technology blogs, CWE (Common Weakness Enumeration) rule documents, cryptography monographs, cryptanalysis paper abstracts, and technical forum Q&A content.

[0028] Step 1.1: Guide the basic large language model to extract knowledge from each CTF cryptography problem. By using structured prompts, the model outputs semantically compact, logically clear, and independently callable text knowledge blocks.

[0029] Step 1.2: Conduct contextual analysis on the cryptography vulnerability knowledge blog, segment it according to content coherence, and extract key information related to vulnerability detection using prompt words, and convert it into text knowledge blocks in a unified format.

[0030] Step 1.3: Use open-source text segmentation tools (such as the langchain_text_splitters library) to segment cryptography monographs and cryptanalysis papers to ensure that each segment of text fits within the language model context length limit for subsequent embedding and retrieval.

[0031] Step 1.4: Use the OpenAIEmbeddings method to vectorize all the above text knowledge blocks, and use the FAISS vector retrieval engine to build a local knowledge index database.

[0032] Step 1.5: In addition to the above sources, we also supplemented the content including CWE cryptographic rule documents, high-quality StackExchange Q&A, and cryptography research paper abstracts. After cleaning and structuring, we quantized and stored them in the knowledge base.

[0033] Step 2: Guide the model to generate a semantic summary of the target code: Perform semantic analysis on the cryptographic code to be detected (vulnerability detection target) and guide the basic large language model to generate a semantic summary including cryptographic logic, parameter structure and mathematical formulas.

[0034] Step 2.1: Input the cryptographic code to be tested into the large language model, and use prompt words to guide the model to understand the code's encryption process, call logic, parameter structure, core algorithm and other information.

[0035] Step 2.2: The model outputs a structured semantic summary text, which will serve as the query basis for subsequent knowledge retrieval and vulnerability analysis.

[0036] Step 3: Perform pre-detection analysis on the target code: Guide the vulnerability detection model to perform pre-detection on the cryptographic code to be detected, including compliance detection of cryptographic algorithms and chain thinking reasoning of common vulnerabilities, to obtain pre-detection results, thereby identifying potential cryptographic logic defects.

[0037] Step 3.1: Based on the manually organized FIPS (Federal Information Processing Standards) cryptographic algorithm standards, use structured prompts to guide the model to perform compliance testing on the implementation of the standard cryptographic algorithm, identify whether the algorithm implementation in the code complies with the standards, and whether there are any problems such as inconsistent algorithm logic and non-standard parameter configuration.

[0038] Step 3.2: For non-standard encryption logic or common cryptographic code, use chain thinking prompts based on a small number of vulnerability examples to guide the model to chain thinking and multi-step reasoning to discover potential logical flaws in the code and determine whether the following cryptographic logic flaws exist:

[0039] (1) Key misuse: including use of default keys, use of weak keys, key leakage, insufficient key length, and insufficient key entropy security;

[0040] (2) Parameter misuse: including using insecure curves, insecure algorithms, insecure nonce values, insecure initialization vectors, and insecure encryption modes;

[0041] (3) Lack of data validation: for example, missing primality detection, abnormal data leading to code dead loops, illegal data signature value bypass, invalid curve attack vulnerabilities, and lack of illegal certificate format detection;

[0042] (4) Parameter generation process defects: such as weak curve generation, non-standard parameter generation process and non-standard key generation process;

[0043] (5) Error handling defects: such as failure to catch exceptions leading to time side channel leakage or introduction of prediction attack risks;

[0044] (6) Pseudo-random number generation flaws: including predictable random number seeds and fragile random number generator structures.

[0045] The typical structure of chain thinking prompt words based on a small number of examples is as follows Figure 2As shown, the overall markdown format is adopted, which is easier for large language models to understand, and is divided into three parts. The Instruction section introduces a certain type of cryptographic logic vulnerability and then designs chain prompts based on the cause of the vulnerability. The Few-Shot section is a case study of some confirmed code snippets of this type of vulnerability, and the chain thinking analysis process and structured result output of the large language model are clearly defined. The Notice section clearly specifies the specific format of the detection model output results and some precautions in the reasoning process.

[0046] Among them, chain thinking reasoning is based on a preset structured prompt word template, which includes vulnerability type description, example samples and reasoning format specifications.

[0047] Step 3.3: Use structured prompt words to guide the vulnerability detection model to uniformly summarize and summarize the model output content generated in steps 3.1 and 3.2, and construct a standardized pre-detection result text.

[0048] Step 4: Perform knowledge retrieval based on vector similarity: vectorize the semantic summary of the cryptographic code to be detected obtained in step 2 and the pre-detection result obtained in step 3, and then perform similarity retrieval in the diversified cryptographic vector knowledge base to obtain text knowledge blocks for knowledge enhancement.

[0049] Step 4.1: Set the similarity threshold τ to avoid introducing knowledge fragments with low semantic relevance to interfere with model reasoning during knowledge enhancement.

[0050] Step 4.2: Vectorize the semantic summary text generated in Step 2 and Step 3 and the pre-detection result text respectively, use them as retrieval query vectors, and calculate the cosine similarity with the knowledge blocks in the knowledge vector library constructed in Step 1.

[0051]

[0052] Among them, q represents the vector used for retrieval, and t is any knowledge block found by brute force search in the knowledge base.

[0053] Step 4.3: Sort the calculation results by similarity from high to low, select several knowledge blocks with similarity higher than the threshold τ as knowledge enhancement content for subsequent models to use, and remove the result if the similarity is lower than the threshold.

[0054] Step 5: Output a structured vulnerability detection report: Input the text knowledge block and the pre-detection results into a large language model to generate a structured vulnerability detection report.

[0055] The pre-detection results from step 3 and the knowledge-enhanced content retrieved in step 4 are fed into the large language model. After comprehensive analysis, the model generates a structured vulnerability detection report. This report includes, but is not limited to, vulnerability name, affected function / module, triggering conditions, vulnerability type, reference knowledge block, risk level, CWE corresponding number, and remediation suggestions.

[0056] The report results support structured format output (such as JSON, Markdown, etc.), which is convenient for integration into security audit systems or code quality inspection processes.

[0057] Through the above steps, the present invention constructs an automated cryptographic logic vulnerability detection method based on a large language model. This method has the technical advantages of requiring no executable code, adapting to multiple programming language codes, strong scalability, and high detection accuracy, and has broad practical application value.

[0058] Another embodiment of the present invention:

[0059] A cryptographic logic vulnerability detection system based on a large language model, the system comprising:

[0060] Knowledge base construction module: used to extract, segment and vectorize cryptography knowledge text to generate a knowledge base. It also includes a FAISS vector database to efficiently support cosine similarity retrieval

[0061] Semantic understanding module: used to generate semantic summaries for the code to be detected.

[0062] Pre-detection module: used for compliance analysis and logical vulnerability reasoning; includes compliance detection unit and reasoning analysis unit; the compliance detection unit is used to compare whether the code logic complies with FIPS and other standards; the reasoning analysis unit is used to perform chain reasoning analysis on common cryptographic vulnerabilities based on a small number of examples and structured prompt words.

[0063] Retrieval module: used to perform vector retrieval on the knowledge base and obtain relevant knowledge blocks.

[0064] Vulnerability analysis module: used to generate vulnerability detection reports by combining semantic summaries, pre-detection results, and retrieval knowledge.

[0065] For the prompt words used in the present invention, some cases are provided here for reference.

[0066] Code semantic summary prompt words

[0067] instruction:

[0068] You are a cryptography expert. You are given a cryptographic algorithm implementation from the source file <target code filename>. The following is the code snippet: <target code>. Your task is to summarize the semantics of the code. You should focus on the cryptographic logic and execution steps, parameter sizes (e.g., the bit length of an RSA key), and the core mathematical formulas in the code.

[0069] Note the following:

[0070] (1) Code semantic summary prompt words

[0071] instruction:

[0072] You are a cryptography expert. You are given a cryptographic algorithm implementation from the source file <target code filename>. The following is the code snippet: <target code>. Your task is to summarize the semantics of the code. You should focus on the cryptographic logic and execution steps, parameter sizes (e.g., the bit length of an RSA key), and the core mathematical formulas in the code.

[0073] Note the following:

[0074] 1) Please do not use phrases such as "here," "in this case," or "in the examples provided."

[0075] 2.) Please summarize your answer in a coherent, logically coherent paragraph. Do not use bullet points.

[0076] 3) Please highlight the content related to cryptography and downplay the parts not related to cryptography.

[0077] 4) Please focus on the code logic. There is no need to discuss the security of the code or algorithm, and do not add any commentary.

[0078] 5) When summarizing, please pay special attention to the specific values ​​of cryptographic parameters.

[0079] (2) Compliance check step prompt words

[0080] instruction:

[0081] Please follow the algorithm flow in the standard implementation document and check the code step by step to identify any non-compliant parts. For each non-compliant part, please provide an analysis in the following format:

[0082] Non-compliant code block:

[0083] <Detected non-compliant code>

[0084] Analysis and reasons:

[0085] <Point out where the code does not conform to the standard and briefly explain why>

[0086] Notes:

[0087] 1) Please pay special attention to logical consistency, mathematical rigor, and correctness of parameter ranges.

[0088] 2) Ignore variable naming differences and code content that is not related to the algorithm.

[0089] 3) The code snippet provided may not be a complete algorithm implementation. For example, if the code snippet only contains the signature part of the DSA algorithm but not the verification part, you only need to check whether the signature part conforms to the standard.

[0090] 4) If the code fully complies with the standard, return "None".

[0091] 5) Pay special attention to the sections marked with "====" in the standard implementation document, as these are often areas prone to problems. For example: == verifies that r and s are integers in the interval [1, n-1].

[0092] 6) If there are undefined functions in the code, this is normal, as our code is usually a single file or fragment extracted from a complete code base. Please infer its function based on the function name.

[0093] Contents of the standard implementation document:

[0094] <Algorithm Implementation Standard>

[0095] (3) Knowledge enhancement step prompt words

[0096] instruction:

[0097] Based on the key points of the previous analysis, please analyze the following code snippets for cryptographic vulnerabilities.

[0098] Code snippet: <target code>

[0099] The code logic semantics you summarized earlier are as follows: <Code semantics summary>

[0100] The potential cryptographic logic vulnerabilities you pre-tested are as follows:

[0101] Notes:

[0102] - Please summarize the cryptographic vulnerability in a precise, concise, and focused manner, without omitting key information. You may use bullet points for your answer. If there is a basis for certain vulnerability assessment criteria, you may cite relevant standards, such as FIPS or CWE.

[0103] -You will get some additional reference knowledge, but if they are not relevant to your task, please ignore them.

[0104] - Please do not list parts of the code that do not have issues, such as "no timing side channels" or "no memory safety issues", as this introduces unnecessary information noise.

[0105] Reference knowledge blocks:

[0106] Knowledge Block 1;

[0107] Knowledge block 2.

[0108] The present invention constructed a baseline test for detecting cryptographic logic vulnerabilities using six existing advanced large language models, and used these models to build the vulnerability detection system of the present invention. The effectiveness and robustness of the enhanced reasoning ability were verified through before-and-after comparison.

[0109] At the same time, the detection framework proposed in the present invention is adopted to build a system based on the DeepSeek-V3 basic model. By performing batch automated audits on the source codes in 10 cryptography code libraries, including goEncrypt, cryptography, crypto-random-string and nimcrypto, more than ten implementation defects of algorithms including PBKDF2, AES, RSA, ECDSA, etc. are detected, verifying the effectiveness and practicality of the present invention in automated vulnerability detection.

Claims

1. A cryptographic logic vulnerability detection method based on a large language model, characterized in that: The following steps are involved: Step 1: Collect cryptography-related unstructured text data, use prompt words to guide the basic large language model to extract cryptography knowledge text, and build a local diversified cryptography vector knowledge base through text segmentation and vectorization processing; Step 2: Perform semantic analysis on the cryptographic code to be detected, guiding the basic large language model to generate a semantic summary including cryptographic logic, parameter structure and mathematical formulas; Step 3: Build a vulnerability detection model to pre-test the cryptographic code to be tested, including compliance testing of cryptographic algorithms and chain-thinking reasoning, to obtain pre-test results and identify potential cryptographic logic flaws; Step 4: Vectorize the semantic summary of the cryptographic code to be tested obtained in step 2 and the pre-test results obtained in step 3. Then perform similarity search in the diversified cryptographic vector knowledge base to obtain text knowledge blocks for knowledge enhancement. Step 5: The text knowledge block and the pre-detection results are used together as knowledge enhancement content to improve the detection capability of the model and generate a structured vulnerability detection report.

2. The cryptographic logic vulnerability detection method based on a large language model according to claim 1 is characterized in that: The step 1 is specifically as follows: Step 1.1: Use structured prompts to guide the basic large language model to extract knowledge from a single question in the CTF cryptography problem-solving report to obtain an independent and compact single text knowledge block; Step 1.2: Use structured prompt words to segment the text based on the relevance of the context of the cryptography vulnerability knowledge blog, and perform knowledge extraction on the segmented text to obtain concise text knowledge blocks; Step 1.3: Use open source text segmentation tools to segment the cryptography monographs. Step 1.4: Use the OpenAIEmbeddings method to vectorize the extracted and segmented text knowledge blocks and store them in the FAISS vector database to form a diversified cryptography vector knowledge base.

3. The cryptographic logic vulnerability detection method based on a large language model according to claim 2 is characterized in that: The step 3 specifically includes: Step 3.1: Based on the manually curated FIPS cryptographic algorithm standards, use structured prompts to guide the large language model to perform compliance testing on the implementation of the standard cryptographic algorithm, identifying whether the algorithm implementation in the cryptographic code complies with the standards; Step 3.2: For non-standard encryption logic or common cryptographic code, use chaining prompts based on fewer than a set number of vulnerability examples to guide the model to chain thinking, exploring and detecting potential logical flaws in the cryptographic code. Determine whether the following cryptographic logic flaws exist: key misuse, parameter misuse, lack of data validation, parameter generation process flaws, error handling flaws, or pseudo-random number generation flaws. Step 3.3: Use structured prompt words to guide the vulnerability detection model to uniformly summarize and summarize the model output content generated in steps 3.1 and 3.2, and construct a standardized pre-detection result text.

4. The cryptographic logic vulnerability detection method based on a large language model according to claim 3 is characterized in that: The step 4 specifically includes: Step 4.1: Set the similarity threshold τ to reduce the misleading effect of weakly correlated knowledge blocks on vulnerability detection. Step 4.2: Vectorize the semantic summary and pre-detection results respectively, use them as search query vectors, and perform cosine similarity calculation with the text knowledge blocks in the diversified cryptography vector knowledge base: Where q is the retrieval query vector, t is any text knowledge block found by brute force search in the knowledge base; ω(q,t) represents the cosine similarity between the retrieval query vector q and the text knowledge block t; Step 4.3: Sort the calculation results by cosine similarity from high to low, select the text knowledge blocks with cosine similarity higher than the similarity threshold τ as the knowledge enhancement content for subsequent models, and remove the text knowledge blocks with cosine similarity lower than the similarity threshold τ.

5. A cryptographic logic vulnerability detection system based on a large language model, characterized in that: include: Knowledge base construction module: used to extract, segment and vectorize cryptographic knowledge text to generate a diversified cryptographic vector knowledge base; Semantic understanding module: used to generate semantic summaries for the cryptographic code to be detected; Pre-detection module: Through the constructed vulnerability detection model, compliance analysis and cryptographic logic flaw reasoning are performed to obtain pre-detection results; Retrieval module: used to perform vector retrieval on the knowledge base and obtain text knowledge blocks for knowledge enhancement; Vulnerability analysis module: used to generate vulnerability detection reports by combining semantic summaries, pre-detection results and retrieved text knowledge blocks.

6. The cryptographic logic vulnerability detection system based on a large language model according to claim 5 is characterized in that: The pre-detection module includes: Compliance detection unit: used to check whether the code logic complies with FIPS standards; Reasoning Analysis Unit: Used to perform chain reasoning analysis on common cryptographic vulnerabilities based on fewer than a set number of examples and structured prompts; The reasoning analysis unit includes: Instruction: Used to introduce various types of cryptographic logic vulnerabilities and then design chain prompts based on the causes of the logic vulnerabilities; Few-Shot: Includes code snippets of confirmed vulnerability types, the analysis process of chain thinking using a large language model, and structured result output; Notice: Used to specify the specific format of the detection model output results and some precautions during the reasoning process.

7. The cryptographic logic vulnerability detection system based on a large language model according to claim 5 is characterized in that: The knowledge base construction module includes a FAISS vector database for supporting cosine similarity retrieval.

Images