Plug-and-play security enhancement method and system based on large model self-reflection mechanism

Abstract

The invention provides a plug-and-play security enhancement method and system based on a large model self-reflection mechanism, and the method comprises the steps: S1, receiving a user input demand, generating a preliminary code, and extracting a potential risk; s2, starting a security detection module to perform reflection verification, identifying potential security issues, and marking the security issues through a self-generated security knowledge base; s3, calling a correction module to perform code optimization, dynamically retrieving a security knowledge base to perform security problem matching as reference content, performing automatic correction and code optimization, and returning to extract potential risks and perform reflection verification; and S4, outputting a final code, generating a safety analysis report, summarizing the accumulated safety knowledge, and updating and expanding a safety knowledge base. According to the method, automatic correction, code optimization and reflection verification can be performed through a self-reflection mechanism in each stage of the code generation process, and it is ensured that each output result meets the safety requirement so as to meet the requirement for universality between different models.

Description

Technical Field

[0001] The present invention relates to a technical solution for security enhancement, and in particular to a plug-in security enhancement method based on a large model self-reflection mechanism, and further to a plug-in security enhancement system that adopts the plug-in security enhancement method based on a large model self-reflection mechanism. Background Art

[0002] With the continuous development of artificial intelligence technology, large language models (LLMs), such as ChatGPT and Claude, have been widely used in many fields such as reasoning, programming, and scientific research, and have achieved remarkable results in many practical applications. These large language models can not only generate high-quality natural language text, but also assist programming and provide services such as code generation, debugging, and optimization. In the field of programming, LLMs are used to automatically generate code, optimize algorithms, and assist developers in improving programming efficiency. However, despite the great potential of these technologies, there are still some important security risks and problems, especially in the generated code, which may hide serious security vulnerabilities.

[0003] In the traditional software development process, security issues are usually detected and fixed through static analysis, dynamic testing and other means after the code is written. However, the current LLMs technology does not fully consider the security of the code, especially when generating complex programs, the code generated by these models often lacks the necessary security review. For example, some models may generate code containing SQL injection vulnerabilities, allowing attackers to manipulate the database through malicious input. For another example, there may be unvalidated user input in the generated code, which in turn causes security vulnerabilities such as cross-site scripting (XSS). In November 2024, a user built an auxiliary trading robot through the code generated by ChatGPT. However, because the code generated by the model called a malicious API and directly passed the user's private key to the API, the user's wallet was stolen for $2.5k. This incident exposed the potential security threats that may be caused by not reviewing and filtering malicious code during the generation process.

[0004] What's more serious is that the existing LLMs models do not actively evaluate the security of the code when generating it, which may cause unpredictable security vulnerabilities in the actual runtime of the generated code. Taking SQL injection as an example, LLMs may generate a code snippet containing a database query, but if the user input is not properly processed, malicious users may cause database attacks and data leaks by entering SQL statements. In addition, as more and more companies and organizations apply LLMs to actual workflows, how to ensure that large language models do not contain potential security risks and vulnerabilities when generating code and other content has become a technical problem that needs to be solved urgently.

[0005] Existing security detection methods are mostly concentrated in the review phase after code generation, such as using static code analysis tools and fuzz testing. Such existing methods often have high costs and long time delays. In a rapidly developing development environment, this post-repair method cannot effectively meet the security requirements in large-scale automated programming scenarios. Therefore, how to achieve real-time security assessment of code during the generation process has become one of the key technical challenges to improve the security of LLMs applications.

[0006] In addition, most existing methods focus on fine-tuning and secure alignment of specific models, or focus on content filtering mechanisms at the input and output ends. Although these existing solutions can solve some security problems to a certain extent, they still have some limitations and shortcomings. For example, the methods of fine-tuning and secure alignment require a large amount of labeled data and training resources, while the filtering mechanisms at the input and output ends rely on rules and templates, which are difficult to adapt to complex and changing generation tasks. More importantly, these methods are usually specific to a certain model or task, cannot be used across different models, and have certain limitations, and cannot meet the security requirements of large-scale, real-time, and automated generation. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a plug-in security enhancement method based on the self-reflection mechanism of a large model, which aims to automatically correct, optimize and verify the code through the self-reflection mechanism at each stage of the code generation process; on this basis, it can also self-generate and expand the security knowledge base in this process, and support and promote each other with the self-reflection mechanism to meet the universality requirements between different models, and provide a better foundation for meeting the security requirements of large-scale, real-time and automatic generation. Furthermore, a plug-in security enhancement system that adopts the plug-in security enhancement method based on the self-reflection mechanism of a large model is also provided.

[0008] To this end, the present invention provides a plug-in security enhancement method based on a large model self-reflection mechanism, comprising the following steps: Step S1, receiving user input requirements in real time, generating preliminary code through a large language model, first identifying sensitive data based on rule matching using regular expressions and static analysis tools, and extracting potential risks in the preliminary code through a large language model, the potential risks including unencrypted password information, sensitive data, and operations that do not comply with security rules or coding specifications, the sensitive data including API keys, passwords, and private information; Step S2, starting the security detection module to perform reflection verification, identify potential security issues, and mark the security issues through the security knowledge base self-generated by the large language model; the security issues include SQL injection, XSS attack and buffer overflow; Step S3, dynamically search the security knowledge base, generate repair suggestions, and call the correction module to optimize the code. The security knowledge base is dynamically searched based on the RAG retrieval enhancement generation model, and the security issues are matched with the security knowledge base as reference content. After matching, automatic correction and code optimization are performed according to the security knowledge base, and potential risks are extracted and reflected verification is performed; Step S4, output the final code, generate a security analysis report, summarize the security knowledge accumulated during the reflection and verification process, and update and expand the security knowledge base; the content of the security analysis report includes security issues, optimized code and scores.

[0009] A further improvement of the present invention is that step S1 includes the following sub-steps: Step S101, parsing the user's input requirements through a large language model, and generating preliminary codes according to the natural language description; Step S102, analyze each generated preliminary code through a large language model, extract security-related knowledge, and obtain potential risks in the preliminary code; security-related knowledge includes security vulnerabilities, attack methods, security rules and coding specifications.

[0010] A further improvement of the present invention is that in step S102, for security-related knowledge, security vulnerabilities corresponding to code patterns are identified through a large language model, potential attack methods are identified, and corresponding security rules and coding specifications are synchronously stored in the security knowledge base.

[0011] A further improvement of the present invention is that step S2 includes the following sub-steps: Step S201, start the security detection module, analyze the preliminary code and its potential risks through static analysis tools, and identify potential security issues; Step S202: classify and mark potential security issues based on the security knowledge in the security knowledge base, and provide targeted repair suggestions or alternative solutions in combination with domain knowledge and best practices.

[0012] A further improvement of the present invention is that in step S202, when generating codes involving database interaction, the large language model avoids the security issue of SQL injection through parameterized queries or ORM framework; when generating Web application codes, the large language model will remind users to verify and escape input requirements to avoid the security issue of XSS attacks; the large language model dynamically adjusts the security policy according to the domain corresponding to the task scenario and in combination with domain knowledge to ensure that the generated content complies with the latest security rules, coding specifications and industry standards.

[0013] A further improvement of the present invention is that step S3 includes the following sub-steps: Step S301, calling the correction module to perform code optimization, dynamically searching the security knowledge base based on the RAG retrieval enhancement generation model, matching the security problem with the security knowledge base, calculating the vector similarity between the security problem and each knowledge item in the security knowledge base, and obtaining the knowledge item whose vector similarity with the security problem is greater than a preset similarity threshold as the matching knowledge item; Step S302, inputting the matched knowledge items as context information into the correction module to generate code correction suggestions for security issues; Step S303, automatically optimizing the code according to the generated code correction suggestions; Step S304, return to step S2, and perform reflection verification on the generated content each time. If the generated code contains unsafe function calls or hard-coded passwords, they are automatically replaced or deleted to ensure that the generated content meets security standards; if the generated code contains unencrypted password information, the password is automatically encrypted or sensitive information is deleted through its own reflection mechanism; until the content after reflection verification passes the security assessment.

[0014] A further improvement of the present invention is that in step S303, during the automatic optimization process, adjustments are also made based on user feedback. If the user objects to the security of the generated content or discovers new security risks, the content is reflected and verified again through the large language model to ensure that the output results meet the security requirements.

[0015] A further improvement of the present invention is that in step S303, after the code is automatically optimized, verification and permission control are automatically added to the optimized code.

[0016] A further improvement of the present invention is that step S4 includes the following sub-steps: Step S401, after completing code optimization and reflection verification, output the final code; Step S402, listing the security issues found in the code optimization and reflection verification process, displaying the optimized code snippets, marking the comparison before and after the optimization, and scoring the code optimization according to the preset scoring criteria to generate a security analysis report; Step S403, summarize the security knowledge accumulated in each code optimization and reflection verification process, associate security issues, optimization solutions and best practices, update and expand the security knowledge base, and implement and maintain its own reflection mechanism through a large language model.

[0017] The present invention also provides a plug-in security enhancement system based on a large model self-reflection mechanism, which adopts the plug-in security enhancement method based on a large model self-reflection mechanism as described above, and includes: The code generation module receives user input requirements in real time and generates preliminary code through a large language model. It first uses regular expressions and static analysis tools based on rule matching to identify sensitive data, and uses a large language model to extract potential risks in the preliminary code. The potential risks include unencrypted password information, sensitive data, and operations that do not comply with security rules or coding specifications. The sensitive data includes API keys, passwords, and private information. The reflection and verification module starts the security detection module to perform reflection and verification, identify potential security issues, and mark the security issues through the security knowledge base self-generated by the large language model; The security optimization module dynamically searches the security knowledge base, generates repair suggestions, and calls the correction module to perform code optimization. It dynamically searches the security knowledge base based on the RAG retrieval enhancement generation model, matches the security issues with the security knowledge base as reference content, and performs automatic correction and code optimization according to the security knowledge base after matching, and returns to extract potential risks and perform reflection verification; The output module outputs the final code, generates a security analysis report, summarizes the security knowledge accumulated during the reflection and verification process, and updates and expands the security knowledge base.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: after receiving the user's input requirements in real time, a preliminary code is first generated through a large language model, and potential risks in the preliminary code are extracted through the large language model; then, the security detection module is started to perform reflective verification, identify potential security issues, and mark the security issues through the security knowledge base self-generated by the large language model; then, the correction module is called to perform code optimization, the security knowledge base is dynamically retrieved based on the RAG retrieval enhancement generation model, and the security issues are matched with the security knowledge base. After matching, automatic correction and code optimization are performed according to the security knowledge base, and after automatic correction and optimization, the code is returned for reflective verification until the security assessment is passed; finally, the final code is output, a security analysis report is generated, and the security knowledge accumulated during the reflective verification process is summarized, and the security knowledge base is updated and expanded.

[0019] Therefore, the present application can perform automatic correction, code optimization and reflective verification through its own reflection mechanism at each stage of the code generation process, instead of performing simple error correction after generating the task / output code like the prior art, to ensure that each output result meets the security requirements; on this basis, it can also self-generate and expand the security knowledge base in this process, so that the self-generated security knowledge base and its own reflection mechanism form a mutually supportive and promoting relationship, without the need to reference an external knowledge base, and no longer limited to a specific model or task, it can well meet the universality requirements between different models, and provide a better foundation for meeting the security requirements of large-scale, real-time and automated generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the working process of an embodiment of the present invention; Figure 2 It is a schematic block diagram of the principle of the self-reflection mechanism of an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In the description of the present invention, if "several" is involved, it means more than one; if "multiple" is involved, it means more than two; if "greater than", "less than", or "exceed" is involved, it should be understood as not including the number itself; if "above", "below", or "within" is involved, it should be understood as including the number itself. If "first", "second", etc. are involved, they should be understood as being used only to distinguish the names of the same or similar technical features, and should not be understood as implying / indicating the relative importance of the technical features, the number of the technical features, or the order of the technical features.

[0022] Before describing the preferred embodiments of the present invention in detail, the related technologies are explained first. The abbreviations and key term definitions involved include: LLM refers to Large Language Model, which is an artificial intelligence model based on deep learning technology, with a large number of parameters and powerful natural language processing capabilities. Transformer, a machine learning model based on the Self-Attention mechanism proposed by the paper "Attention is All You Need". Self-reflection refers to the ability of large language models to evaluate and improve their own generated outputs, aiming to improve model reliability.

[0023] Existing security detection methods are mostly concentrated in the review stage after code output. This post-repair method cannot effectively meet the security needs in large-scale automated programming scenarios.

[0024] In view of the above problems, the present invention aims to propose a plug-in security enhancement method and system based on the self-reflection mechanism of a large model. The large model refers to a large language model. By introducing a security self-checking mechanism at each stage of the code generation process, potential security issues in the generated code can be monitored and repaired in real time. Specifically, the method introduces a reflection mechanism so that LLMs can self-verify and correct the generated code according to preset security rules when generating code. During the generation process, the model of the present invention will not only evaluate the functionality of the code, but also evaluate its security, and promptly discover and repair potential security vulnerabilities, thereby reducing the time and cost of later repairs.

[0025] For example, when a user requests to generate a piece of code that includes database operations, the reflection mechanism will check whether there is a potential SQL injection risk in the code. If a potential vulnerability is detected, the model will automatically adjust the code and use safe means such as parameterized queries to replace unsafe string concatenation methods to avoid SQL injection attacks. For another example, when generating code that includes user input processing, the reflection mechanism will check whether the user input has been strictly verified and disinfected to prevent security issues such as cross-site scripting attacks (XSS).

[0026] In addition, the self-reflection mechanism can also self-optimize the code generated by the model. For example, when generating code involving sensitive information (such as API keys, private keys, etc.), the model will automatically detect whether there is a risk of exposing sensitive information and prompt the user to encrypt it or use a safer method such as environment variable storage. In this process, the reflection mechanism not only improves the security of the generated code, but also enhances the robustness and stability of the code.

[0027] On this basis, in order to further improve the security of the generation process, the present invention also proposes a security knowledge enhancement method combined with a security knowledge base. By introducing a security knowledge base, LLMs can call security knowledge in real time when generating code to further enhance the security of the generated code. For example, when generating code involving encryption operations, the model can refer to the best practices in the security knowledge base to ensure that the generated code meets the current encryption standards and avoids common encryption vulnerabilities.

[0028] It should be noted that if an external knowledge base is used, it may also face the problem of introducing noise and irrelevant information. In order to solve this technical problem, the present invention proposes a knowledge filtering and concentration method, which forms a self-generated security knowledge base by filtering and refining external security knowledge, ensuring that only the most relevant security information is introduced into the generation process, thereby avoiding the problem of guiding the model to generate wrong or irrelevant security recommendations.

[0029] By introducing a self-reflection mechanism and enhancing security knowledge, the present invention can achieve real-time security review and repair at each settlement in the code generation process, thereby reducing security risks in the code and improving the reliability and robustness of code generation. This innovative technical solution can not only effectively reduce the cost of later security vulnerability repairs, but also improve the overall security of automated programming, providing technical support for the promotion and application of large language models in more practical applications.

[0030] In a technical solution related to the present invention, the two-party game alignment framework proposed by Rui Zhang et al. has significantly expanded and innovated on the basis of existing methods. Traditional alignment methods, such as reinforcement learning with human feedback (RLHF), are widely used in practice, but their limitations are that they are difficult to fully cover all task types and are prone to model performance saturation due to over-reliance on pre-collected prompts. Preference learning methods and methods that learn directly from preference data attempt to reduce reliance on human feedback, but they still face challenges in the quality of labeled data and dynamic adjustment capabilities in practical applications.

[0031] In contrast, the two-party game framework proposed by this technical solution provides a more dynamic and adaptive alignment environment by introducing an adversarial defense game mechanism. The core of this method lies in identifying and exploiting potential weaknesses through iterative interactions, which surpasses traditional methods in many aspects. First, through the interaction between the adversary and the defender, the model can continuously learn and improve in a constantly changing environment, significantly improving the generalization ability. Second, the introduction of diversity constraints ensures that the generated attack prompts are diverse, which not only enhances the robustness of the model, but also promotes the development of a wider range of attack and defense strategies.

[0032] In theory, this technical solution provides stability guarantees for the algorithm by proving convergence to Nash equilibrium. This theoretical basis makes the method more reliable and predictable in practice. However, although this technical solution has significant advantages in improving model security and generalization capabilities, it also faces some challenges, including: high computational complexity, requiring multiple iterations of optimization, and relying on high-performance computing resources, which places high demands on hardware facilities; it is difficult to implement and relies on high-quality data sets and annotations, which increases the complexity in data acquisition and processing.

[0033] In another technical solution related to the present invention, Chen et al. proposed SecAlign, a defense method based on preference optimization technology, which aims to protect large language models (LLMs) from prompt injection attacks. SecAlign teaches LLMs to prefer secure outputs over unsafe outputs by constructing a preference dataset containing prompt injection inputs, safe outputs (outputs in response to legal instructions), and unsafe outputs (outputs in response to injections), and performing preference optimization on this dataset. This method reduces the success rate of various prompt injections to about 0% for the first time, and shows good generalization ability even in the face of more complex attacks than those seen during training. In addition, the defense-trained model is comparable in practicality to the one before defense training, with similar utility.

[0034] The main advantages of SecAlign are its simplicity, preservation of model utility, and strong security against unknown attacks, even against optimization-based attacks. By favoring optimization, SecAlign also establishes a connection between LLM security and alignment, two topics that were previously studied separately.

[0035] However, SecAlign still has some limitations. First, it is only applicable to scenarios where the instruction part and the data part are clearly separated (for example, by delimiters). Second, the expected output constructed by SecAlign shares a shortcoming with other fine-tuning-based defense methods: the expected output ignores the injected instructions in the data instead of processing them as part of the data. This may cause LLM to ignore some command sentences in the data (which may not be injected but should be processed as data, such as command sentences that need to be translated). Solving this problem requires carefully selecting injections based on benign instructions and generating a special expected response instead of the response from the SFT dataset. In addition, as a defense method for AI systems, SecAlign cannot achieve 100% security. For stronger security requirements in LLM integrated applications, a multi-level defense system may be required to combine SecAlign with other technologies such as detection and input reformatting. Finally, SecAlign currently cannot defend against attacks other than prompt injection, such as jailbreak attacks and data extraction.

[0036] Therefore, the above two related technical solutions obviously cannot solve the technical problem to be solved by the present invention.

[0037] The preferred embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0038] It should be noted that most existing technical solutions focus on fine-tuning and security alignment of specific models, or focus on content filtering mechanisms at the input and output ends. Although these existing solutions can solve some security problems to a certain extent, they still have some limitations and shortcomings. For example: the methods of fine-tuning and security alignment require a large amount of labeled data and training resources, while the filtering mechanisms at the input and output ends rely on rules and templates, which are difficult to adapt to complex and changeable generation tasks. More importantly, existing solutions are usually specific to a certain model or task, cannot be used between different models, and have certain limitations, and cannot meet the security requirements of large-scale, real-time and automated generation.

[0039] For this reason, Figure 1 and Figure 2 As shown, this embodiment provides a plug-in security enhancement method based on a large model self-reflection mechanism, comprising the following steps: Step S1, receiving the user's input requirements in real time, generating preliminary code through a large language model, firstly using regular expressions and static analysis tools (such as Semgrep) based on rule matching to quickly identify sensitive data (such as API keys, passwords, and other private information, etc.), and extracting potential risks in the preliminary code through the large language model, including analyzing the code context of the preliminary code through the large language model to identify operations that do not comply with security rules or coding specifications; the potential risks include unencrypted password information, sensitive data, and operations that do not comply with security rules or coding specifications, etc., and the sensitive data includes API keys, passwords, and private information; Step S2, starting the security detection module to perform reflection verification, identify potential security issues, and mark the security issues through the security knowledge base self-generated by the large language model; the security knowledge base refers to a self-generated rule base containing common vulnerability patterns, and the common vulnerability patterns include vulnerability patterns corresponding to security issues such as SQL injection, XSS attack, buffer overflow, etc. The main source of security knowledge of the security knowledge base includes the history of iterative retrieval and the previous conversation history; the security issues include SQL injection, XSS attack, buffer overflow, etc.; Step S3, dynamically retrieve the security knowledge base and generate repair suggestions, and call the correction module to perform code optimization. Dynamically retrieve the security knowledge base based on the RAG retrieval enhancement generation model, match the security issues with the security knowledge base as reference content, and perform automatic correction and code optimization according to the security knowledge base after matching, and return to extract potential risks and perform reflection verification; Step S4, output the final code, generate a security analysis report, summarize the security knowledge accumulated during the reflection and verification process, and update and expand the security knowledge base; the content of the security analysis report includes security issues, optimized code and scores.

[0040] This embodiment aims to provide a general, plug-in security enhancement method and system that does not require fine-tuning training, and aims to reflect and strengthen its security through the task generation information of the large language model itself. After receiving the user's input requirements in real time, this embodiment first generates preliminary code through the large language model, and extracts potential risks in the preliminary code through the large language model; in the code generation stage, the possible risk points of the preliminary code are extracted. Then, in the reflection verification stage, the security detection module is started to perform reflection verification, identify potential security issues, and mark the security issues through the security knowledge base self-generated by the large language model. Next, in the security optimization stage, the correction module is called to optimize the code, and the security knowledge base is dynamically retrieved based on the RAG retrieval enhancement generation model to match the security issues with the security knowledge base. After matching, automatic correction and code optimization are performed according to the security knowledge base, and after automatic correction and optimization, it returns to reflective verification until the security assessment is passed. Finally, in the output stage, the final code is output, a security analysis report is generated, and the security knowledge accumulated during the reflection verification process is summarized, and the security knowledge base is updated and expanded.

[0041] It should be noted that in step S3 of this embodiment, after automatic correction and code optimization according to the security knowledge base, the system automatically returns to extract potential risks and conducts reflection verification. In the process of returning to extract potential risks, what is actually returned is the code production stage corresponding to step S1. Before the user's new input, there is no need to generate preliminary code again; at this time, the user input requirement is directly skipped, the potential risk is identified again, and the system jumps to step S2 for reflection verification in sequence, until the code optimization and security assessment of step S3 are passed, and then the system jumps to step S4 to output the final code and generate a security analysis report.

[0042] Therefore, in each stage of the code generation process, this embodiment performs automatic correction, code optimization and reflective verification through its own reflection mechanism, rather than performing simple error correction after generating the task / output code as in the prior art, to ensure that each output result meets the security requirements; on this basis, it can also self-generate and expand the security knowledge base in this process, so that the self-generated security knowledge base and the self-reflection mechanism form a mutually supportive and promoting relationship, without the need to reference an external knowledge base, and no longer limited to a specific model or task, which can well meet the universality requirements between different models and provide a better foundation for meeting the security requirements of large-scale, real-time and automated generation.

[0043] Specifically, this application introduces a self-reflection mechanism so that the large model (i.e., large language model) can automatically identify and repair potential security vulnerabilities, including potential risks and security issues, at each stage of the generation process, and generate security-related knowledge through self-summary and condensation. Different from traditional technical solutions that rely on external security knowledge bases or manual intervention, the security mechanism of this embodiment is based on the generation process within the model, and can identify security issues in real time, generate security rules, coding specifications, and security knowledge, etc., to form a self-generated security knowledge base for the large model, and apply it in subsequent generation tasks.

[0044] It is worth noting that the security mechanism of this embodiment does not rely on an external knowledge base, but at each stage of the generation process, it is accompanied by its own reflection mechanism to self-generate and maintain its security knowledge base. Therefore, it can effectively avoid the interference of irrelevant or noise external information on the model, and at the same time reduce the dependence on external security resources, with higher independence and flexibility. Through deep reflection and knowledge condensation of model generation content, this embodiment can achieve self-optimization of large models during the generation process and improve the security of generation tasks. The core advantage of this technical solution lies in its "out-of-the-box" capability, that is, plug-in security enhancement methods and systems, which can be applied to various large models without special fine-tuning or training, and are no longer limited to a specific model or task, thereby greatly simplifying the implementation process of security protection and improving the universal applicability and security of large model applications.

[0045] In practical applications, this general plug-in security enhancement method and system is particularly important, especially in areas such as government and enterprises that have extremely high requirements for data security. The content generated by large models, such as text, code, etc., often involves sensitive data and information. If there is a lack of sufficient security protection, it may cause serious security risks. For example, the generated code may contain unencrypted sensitive information or unverified user input, leading to data leakage or attack risks. Therefore, this embodiment can perform real-time security review and repair of the content generated by the large model to ensure that its output does not contain potential risks and security vulnerabilities, and has no limitations on use, which will be of great significance to protecting data security and improving the credibility and reliability of large model applications.

[0046] The core of this embodiment lies in the mutually collaborative and promoting large model self-reflection mechanism and self-generated security knowledge base, which can improve the security of generated content through the large model's own generation tasks and self-reflection mechanism without relying on an external knowledge base.

[0047] Specifically, step S1 in this embodiment includes the following sub-steps: Step S101, parsing the user's input requirements through a large language model, and generating preliminary codes according to the natural language description; Step S102, analyze each generated preliminary code through a large language model, extract security-related knowledge, and obtain potential risks in the preliminary code; security-related knowledge includes security vulnerabilities, attack methods, security rules and coding specifications, referred to as security knowledge.

[0048] In step S102 described in this embodiment, for security-related knowledge, security vulnerabilities corresponding to code patterns are identified through a large language model, and the security issues corresponding to the security vulnerabilities include but are not limited to SQL injection, XSS attack, cross-site request forgery, buffer overflow, etc.; potential attack methods (such as SQL injection, XSS attack, etc.) are identified, and the corresponding security rules and coding specifications are synchronously stored in the security knowledge base; for example, for SQL injection vulnerabilities, the extracted security rule may be "use parameterized queries", and the corresponding coding specification may be "when writing SQL statements, avoid directly splicing user input, but use parameterized method to pass user input", etc.

[0049] In this embodiment, the self-reflection mechanism is one of the cores, and its purpose is to enable the big model to perform real-time security review and optimization when generating content. At the beginning of the generation task, the big model generates output content based on user input, which may be in the form of text, code, etc. The generated content will enter a security review stage, which comprehensively evaluates the generated content and identifies potential security risks.

[0050] For example, when generating code, the model may find that the generated code contains sensitive data (such as API keys, private information, etc.) or operations that do not comply with security specifications. At this time, the self-reflection mechanism will be activated to analyze the possible sources of risk in the generated content and then correct them. This process can help large models automatically avoid potential security vulnerabilities such as leaking sensitive information and bypassing security controls when generating content.

[0051] If problems are found in the content after the first generation, the large model will further reflect and correct the output content to ensure that it meets security standards. For example, if unencrypted password information is found in the generated code, the self-reflection mechanism will automatically encrypt the password or delete sensitive information, and the corrected content will be submitted to the security detection module for analysis and evaluation again.

[0052] Specifically, step S2 in this embodiment is used to implement the reflection verification stage, including the following sub-steps: Step S201, start the security detection module, analyze the preliminary code and its potential risks through static analysis tools, and identify potential security issues; Step S202: classify and mark potential security issues based on the security knowledge in the security knowledge base, and provide targeted repair suggestions or alternative solutions in combination with domain knowledge and best practices.

[0053] In step S202 of the present embodiment, when generating codes involving database interaction, the large language model avoids the security issue of SQL injection through parameterized query or ORM framework; when generating Web application code, the large language model will remind the user to verify and escape the input requirements to avoid the security issue of XSS attack; in the task generation process, the length of the generated data is limited, and the data format is checked using regular expressions or other verification mechanisms. When accessing the array, it is ensured that the index value is within the valid range of the array, and when using dynamically allocated memory, it is ensured that the operation does not exceed the allocated memory range, and strncpy(), strncat(), fgets() and other security functions are used for data processing and calling; in addition, the large language model is used to dynamically adjust the security policy according to the domain corresponding to the task scenario and in combination with domain knowledge to ensure that the generated content complies with the latest security rules, coding specifications and industry standards.

[0054] As the model continuously generates new content and reflects on it, the knowledge in the security knowledge base will be continuously updated and expanded. After each task is generated, the model will analyze the generated content, identify new potential security issues, and add corresponding security rules and coding standards to the security knowledge base to keep it updated and timely.

[0055] In future generation tasks, this embodiment can call the content in the security knowledge base to quickly identify and avoid potential security issues. For example, when generating a piece of API call code, the model can refer to the specifications in the security knowledge base to ensure that the generated code does not leak sensitive information or contain potential attack vulnerabilities.

[0056] Step S3 in this embodiment is used to implement the security optimization stage, and includes the following sub-steps: Step S301, calling the correction module to perform code optimization, dynamically searching the security knowledge base based on the RAG retrieval enhancement generation model, matching the security problem with the security knowledge base, calculating the vector similarity between the security problem and each knowledge item in the security knowledge base, and obtaining the knowledge item whose vector similarity with the security problem is greater than the preset similarity threshold as the matching knowledge item; the preset similarity threshold refers to a pre-set similarity threshold, which can be set and adjusted according to the actual situation and needs; Step S302, inputting the matched knowledge items as context information into the correction module to generate code correction suggestions for security issues; Step S303, automatically optimizing the code according to the generated code correction suggestions; Step S304, return to step S2, and perform reflection verification on the content after each generation. If the generated code contains unsafe function calls or hard-coded passwords, it will be automatically replaced or deleted to ensure that the generated content meets security standards; if the generated code contains unencrypted password information, the password will be automatically encrypted or sensitive information will be deleted through its own reflection mechanism; until the content after reflection verification passes the security assessment. The security assessment of the content after reflection verification includes static assessment and dynamic assessment. The static assessment includes SonarQube and Pyroscope code security verification tools. The dynamic assessment mainly uses a large model to perform a security check on the generated code to determine whether there are common vulnerability codes in the code paradigm. If both the static assessment and the dynamic assessment pass, it is determined that the content after reflection verification has passed the security assessment; otherwise, it will automatically return to extract potential risks and perform reflection verification, and repeat in this way.

[0057] The security detection and optimization process implemented by steps S2 to S3 in this embodiment exists in each generation task. After the content is initially generated, this embodiment will conduct a security assessment on it through a large model. This assessment not only includes checking whether the content has sensitive information leakage, malicious code, and privacy infringement, but also conducts a comprehensive review of whether the content complies with industry security specifications.

[0058] Once a potential security issue is detected, the system will start an automated correction mechanism to optimize through a self-generated security knowledge base. For example, if the generated code contains unsafe function calls or hard-coded passwords, they will be automatically replaced or deleted to ensure that the generated content meets security standards. This process not only improves the security of the generated content, but also enhances the model's sensitivity to potential risks, enabling it to perform security optimization in real-time generation tasks.

[0059] In step S303 of this embodiment, during the automatic optimization process, adjustments are also made based on user feedback. If the user objects to the security of the generated content or discovers new security risks, the content is reflected and verified again through the large language model to ensure that the output results meet security requirements. Through this feedback mechanism, the security of the model is continuously improved, ensuring that the final output content is always in a safe state.

[0060] In step S303 of this embodiment, after the code is automatically optimized, verification and permission control are automatically added to the optimized code to further improve the reliability of the security optimization stage and reduce the possibility of reflection verification failure.

[0061] Step S4 in this embodiment is used to implement the output stage, and includes the following sub-steps: Step S401, after completing code optimization and reflection verification, output the final code; Step S402, listing the security issues found in the code optimization and reflection verification process, displaying the optimized code snippets, marking the comparison before and after the optimization, and scoring the code optimization according to the preset scoring criteria to generate a security analysis report; Step S403, summarize the security knowledge accumulated in each code optimization and reflection verification process, associate security issues, optimization solutions and best practices, update and expand the security knowledge base, and implement and maintain its own reflection mechanism through a large language model.

[0062] That is to say, after each task is generated, this embodiment will summarize the security knowledge accumulated during the generation process to form a self-generated security knowledge base. The self-generated security knowledge base contains information about different types of security vulnerabilities, coding standards, and security rules. This security knowledge base is not static, but is expanded and updated as the model generation task continues. Whenever the model finds new potential risks or new security knowledge in the generation task, it will automatically add it to the knowledge base. This is one of them.

[0063] This self-generated security knowledge base is highly scalable and adaptive. When the model performs a new generation task, it can extract relevant security knowledge from the knowledge base and apply it to the current task. For example, when generating API call code, the model can refer to the specifications in the security knowledge base to ensure that the generated code does not contain possible attack surfaces or code patterns that do not meet security requirements.

[0064] Secondly, the biggest advantage of the security knowledge base of this embodiment is that it does not rely on external resources and is completely generated and maintained by the large model itself. This enables the large model to independently expand its security capabilities without external intervention and continuously enhance the security of the generated tasks. Therefore, this embodiment not only provides security guarantees for the widespread application of large models, but also provides strong technical support for application scenarios that require high security (such as finance, medical care, government, etc.).

[0065] This embodiment also provides a plug-in security enhancement system based on a large model self-reflection mechanism, which adopts the plug-in security enhancement method based on a large model self-reflection mechanism as described above, and includes: The code generation module receives user input requirements in real time and generates preliminary code through a large language model. It first uses regular expressions and static analysis tools based on rule matching to identify sensitive data, and uses a large language model to extract potential risks in the preliminary code. The potential risks include unencrypted password information, sensitive data, and operations that do not comply with security rules or coding specifications. The sensitive data includes API keys, passwords, and private information. The reflection and verification module starts the security detection module to perform reflection and verification, identify potential security issues, and mark the security issues through the security knowledge base self-generated by the large language model; The security optimization module dynamically searches the security knowledge base, generates repair suggestions, and calls the correction module to perform code optimization. It dynamically searches the security knowledge base based on the RAG retrieval enhancement generation model, matches the security issues with the security knowledge base as reference content, and performs automatic correction and code optimization according to the security knowledge base after matching, and returns to extract potential risks and perform reflection verification; The output module outputs the final code, generates a security analysis report, summarizes the security knowledge accumulated during the reflection and verification process, and updates and expands the security knowledge base.

[0066] To sum up, first, this embodiment can self-examine and reflect on the content generated by the large model through a self-reflection mechanism, so that it can actively identify potential security vulnerabilities and security issues in the generated content, and make real-time corrections. It should be noted that the self-reflection mechanism of this embodiment is not like the prior art, which performs simple error correction after generating tasks / output codes, but performs continuous security assessments at each stage of generating content to ensure that each generated output meets security standards. This plug-in security enhancement method and system not only improves the security in the generation task, but also enhances the security and reliability of large models in practical applications.

[0067] Second, the large model continuously accumulates and expands security knowledge through a self-generated security knowledge base. This security knowledge base not only covers common security vulnerabilities, coding specifications, and security rules, but can also automatically summarize new security knowledge from the generation tasks as the large model generation tasks are continuously carried out, forming a sustainably updated security knowledge system. Especially without the support of an external knowledge base, this embodiment can still generate security knowledge through its own reflection and learning to adapt to the security requirements of different tasks. Through this self-generated and self-optimizing security mechanism, the risk of security vulnerabilities in large models in actual applications can be significantly reduced, especially in industries that require a high degree of data security, and has important application value.

[0068] Third, add an automated security detection and correction mechanism to the generation process to ensure that the large model can identify and fix security issues in the output in real time, so that a comprehensive check can be performed on the content after it is generated, and risk points such as sensitive data leakage, malicious code, and privacy infringement in the content can be discovered in a timely manner, and corresponding measures can be taken to correct them.

[0069] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A plug-in security enhancement method based on a large model self-reflection mechanism, characterized in that: The following steps are involved: Step S1, receiving user input requirements in real time, generating preliminary code through a large language model, first identifying sensitive data based on rule matching using regular expressions and static analysis tools, and extracting potential risks in the preliminary code through a large language model, the potential risks including unencrypted password information, sensitive data, and operations that do not comply with security rules or coding specifications, the sensitive data including API keys, passwords, and private information; Step S2, starting the security detection module to perform reflection verification, identify potential security issues, and mark the security issues through the security knowledge base self-generated by the large language model; the security issues include SQL injection, XSS attack and buffer overflow; Step S3, dynamically search the security knowledge base, generate repair suggestions, and call the correction module to optimize the code. The security knowledge base is dynamically searched based on the RAG retrieval enhancement generation model, and the security issues are matched with the security knowledge base as reference content. After matching, automatic correction and code optimization are performed according to the security knowledge base, and potential risks are extracted and reflected verification is performed; Step S4, output the final code, generate a security analysis report, summarize the security knowledge accumulated during the reflection and verification process, and update and expand the security knowledge base; The content of the security analysis report includes security issues, optimized codes and scores.

2. The plug-in security enhancement method based on large model self-reflection mechanism according to claim 1 is characterized in that: The step S1 comprises the following sub-steps: Step S101, parsing the user's input requirements through a large language model, and generating preliminary codes according to the natural language description; Step S102, analyze each generated preliminary code through a large language model, extract security-related knowledge, and obtain potential risks in the preliminary code; security-related knowledge includes security vulnerabilities, attack methods, security rules and coding specifications.

3. The plug-in security enhancement method based on large model self-reflection mechanism according to claim 2 is characterized in that: In step S102, for security-related knowledge, security vulnerabilities corresponding to code patterns are identified through a large language model, potential attack methods are identified, and corresponding security rules and coding specifications are synchronously stored in the security knowledge base.

4. The plug-in security enhancement method based on the large model self-reflection mechanism according to any one of claims 1 to 3 is characterized in that: The step S2 comprises the following sub-steps: Step S201, start the security detection module, analyze the preliminary code and its potential risks through static analysis tools, and identify potential security issues; Step S202: classify and mark potential security issues based on the security knowledge in the security knowledge base, and provide targeted repair suggestions or alternative solutions in combination with domain knowledge and best practices.

5. The plug-in security enhancement method based on large model self-reflection mechanism according to claim 4 is characterized in that: In step S202, when generating codes involving database interaction, the large language model avoids the security issue of SQL injection through parameterized query or ORM framework; when generating Web application code, the large language model will remind the user to verify and escape the input requirements to avoid the security issue of XSS attack; Through the large language model, security policies are dynamically adjusted according to the domain corresponding to the task scenario and combined with domain knowledge to ensure that the generated content complies with the latest security rules, coding specifications and industry standards.

6. The plug-in security enhancement method based on large model self-reflection mechanism according to any one of claims 1 to 3, characterized in that: The step S3 comprises the following sub-steps: Step S301, calling the correction module to perform code optimization, dynamically searching the security knowledge base based on the RAG retrieval enhancement generation model, matching the security problem with the security knowledge base, calculating the vector similarity between the security problem and each knowledge item in the security knowledge base, and obtaining the knowledge item whose vector similarity with the security problem is greater than a preset similarity threshold as the matching knowledge item; Step S302, inputting the matched knowledge items as context information into the correction module to generate code correction suggestions for security issues; Step S303, automatically optimizing the code according to the generated code correction suggestions; Step S304, return to step S2, and perform reflection verification on the generated content each time. If the generated code contains unsafe function calls or hard-coded passwords, they are automatically replaced or deleted to ensure that the generated content meets the security standards; if the generated code contains unencrypted password information, the password is automatically encrypted or sensitive information is deleted through its own reflection mechanism; Until the content passes the safety assessment after reflection and verification.

7. The plug-in security enhancement method based on large model self-reflection mechanism according to claim 6 is characterized in that: In step S303, during the automatic optimization process, adjustments are also made based on user feedback. If the user objects to the security of the generated content or discovers new security risks, the content is reflected and verified again through the large language model to ensure that the output results meet security requirements.

8. The plug-in security enhancement method based on large model self-reflection mechanism according to claim 6 is characterized in that: In step S303, after the code is automatically optimized, verification and permission control are automatically added to the optimized code.

9. The plug-in security enhancement method based on large model self-reflection mechanism according to any one of claims 1 to 3, characterized in that: The step S4 comprises the following sub-steps: Step S401, after completing code optimization and reflection verification, output the final code; Step S402, listing the security issues found in the code optimization and reflection verification process, displaying the optimized code snippets, marking the comparison before and after the optimization, and scoring the code optimization according to the preset scoring criteria to generate a security analysis report; Step S403, summarize the security knowledge accumulated in each code optimization and reflection verification process, associate security issues, optimization solutions and best practices, update and expand the security knowledge base, and implement and maintain its own reflection mechanism through a large language model.

10. A plug-in security enhancement system based on a large model self-reflection mechanism, characterized in that: The plug-in security enhancement method based on the large model self-reflection mechanism as described in any one of claims 1 to 9 is adopted, and includes: The code generation module receives user input requirements in real time and generates preliminary code through a large language model. It first uses regular expressions and static analysis tools based on rule matching to identify sensitive data, and uses a large language model to extract potential risks in the preliminary code. The potential risks include unencrypted password information, sensitive data, and operations that do not comply with security rules or coding specifications. The sensitive data includes API keys, passwords, and private information. The reflection and verification module starts the security detection module to perform reflection and verification, identify potential security issues, and mark the security issues through the security knowledge base self-generated by the large language model; The security optimization module dynamically searches the security knowledge base, generates repair suggestions, and calls the correction module to perform code optimization. It dynamically searches the security knowledge base based on the RAG retrieval enhancement generation model, matches the security issues with the security knowledge base as reference content, and performs automatic correction and code optimization according to the security knowledge base after matching, and returns to extract potential risks and perform reflection verification; The output module outputs the final code, generates a security analysis report, summarizes the security knowledge accumulated during the reflection and verification process, and updates and expands the security knowledge base.

Images