A method to automatically analyze POC and develop rules through large models

Abstract

The present invention relates to a method for automatically analyzing POC and developing rules through a large model. The method includes: monitoring Internet data through a crawler, obtaining vulnerability and rule information, and building a vulnerability intelligence library and a rule intelligence library; the intelligence library outputs demand, uses a large model to link with the crawler, obtains the analysis result of the demand after the large model optimizes the retrieval, and outputs POC information; the large model designs and develops rules based on the POC information, and performs rule testing on the developed rules; for the successfully tested rules, data is obtained from the large model vector knowledge base to complete the metadata information of the rule, and finally the complete rule is output to the rule library. The method solves the problem of rule demand analysis and rule development relying heavily on manual labor in the traditional rule development process, improves the rule development efficiency, reduces the rule development cost, and also improves the speed of rule adaptation after the latest threat is released, thereby ensuring real-time update and complete coverage of the rules.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic analysis and rule development of POC, and in particular to a method for automatically analyzing POC and developing rules through a large model. Background Art

[0002] In the field of network security, including the protection of Web or other software applications, the capabilities of security products such as Web Application Firewall (WAF), Firewall (FW), Vulnerability Scanner, etc. are greatly dependent on their built-in rule bases. The quality and coverage of the rule base directly determine whether the security product can effectively defend against attacks and vulnerability exploitation attempts. Currently, the core of security capabilities comes from the development and implementation of these rules. The development of rules mainly focuses on the collection and use of vulnerability or attack exploitation examples (POC), which is the key link in building an effective defense mechanism.

[0003] However, the traditional rule development process faces huge challenges. The traditional method mainly relies on manually collecting and organizing POCs, and then developing rules based on this information. This process is inefficient and particularly cumbersome in the face of growing security threats. In addition, due to the continuous emergence of new vulnerabilities and attack techniques, security teams need to invest a lot of time and resources to manually follow up and analyze the latest attack trends, which greatly limits the speed and efficiency of responding to new threats.

[0004] In addition, due to human resource limitations, this traditional approach is difficult to achieve comprehensive coverage of a large number of ever-changing POCs. The security team may not be able to obtain all relevant POC information immediately, resulting in an incomplete or outdated rule base, which reduces the defense capabilities of security products. At the same time, the process of manually analyzing POCs and developing rules is also very time-consuming and is likely to introduce inaccurate or inefficient rules, affecting the overall security defense effect. Summary of the invention

[0005] In order to overcome the above-mentioned defects of existing POC collection and analysis and traditional rule development methods, the present invention proposes a new method for automatically analyzing POC and developing rules through a large model. The method of the present invention can automatically collect and analyze POC information, and use the powerful processing power and intelligent analysis of the large model to quickly and accurately generate defense rules. This method can not only greatly improve the efficiency and response speed of rule development, but also continuously improve the rules through self-learning and adaptation, improve the quality, coverage and update timeliness of security rules, and effectively improve the security protection capabilities of Web applications while reducing the cost of security rule development and human resource investment.

[0006] The method of the present invention monitors Internet data to generate a vulnerability intelligence library and a rule intelligence library, then links the crawler and the big model module, drives the target with the big model, optimizes the retrieval capability, obtains relevant results analysis and outputs POC information (this method does not use the knowledge of the big model itself to answer questions, output POC information, etc., effectively avoiding the problems of slow update time of big model data, answer illusion, etc.), and then uses the big model to design and develop rules based on the above POC information. At the same time, the developed rules are connected to the automated testing system. If the test fails, the detailed reasons are fed back to the big model optimization rules. Finally, the rule development is completed and the metadata information of the data completion rule is obtained from the big model vector knowledge base, and the complete rule is output to the rule base.

[0007] Specifically, the present invention provides a method for automatically analyzing POC and developing rules through a large model, and the method comprises:

[0008] S1. Monitor Internet data through crawlers to obtain vulnerability and rule information, and build vulnerability intelligence database and rule intelligence database based on the information obtained;

[0009] S2. The vulnerability intelligence database and rule intelligence database output requirements, use the big model to work with the crawler, obtain the analysis results of the requirements after the big model optimizes the retrieval, and outputs the POC information;

[0010] S3. Use the big model to design and develop rules based on the POC information output in the previous step, and test the developed rules. If the test fails, the failure reason is fed back to the big model for rule optimization. The optimized rules then re-enter the rule testing phase;

[0011] S4. For the successfully tested rules, the big model AiAgent capability is used to obtain data from the big model vector knowledge base to complete the metadata information of the rule, and finally the complete rule is output to the rule base.

[0012] Furthermore, the method of automatically analyzing POC and developing rules by a large model in the present invention includes monitoring Internet data by crawlers in step S1 to obtain vulnerability and rule information, including:

[0013] Monitor GitHub Advisory Database, cve.mitre.org, CNNVD, and public accounts to obtain vulnerability information;

[0014] Monitor the product consoles, rules, and announcement documents of security vendors to obtain their newly added security rule information;

[0015] Monitor Github open source project rules and obtain updated information on the security capabilities of its open source products.

[0016] Furthermore, the present invention uses a large model to automatically analyze the rule intelligence library constructed in step S1 of the method for developing rules. The rules in the library only include the name, description or risk resolved by the rule.

[0017] Furthermore, the method step S2 of the present invention for automatically analyzing POC and developing rules through a large model includes:

[0018] S21. Output requirements for vulnerability intelligence database and rule intelligence database;

[0019] S22. For the topic corresponding to the demand, the big model generates a specified number of search keywords and searches them in the search engine, and the big model optimizes the returned results;

[0020] S23. Using the intelligence crawler module to crawl the search keywords after the large model optimization process;

[0021] S24. For the results obtained in the previous crawling step, the large model is called to determine whether the crawling results contain the POC information necessary for rule development. If not, the intelligence crawler module continues to judge the next search result until the analysis of all search results is completed or the analysis limit set by the user is reached;

[0022] S25. For the POC information necessary for rule development contained in the crawling results, a large model with a large number of parameters is called to understand its content, and then prompt words are used to let the large model output the POC information based on the format required by the user.

[0023] Furthermore, the search engines described in step S22 of the method for automatically analyzing POC and developing rules by a large model of the present invention include but are not limited to: Github, Google;

[0024] The large model is an open source large model with a large number of parameters or a commercial large model API.

[0025] Furthermore, the intelligence crawler module described in step S23 of the method for automatically analyzing POC and developing rules through a large model of the present invention is a plug-in crawling module, which has a built-in Github code search module and a search engine API.

[0026] Furthermore, the large model described in step S24 of the method for automatically analyzing POC and developing rules by a large model of the present invention is a large model with a small number of parameters;

[0027] The POC information necessary for the rule development includes vulnerability analysis reports and rule files of other scanners.

[0028] Furthermore, the method of automatically analyzing POC and developing rules by a large model in the present invention performs rule testing on the developed rules as described in step S3, including:

[0029] (1) Check the legality of the syntax of the rule itself;

[0030] (2) Test the validity of the rules: Use the POC output by the big model to send packets, and the rules developed by the big model are used for detection. If an alarm is reported, the blacklist test passes, otherwise it returns failure. After the blacklist test passes, continue with the whitelist test. There is a fixed test set in the whitelist test. The legal requests contained in the test set should not be matched by the rules. If the rules developed by the big model are matched, it returns failure and returns the matching message information.

[0031] Furthermore, the metadata information of the rules described in step S4 of the method for automatically analyzing POC and developing rules through a large model of the present invention includes: rule name, rule description, rule detection product name, rule detection product description, protection suggestions, and repair version.

[0032] Furthermore, the vulnerability intelligence base and rule intelligence base in the method of automatically analyzing POC and developing rules by a large model, the large model optimization retrieval result information, the large model analysis output results and POC information, and all rules in the rule base are written into the large model vector knowledge base through the embedding model;

[0033] The large model vector knowledge base includes: a vulnerability metadata vector database, a vulnerability POC data vector database, and a vulnerability rule data vector database.

[0034] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the method for automatically analyzing POC and developing rules through a large model are implemented.

[0035] In summary, the present invention designs a method for automatically collecting rule development requirements and completing rule requirement analysis through a large model in the method of automatically analyzing POC and developing rules through a large model, a method for generating POC based on rule development requirement analysis through a large model and completing rule development in this way, a method for using large models with different parameter quantities in different links of rule development to improve model efficiency and reduce overhead, and a method for constructing a large model vector knowledge base for the entire life cycle of rules. By combining crawlers with large models, the present invention solves the problems of rule requirement analysis and rule development relying heavily on manual labor in the traditional rule development process, improves the rule development efficiency, reduces the rule development cost, and also improves the speed of rule adaptation after the latest threat is released, thereby ensuring real-time updates and complete coverage of rules, and in this process, a set of knowledge bases for the entire life cycle of rules are formed, which effectively supports the subsequent maintenance and optimization of the rules. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the following drawings are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 The overall design architecture diagram of the method of the present invention.

[0038] Figure 2 It is a schematic diagram of the implementation process of the method of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. The present invention can also be implemented or applied through other different specific implementation methods. The details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0040] At the same time, it should be understood that the protection scope of the present invention is not limited to the specific embodiments described below; it should also be understood that the terms used in the embodiments of the present invention are for describing specific embodiments rather than for limiting the protection scope of the present invention.

[0041] Example: A method for automatically analyzing POC and developing rules through a large model

[0042] like Figure 1 and Figure 2 As shown, the method comprises the following steps:

[0043] 1. Build a vulnerability intelligence database and a rule intelligence database

[0044] Security rule development often relies on manual experience. In the conventional process, demand management is required, and the requirements are collected and analyzed by the client side or other channels. This process is not only very labor-intensive, but also difficult to cope with the increasing threats on the Internet (for example, vulnerability libraries such as GitHub Advisory Database have a large number of new vulnerabilities every day, and the workload is very huge if they rely on human judgment and analysis one by one). Therefore, the traditional rule development method is difficult to ensure the latest coverage of massive threats at the demand analysis level. The method of the present invention first uses crawlers to monitor GitHub AdvisoryDatabase, cve.mitre.org, CNNVD, public accounts, etc. to obtain vulnerability intelligence from the rule development demand level. Optionally, other intelligence sources are also supported to configure access, and a vulnerability intelligence library is constructed based on the vulnerability information obtained above; at the same time, product consoles, rules and announcement documents of security vendors such as Google are also monitored, so that the newly added security rule capabilities can be perceived. Open source project rules of channels such as Github are also monitored, and the update of open source product security capabilities can be obtained; finally, a rule intelligence library is constructed based on the above-mentioned information obtained. The rule intelligence in the library does not require rule content, but only requires the name, description or risk of the rule. Next, the big model will use this as the demand for analysis, replacing the traditional manual demand sorting work.

[0045] 2. Large Model Analysis Requirements

[0046] Based on the needs generated by the vulnerability and rule intelligence library, such as "GitLab arbitrary user password reset vulnerability (CVE-2023-7028)".

[0047] 1. Reconstruct search keywords for large models

[0048] Through the prompt word project, the big model security engineer is given the identity background to analyze the rule development requirements, and then the big model is required to generate a specified number of search keywords (the default is 3, which can be modified) for the analyzed topic to search Github, Google and other search engines. The big model processes and returns the results, for example:

[0049] [“CVE-2023-7028”,”GitLab arbitrary password reset”,”GitLab password reset vulnerability”]

[0050] Note that in this scenario, the input and output tokens are both small, and the result quality requirements are high. In this case, the large model should choose a large parameter model to ensure the effect without affecting the performance. The large model can choose an open source large model or a commercial large model API.

[0051] 2. Intelligence crawler module, which crawls intelligence based on search keywords processed by the large model. The crawler module is plug-in-based, with built-in support for Github code search module, Google and other search engine APIs, and also supports custom crawler modules.

[0052] 3. For example, for ["CVE-2023-7028", "GitLab arbitrary password reset", "GitLab password reset vulnerability"], each time a result is obtained, the large model is called (it can be a small parameter amount to save overhead. The user can configure the model of this step according to the actual situation). At this time, the call prompt word is designed to let the large model determine whether the crawled content contains the POC information necessary for rule development. For example, if the crawled result is the official CVE announcement information of CVE-2023-7028, the large model will judge no. If the crawled result is a complete vulnerability analysis report of CVE-2023-7028 or other scanner rules, the large model will return yes. If not, the intelligence crawler module will continue to traverse and analyze the search keywords ["CVE-2023-7028", "GitLab arbitrary password reset", "GitLab password reset vulnerability"] on various platforms until all analyses are completed or the user-configured upper limit is reached (for example, CVE-2023-7028 can search for 1,000 results in Github code search, then each result needs to be determined by the large model to determine whether it meets the analysis requirements until it is met).

[0053] 4. After obtaining the material information that satisfies the POC analysis, a large model with a large number of parameters is needed to understand the content POC. At this time, the material information may be a rule file of an open source scanner or a vulnerability analysis report. At this time, the prompt words are used to let the large model output the POC based on the format required by the user (for example, HTTP vulnerabilities use the HTTP network packet format, etc.). At this time, the large model can also autonomously call Github code search and search engine API capabilities based on the AIAgent method to further complete other required information. The entire process is guided by prompt words. The large model must summarize based on existing information and cannot answer or fabricate based on its own knowledge. After testing, it can effectively solve problems such as hallucinations of the large model (for example, if you ask the large model a POC of a CVE vulnerability that it has not trained, it may fabricate it).

[0054] In the above process, the intelligence library, large model optimization retrieval result information, large model analysis output results and POC involved are all written into the large model vector knowledge base through the embedding model. The vector knowledge base provides cache context for the development of large model rules and can be used by users as a knowledge base for development optimization.

[0055] 3. Large Model Rule Development and Testing

[0056] Through a large model with a large number of parameters, security rules are developed based on existing information, such as WAF protection rules, application vulnerability scanning rules, etc. According to different rule scenarios, different roles, development manuals and development examples of the large model can be provided.

[0057] The rules developed by the large model need to be tested, such as WAF rules:

[0058] 1. Rule testing focuses on checking the syntax validity of the rules themselves. If the developed rule syntax is wrong, the corresponding error information will be returned.

[0059] 2. Rule validity test. Taking WAF rules as an example, the test service provides a test interface and uses the POC output by the big model to send packets. The WAF rules developed by the big model are tested. If an alarm is reported, the blacklist test passes. If it fails, it returns failure. After the blacklist test passes, the whitelist test will continue. In the whitelist test, there is a fixed test set, such as 100,000 legal requests, which should not be matched by the rules. If the rules developed by the big model are matched, it fails and returns the matching message information.

[0060] For test failures, the key information of the test failure is returned and then given to the large model for rule optimization. The optimized rules then re-enter the rule testing phase and the cycle continues (the user can configure the upper limit of the number of times).

[0061] For successfully tested rules, the big model AiAgent capability is used to call the big model vector knowledge base to obtain basic information such as rule metadata to develop complete rules, including rule name, rule description, rule detection product name, rule detection product description, protection suggestions, repair version, etc.

[0062] The improved rules are finally merged into the rule base. All rules in the rule base are also embedded into the large model vector knowledge base.

[0063] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any technician familiar with the profession can make some changes or modifications to the technical contents disclosed above without departing from the scope of the technical solution of the present invention to obtain equivalent embodiments of equivalent changes. However, any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for automatically analyzing POC and developing rules through a large model, characterized in that: The method comprises: S1. Monitor Internet data through crawlers to obtain vulnerability and rule information, and build vulnerability intelligence database and rule intelligence database based on the information obtained; S2. The vulnerability intelligence database and rule intelligence database output requirements, use the big model to work with the crawler, obtain the analysis results of the requirements after the big model optimizes the retrieval, and outputs the POC information; including: S21. Output requirements for vulnerability intelligence database and rule intelligence database; S22. For the topic corresponding to the demand, the big model generates a specified number of search keywords and searches them in the search engine, and the big model optimizes the returned results; S23. Using the intelligence crawler module to crawl the search keywords after the large model optimization process; S24. For the results obtained in the previous crawling step, the large model is called to determine whether the crawling results contain the POC information necessary for rule development. If not, the intelligence crawler module continues to judge the next search result until the analysis of all search results is completed or the analysis limit set by the user is reached; S25. For the POC information necessary for rule development contained in the crawled results, a large model with a large number of parameters is called to understand its content, and then prompts the large model to output the POC information based on the format required by the user; S3. Use the big model to design and develop rules based on the POC information output in the previous step, and test the developed rules. If the test fails, the failure reason is fed back to the big model for rule optimization. The optimized rules then re-enter the rule testing phase; S4. For the successfully tested rules, data is obtained from the large model vector knowledge base to complete the metadata information of the rule, and finally the complete rule is output to the rule base.

2. The method for automatically analyzing POC and developing rules through a large model according to claim 1, characterized in that: The step S1 of monitoring Internet data by crawling to obtain vulnerability and rule information includes: Monitor GitHub Advisory Database, cve.mitre.org, CNNVD, and public accounts to obtain vulnerability information; Monitor the product consoles, rules, and announcement documents of security vendors to obtain their newly added security rule information; Monitor Github open source project rules and obtain updated information on the security capabilities of its open source products.

3. The method for automatically analyzing POC and developing rules through a large model according to claim 1, characterized in that: The rule intelligence library constructed in step S1 contains only the name, description or resolved risk of the rule.

4. The method of automatically analyzing POC and developing rules through a large model according to claim 1, characterized in that: The search engines described in S22 include: Github, Google; The large model is an open source large model with a large number of parameters or a commercial large model API.

5. The method of automatically analyzing POC and developing rules through a large model according to claim 1, characterized in that: The intelligence crawler module described in S23 is a plug-in crawling module, which has a built-in Github code search module and search engine API.

6. The method of automatically analyzing POC and developing rules through a large model according to claim 1, characterized in that: The large model described in S24 is a large model with a small number of parameters; The POC information necessary for the rule development includes vulnerability analysis reports and rule files of other scanners.

7. The method of automatically analyzing POC and developing rules through a large model according to claim 1, characterized in that: Conduct rule testing on the developed rules as described in S3, including: (1) Check the legality of the syntax of the rule itself; (2) Test the validity of the rules: Use the POC output by the big model to send packets, and use the rules developed by the big model to detect. If an alarm is reported, the blacklist test passes, otherwise it returns failure. After the blacklist test passes, continue with the whitelist test. There is a fixed test set in the whitelist test. The legal requests contained in the test set should not be matched by the rules. If the rules developed by the big model are matched, it returns failure and returns the matching message information.

8. The method of automatically analyzing POC and developing rules through a large model according to claim 1, characterized in that: The metadata information of the rule described in S4 includes: rule name, rule description, rule detection product name, rule detection product description, protection suggestion, and repair version.

9. The method of automatically analyzing POC and developing rules through a large model according to claim 1, characterized in that: In the method, the vulnerability intelligence base and rule intelligence base, large model optimization retrieval result information, large model analysis output results and POC information, and all rules in the rule base are written into the large model vector knowledge base through the embedding model; The large model vector knowledge base includes: a vulnerability metadata vector database, a vulnerability POC data vector database, and a vulnerability rule data vector database.

Images