Network security auxiliary system based on large model and construction method

Abstract

The invention belongs to the technical field of network security, and provides a network security auxiliary system based on a large model and a construction method, the auxiliary system can utilize a natural language processing technology to deeply understand network attack logs, user query and security policies, and intelligent analysis and response are realized. By training a large model, the method has the capability of identifying unknown threats and variation attacks, so that the limitation of a traditional method based on static rules and signature matching is overcome. By utilizing the system, accurate answers can be given according to questions of users, and disposal suggestions can be automatically generated. The network security large model is internally connected with a Markdown processing unit, and a network security auxiliary system is stipulated to output a Markdown format, so that powerful support is provided for security decision making.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and in particular relates to a network security auxiliary system based on a large model and a construction method. Background Art

[0002] With the continuous development of information technology, the means of network attacks are becoming more and more innovative, and the field of network security is facing increasingly severe challenges. Traditional security protection measures often rely on static rules and signature matching, which are difficult to adapt to rapidly changing network threats. At the same time, the massive amount of network data has put forward higher requirements for security analysts. As network security threats show a trend of diversification and complexity, the rapid acquisition and accurate application of network security expertise are becoming more and more critical for practitioners. Therefore, it is particularly important to develop an intelligent network security auxiliary system to assist security experts in threat detection and response.

[0003] Most of the existing open source network security assistance systems do not use natural language processing technology, but often rely on static rules and signature matching. For example, when encountering a network attack, they simply match the intelligence library with the attack log by retrieving the local threat intelligence library to find out whether there is a corresponding explanation and solution, or they use manual methods to manually analyze the status and solution of the attack log. The problems with this solution are:

[0004] 1. Limited detection capabilities for unknown threats: Signature- and static rule-based systems often have difficulty identifying new or mutated threats.

[0005] 2. Insufficient flexibility and adaptability: Static rules and signature libraries need to be updated regularly, which may lead to a lag in protection measures in a rapidly changing threat environment.

[0006] 3. High resource consumption: Manual analysis requires a lot of time and professional skills, which can be a challenge for organizations with limited resources.

[0007] 4. Slow response: After a threat is detected, manual response usually takes time to evaluate and execute, which may allow the threat to use this time to cause damage.

[0008] 5. Reliance on expert experience: The efficiency and accuracy of manual analysis are highly dependent on the experience and knowledge of security experts, which may lead to inconsistency in analysis results.

[0009] In addition, many open source network security assistance systems exist as separate question-and-answer systems, which can only handle some knowledge questions and answers in the field of network security, and cannot generate reports and give treatment suggestions based on user descriptions. The actual application value of such network security assistance systems is low. Summary of the invention

[0010] In order to solve the technical problems of the existing network security assistance systems, such as insufficient detection capabilities for unknown network security threats, high resource consumption, slow response speed and reliance on expert experience, the present invention proposes a network security assistance system based on a large model and a construction method, which improves the timeliness and accuracy of information notification through the fusion and real-time processing of multi-source data, and effectively integrates information.

[0011] The specific plan is as follows:

[0012] A network security auxiliary system based on a large model, comprising:

[0013] Knowledge base construction module, used to build an updateable knowledge base based on the RAG framework;

[0014] The automated response module includes: a data receiving submodule for receiving user questions, a network security model obtained by fine-tuning the large model using network security data, and a data output submodule for outputting network security model answers and disposal suggestions; when generating answers or suggestions, the network security model retrieves relevant information from a knowledge base; the data output submodule also includes a markdown processing unit for the security report generated by the answer.

[0015] Preferably, the content of the security report includes: threat analysis, response measures and effect evaluation.

[0016] Preferably, the data receiving submodule includes an integrated voice recognition API to support users to ask questions via voice.

[0017] Preferably, the automated response module further comprises: a model optimization submodule for optimizing and fine-tuning the large model based on the user's query pattern and feedback.

[0018] Preferably, the markdown processing unit will answer the generated security report and stipulate that the network security model is output in markdown format.

[0019] Preferably, the system also includes a preprocessing module to process user questions in the data receiving submodule, including text cleaning, word segmentation, and keyword recognition, and finally output a formatted language that can be recognized by the large model.

[0020] Preferably, the network security big model includes: a classification and analysis unit that analyzes the intent and entity of the problem, classifies the problem, and determines whether it is a problem consultation type or a disposal suggestion type; a context management unit that performs context management on the current user's question; and a question and answer generation unit that calls the knowledge base, deep learning, and natural language technology to generate answers for different types of questions.

[0021] Preferably, the question and answer generating unit,

[0022] For question-asking questions, call the knowledge base, retrieve relevant information, and generate answers;

[0023] For problems involving disposal suggestions, deep learning and natural language processing technologies are used to analyze and identify the problems, and output situation analysis, disposal suggestions, and effect evaluation in markdown format.

[0024] A method for constructing a network security auxiliary system based on a large model.

[0025] S1: Build a fine-tuned network security big model: Deploy the big model locally and fine-tune the big model based on the network security corpus to build a network security big model;

[0026] S2: Build an updateable knowledge base based on the RAG framework: The RAG framework allows the cybersecurity big model to retrieve relevant information from the knowledge base when generating answers or suggestions; the knowledge base can supplement and update cybersecurity related content;

[0027] S3: Building an updateable network security big model: When answering user questions, the network security big model calls the knowledge base through the RAG framework to retrieve relevant information;

[0028] S4: Optimizing the network security big model based on user feedback: Users provide feedback based on the answers of the network security big model, and the network security big model is dynamically adjusted according to the feedback.

[0029] Preferably, the knowledge base includes: network security related documents from the Internet, news, industry regulations, and internal company documents.

[0030] Preferably, the network security big model generates a security report based on the markdown processing unit using the answers to the disposal suggestion type, and the output format is markdown. The security report includes: situation analysis, disposal suggestions and effect evaluation, and the user can choose to save the security report.

[0031] Beneficial effects:

[0032] The present invention proposes a network security auxiliary system and construction method based on a large model. The auxiliary system can use natural language processing technology to deeply understand network attack logs, user queries and security policies, and realize intelligent analysis and response. By training the large model, it has the ability to identify unknown threats and variant attacks, thereby overcoming the limitations of traditional methods based on static rules and signature matching. The system of the present invention can give accurate answers based on user questions and automatically generate disposal suggestions. The network security large model of the present invention is connected to a markdown processing unit, and stipulates that the network security auxiliary system outputs the markdown format, providing strong support for security decision-making.

[0033] The present invention designs a user-friendly interactive interface. It supports voice input. Through interactive controls and real-time feedback, the efficiency of user-system interaction is improved to ensure timely response to security incidents. At the same time, users can also rate the content generated this time to provide feedback for the continuous optimization of the system.

[0034] Secondly, the present invention provides auxiliary question consultation: the network security assistance system can answer various network security related questions, from basic security measures to complex technical issues, and can provide accurate information and suggestions. The intelligent question and answer scenario uses natural language processing technology to understand the user's query intention and provide relevant answers or solutions. This instant auxiliary consultation not only improves user satisfaction, but also strengthens the security defense capabilities of the entire organization, because employees can quickly obtain the knowledge needed to solve problems and take timely actions to prevent the occurrence or expansion of security incidents. In addition, the network security assistance system can also continuously learn and optimize according to the user's query patterns and feedback to provide more personalized and accurate consulting services.

[0035] Third, the system of the present invention has the ability to continuously learn and continuously optimize its performance based on new data and user feedback. This self-optimization mechanism enables the security assistant to provide increasingly accurate consulting services over time, staying at the forefront of the field of network security.

[0036] Fourth, the large model of the present invention is pre-trained in multiple languages ​​and multiple fields, which enables the system of this solution to not only support multiple languages, but also provide professional advice across different sub-fields of network security to meet the needs of different user groups.

[0037] Fifth, real-time knowledge update. By building a RAG framework, the network security assistant can update its knowledge base in real time to ensure rapid response to emerging threats. This greatly improves the flexibility and adaptability of the system, enabling it to continuously provide users with the latest network security support. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1A structural diagram of a network security assistance system based on a large model in an embodiment.

[0039] Figure 2 A flow chart of a method for constructing a network security assistance system based on a large model in an embodiment.

[0040] Figure 3 An operation flow chart of the network security assistant constructed based on the system in the embodiment.

[0041] Figure 4 A flowchart of building a network security assistant based on the system in the embodiment. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0043] like Figure 1 , a network security auxiliary system based on a large model, comprising:

[0044] Knowledge base construction module, used to build an updateable knowledge base based on the RAG framework;

[0045] The automated response module includes: a data receiving submodule for receiving user questions, a network security model obtained by fine-tuning the large model using network security data, and a data output submodule for outputting network security model answers and disposal suggestions; when generating answers or suggestions, the network security model retrieves relevant information from a knowledge base; the data output submodule also includes a markdown processing unit for the security report generated by the answer.

[0046] Preferably, the content of the security report includes: threat analysis, response measures and effect evaluation.

[0047] Preferably, the data receiving submodule includes an integrated voice recognition API to support users to ask questions via voice.

[0048] Preferably, the automated response module further comprises: a model optimization submodule for optimizing and fine-tuning the large model based on the user's query pattern and feedback.

[0049] Preferably, the markdown processing unit will answer the generated security report and stipulate that the network security model is output in markdown format.

[0050] Preferably, the system also includes a preprocessing module to process user questions in the data receiving submodule, including text cleaning, word segmentation, and keyword recognition, and finally output a formatted language that can be recognized by the large model.

[0051] Preferably, the network security big model includes: a classification and analysis unit that analyzes the intent and entity of the problem, classifies the problem, and determines whether it is a problem consultation type or a disposal suggestion type; a context management unit that performs context management on the current user's question; and a question and answer generation unit that calls the knowledge base, deep learning, and natural language technology to generate answers for different types of questions.

[0052] Preferably, the question and answer generating unit,

[0053] For question-asking questions, call the knowledge base, retrieve relevant information, and generate answers;

[0054] For problems involving disposal suggestions, deep learning and natural language processing technologies are used to analyze and identify the problems, and output situation analysis, disposal suggestions, and effect evaluation in markdown format.

[0055] like Figure 2 ,A method for constructing network security auxiliary system based on large model,

[0056] S1: Build a fine-tuned network security big model: Deploy the big model locally and fine-tune the big model based on the network security corpus to build a network security big model;

[0057] S2: Build an updateable knowledge base based on the RAG framework: The RAG framework allows the cybersecurity big model to retrieve relevant information from the knowledge base when generating answers or suggestions; the knowledge base can supplement and update cybersecurity related content;

[0058] S3: Building an updateable network security big model: When answering user questions, the network security big model calls the knowledge base through the RAG framework to retrieve relevant information;

[0059] S4: Optimizing the network security big model based on user feedback: Users provide feedback based on the answers of the network security big model, and the network security big model is dynamically adjusted according to the feedback.

[0060] Preferably, the knowledge base includes: network security related documents from the Internet, news, industry regulations, and internal company documents.

[0061] Preferably, the network security big model generates a security report based on the markdown processing unit using the answers to the disposal suggestion type, and the output format is markdown. The security report includes: situation analysis, disposal suggestions and effect evaluation, and the user can choose to save the security report.

[0062] In another embodiment, a detailed construction and operation process of a network security assistant based on the system is described:

[0063] Figure 3 The overall operation process of the network security assistant is shown. Users ask questions to the network security assistant, and the questions are pre-processed and converted into language that can be recognized by the large model. The questions are classified. For questions related to problem consultation, the AI ​​assistant answers the questions and the users provide feedback. For questions related to treatment suggestions, the answers are given in markdown format, including: scenario analysis, treatment suggestions and effect evaluation. The report generation module generates a report in a specified format and gives it to the user for feedback.

[0064] Figure 4 The construction process of the network security assistant is demonstrated. The network security big model is obtained by fine-tuning the deepseek big model, and the RAG framework is built in combination with the network security knowledge base. The network security big model with updateable knowledge base is built, and the report generation module and user feedback module are combined to build a complete network security AI assistant.

[0065] 1. User Interface Design:

[0066] Develop a user-friendly interface robot (UI) as the front end for users to interact with the AI ​​system.

[0067] The interface should support text input, or integrate voice recognition technology to allow voice questions.

[0068] 1-1. Identify target users: Understand the target user groups, their needs and preferences.

[0069] Functional definition: Determine the functions that the interface robot needs to support, such as text input, voice recognition, multi-language support, etc.

[0070] Technology assessment: Evaluate the required technology stack, including front-end frameworks, back-end services, speech recognition APIs, etc.

[0071] 1-2. Interface design: Design an intuitive and easy-to-use interface layout, including text input boxes, voice input buttons, feedback areas, etc.

[0072] User Experience (UX) Design: Ensure that the interaction process of the interface is simple and clear, and users can operate it easily.

[0073] Prototyping: Creating a prototype of the interface, either low-fidelity or high-fidelity, for testing and gathering feedback.

[0074] 1-3. Choose the technology stack: Choose the appropriate front-end framework (Vue.js, etc.) and back-end technology (Java, etc.) according to your needs.

[0075] Front-end development: Text input: Implement a text input box to allow users to enter questions.

[0076] Speech recognition integration: Integrate speech recognition API (Google Speech-to-Text) to allow users to ask questions via voice.

[0077] Response Display: Design and implement areas that display the responses of AI systems.

[0078] Back-end development: API development: Develop back-end APIs to process requests sent by the front-end, interact with the AI ​​system, and return responses.

[0079] Database: Set up a database to store user data, session history, etc.

[0080] 2. Build a big network security model:

[0081] 2-1. Fine-tuning the basic large model: Use the network security corpus to fine-tune the deepseek large model to obtain a network security large model suitable for the network security field.

[0082] Cybersecurity corpus: Construct cybersecurity corpus suitable for fine-tuning format, whose data sources are network and company knowledge base.

[0083] Fine-tuning: Fine-tune the DeepSeek large model to make it familiar with basic concepts of network security.

[0084] 2-2. Build the RAG framework to provide a real-time updated cybersecurity knowledge base for the cybersecurity big model. The RAG framework allows the cybersecurity big model to retrieve relevant information from the knowledge base when generating answers or suggestions, thereby improving the accuracy and completeness of the answers.

[0085] Knowledge base construction: Extensively collect network security related documents from the Internet, news, industry regulations, and internal company information to construct the RAG knowledge base.

[0086] Framework building: Build the RAG framework so that the large model can use the information in the knowledge base to answer questions.

[0087] 3. Add user feedback module and report generation module:

[0088] 3-1. Based on the network security big model, add a user feedback module to receive user feedback on answers or suggestions, and provide data support for the continuous optimization of the system.

[0089] 3-2. Add a report generation module to convert the situation analysis, disposal suggestions and effect evaluation in markdown format output by the network security assistant into a report in a specified format for users to view and save.

[0090] 4. Implement workflow:

[0091] After the user asks a question through the user interface, the system first performs preprocessing, including text cleaning (removing irrelevant characters), word segmentation, keyword recognition, etc.

[0092] 4-1-1. Input acquisition: The system receives the question input by the user, which can be text input or the result of speech-to-text conversion.

[0093] 4-1-2. Removing irrelevant characters: Delete irrelevant characters in the text, such as special symbols, numbers, HTML tags, etc., which may interfere with subsequent text processing.

[0094] Removing stop words: Remove common but insignificant words in the text, such as "de", "shi", "zai", etc., which usually contribute little to understanding the meaning of the question.

[0095] Text normalization: Convert all characters to a unified case (usually lowercase) to reduce lexical variations.

[0096] 4-1-3. Chinese word segmentation: For Chinese text, it is necessary to split continuous character sequences into meaningful words or phrases.

[0097] English word segmentation: For English text, it is usually segmented according to spaces and punctuation marks, but some special cases, such as hyphens, abbreviations, etc., also need to be processed.

[0098] 4-1-4. Part-of-speech tagging: Tag the part of speech of each word after word segmentation, such as noun, verb, adjective, etc.

[0099] 4-1-5. Keyword extraction: Use an algorithm (TextRank) to extract keywords or phrases from the text, which can represent the main content of the text.

[0100] 4-1-6. Entity recognition: Identify specific entities in the text, such as person names, locations, organizations, times, etc., and classify them.

[0101] 4-1-7. Intent determination: Analyze the user's question to determine its intent, such as asking for information, seeking help, or making suggestions.

[0102] 4-1-8. Solving the problem of polysemy: For words with multiple meanings, determine their exact meaning in the current text according to the context.

[0103] 4-1-9. Semantic role annotation: Identify the predicate in the sentence and its corresponding arguments (such as the agent, the patient, etc.), and annotate their semantic roles.

[0104] 4-1-10. Formatted output: Format the results of the preprocessing so that subsequent NLP tasks or AI systems can use this data for further processing.

[0105] 4-1-11.API development: Develop API interface so that the preprocessing module can be called by other systems or modules.

[0106] Modularity: Ensure that the preprocessing module has good encapsulation for easy maintenance and upgrading.

[0107] 4-2. Question understanding and classification. Use natural language understanding (NLU) technology to parse the intent and entity of the question to ensure that the user's question is correctly understood. Classify the question to determine whether it is a question consultation (knowledge question and answer) or a solution suggestion (a practical problem that needs to be solved).

[0108] 4-3. Context management: If the question is related to a previous interaction, the system will use context management capabilities to maintain the continuity of the conversation.

[0109] 4-4. Question answering: The network security assistant will answer different types of questions.

[0110] For problem consultation questions, the Cyber ​​Security Assistant directly retrieves relevant information from the knowledge base and generates answers.

[0111] For issues related to handling suggestions, the Cybersecurity Assistant uses deep learning and natural language processing technologies to analyze and identify the issues, and outputs situation analysis, handling suggestions, and effect evaluation in markdown format.

[0112] 4-5. Report generation. For the disposal suggestion module, the answers of the security assistant are converted into reports in a specified format. When generating reports, the answers generated by the large model need to be post-processed, including language polishing, format adjustment, key information extraction, etc., to improve the readability and accuracy of the answers.

[0113] 4-5-1. Language polishing

[0114] Grammar check: Check the answers for grammatical errors such as subject-verb agreement, wrong tense, etc.

[0115] Spelling Correction: Fix spelling mistakes and make sure all words are correct.

[0116] Stylistic consistency: Adjust the language style to match the expectations of the target audience, such as formality or informality.

[0117] Remove redundancy: Remove unnecessary repeated information to make the answer more concise.

[0118] Improve coherence: Make sure the sentences in your answer flow smoothly and are logically coherent.

[0119] 4-5-2. Extraction of key information

[0120] Keyword Identification: Identify and highlight key words or phrases in answers.

[0121] Summary generation: Generate a short summary that outlines the main content of the answer.

[0122] Information point extraction: Extract key information points from the answers for quick browsing.

[0123] 4-5-3. Fact-checking

[0124] Data Verification: Verify the data and facts in the answer to ensure their accuracy.

[0125] Source checking: Checking the source of cited information to ensure its reliability.

[0126] 4-5-4. Sensitive content review

[0127] Sensitive word filtering: Check and remove potentially sensitive or inappropriate content.

[0128] Cultural adaptability: Ensure that the answer content is appropriate for users from different cultural backgrounds.

[0129] 4-6. User rating: Users can rate the content generated this time and provide feedback for the continuous optimization of the system.

[0130] 5.Continuous optimization and update:

[0131] Based on user feedback and the latest threat intelligence, the cybersecurity model is regularly fine-tuned and the knowledge base is updated to improve the accuracy and adaptability of the system.

[0132] Continuously optimize the user interface and workflow to improve user experience and satisfaction.

[0133] 5-1. Data analysis

[0134] Feedback analysis: Regularly analyze user feedback to identify the strengths and weaknesses of the model in a specific task or domain.

[0135] Trend identification: By analyzing feedback data, we can discover the changing trends and potential problems of model performance.

[0136] 5-2. Model retraining

[0137] Data screening: Filter out valuable data from feedback for model retraining.

[0138] Model fine-tuning: Based on user feedback, the model is fine-tuned to improve its performance on specific tasks.

[0139] 5-3. Model Optimization

[0140] Parameter adjustment: Adjust model parameters based on feedback results to optimize model performance.

[0141] Algorithm improvement: It may be necessary to improve the model algorithm or introduce a new algorithm to solve a specific problem.

[0142] 5-4. Monitoring and evaluation

[0143] Performance monitoring: Continuously monitor the performance of the model to ensure the effectiveness of optimization measures.

[0144] Effect evaluation: Regularly evaluate the effects of model optimization, including internal evaluation and user satisfaction surveys.

[0145] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present invention.

Claims

1. A network security assistance system based on a large model, characterized in that: include: Knowledge base construction module, used to build an updateable knowledge base based on the RAG framework; The automated response module includes: a data receiving submodule for receiving user questions, a network security big model constructed by fine-tuning using network security data, a data output submodule for outputting the network security big model's answers and disposal suggestions, and a user feedback submodule for users to give feedback on the big model's answers and disposal suggestions; when generating answers or suggestions, the network security big model retrieves relevant information from a knowledge base; the data output submodule also includes a markdown processing unit for generating a security report based on the answers.

2. A network security assistance system based on a large model according to claim 1, characterized in that: The contents of the security report include: threat analysis, response measures and effect evaluation.

3. A network security assistance system based on a large model according to claim 1, characterized in that: The data receiving submodule includes an integrated voice recognition API to support users to ask questions via voice.

4. A network security assistance system based on a large model according to claim 1, characterized in that: The automated response module also includes a model optimization submodule, which optimizes and fine-tunes the large model based on the user's query pattern and feedback.

5. A network security assistance system based on a large model according to claim 1, characterized in that: The markdown processing unit will answer the generated security report and stipulate that the network security model is output in markdown format.

6. A network security assistance system based on a large model according to claim 1, characterized in that: The system also includes a preprocessing module, which processes user questions in the data receiving submodule, including text cleaning, word segmentation, and keyword recognition, and finally outputs a formatted language that can be recognized by the network security model.

7. A network security assistance system based on a large model according to claim 1, characterized in that: The network security big model includes: a classification and parsing unit that analyzes the intent and entity of the question, classifies the question, and determines whether it is a question consultation type or a disposal suggestion type; a context management unit that performs context management on the current user's question; and a question and answer generation unit that calls the knowledge base, deep learning, and natural language technology to generate answers for different types of questions.

8. A network security assistance system based on a large model according to claim 1, characterized in that: The question-answer generating unit, For question-asking questions, call the knowledge base, retrieve relevant information, and generate answers; For problems involving disposal suggestions, deep learning and natural language processing technologies are used to analyze and identify the problems, and output situation analysis, disposal suggestions, and effect evaluation in markdown format.

9. A method for constructing a network security auxiliary system based on a large model according to any one of claims 1 to 8, characterized in that: S1: Build a fine-tuned network security big model: Deploy the big model locally and fine-tune the big model based on the network security corpus to build a network security big model; S2: Build an updateable knowledge base based on the RAG framework: The RAG framework allows the cybersecurity big model to retrieve relevant information from the knowledge base when generating answers or suggestions; the knowledge base can supplement and update cybersecurity related content; S3: Building an updateable network security big model: When answering user questions, the network security big model calls the knowledge base through the RAG framework to retrieve relevant information; S4: Optimizing the network security big model based on user feedback: Users provide feedback based on the answers of the network security big model, and the network security big model is dynamically adjusted according to the feedback.

10. A method for constructing a network security auxiliary system based on a large model according to claim 9, characterized in that: The knowledge base includes: network security related documents from the Internet, news, industry regulations, and internal company information.

11. A method for constructing a network security auxiliary system based on a large model according to claim 9, characterized in that: The network security big model generates a security report based on the markdown processing unit using the answers to the disposal suggestion type. The output format is markdown. The security report includes: situation analysis, disposal suggestions and effect evaluation. The user can choose to save the security report.

Images