A method for constructing an intelligent question-answering system in the security field and the intelligent question-answering system itself.

Abstract

This application provides a method for constructing an intelligent question-answering system in the security field and an intelligent question-answering system. The construction method includes: acquiring training data, which includes at least historical business data in the security field provided by the user of the intelligent question-answering system, and historical business data obtained from a third-party security platform through a first interface API; constructing a large question-answering model based on the training data, and constructing at least one lightweight model; wherein, the large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, the target tasks being published by the user; and constructing an intelligent question-answering system based on the large question-answering model and the lightweight model. In this way, intelligent question-answering technology can be closely integrated with actual security needs, realizing question-answering interaction and task execution in security scenarios.

Description

Technical Field

[0001] This application relates to the field of intelligent question answering technology, and in particular to a method for constructing an intelligent question answering system in the field of security and an intelligent question answering system. Background Technology

[0002] Large models generally refer to deep learning models with a large number of parameters, neurons, and hierarchical structures, such as large language models (LLMs) trained on text data (also known as corpora). Large language models, trained on large datasets, are capable of performing various natural language processing (NLP) tasks.

[0003] When implementing a question-answering system based on a large language model, it is usually necessary to train the system with massive amounts of data so that the system acquires extensive knowledge. In other words, the knowledge base used by the question-answering system when answering questions mainly comes from the training data.

[0004] However, if the training dataset lacks sufficient information in a particular domain, the question-answering system may provide inaccurate answers or even fail to answer specific questions within that domain. For example, in highly specialized fields such as security, the system may lack sufficient in-depth expertise to provide accurate solutions. Therefore, question-answering systems built using existing methods often lack a strong connection to real-world application scenarios and are therefore less practical. Summary of the Invention

[0005] This application provides a method for constructing an intelligent question-answering system in the security field and an intelligent question-answering system, in order to solve the problem that the question-answering system constructed by the existing question-answering system construction method lacks connection with real application scenarios.

[0006] In a first aspect, embodiments of this application provide a method for constructing a security domain intelligent question-answering system based on a large model, comprising: acquiring training data, the training data including at least historical business data in the security domain provided by the user of the intelligent question-answering system, and historical business data acquired from a third-party security platform through a first interface API; constructing a large question-answering model based on the training data, and constructing at least one lightweight model; wherein the large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, the target tasks being published by the user; and constructing an intelligent question-answering system based on the large question-answering model and the lightweight model.

[0007] In some feasible approaches, the steps of constructing at least one lightweight model include: determining the target running nodes included in the lightweight model and the arrangement relationship between each target running node based on the user's task requirements for the target task; the task requirements include at least the requirements for detection and analysis of business data, the requirements for prediction of numerical values, and the requirements for classification of business data; in the modeling interface, arranging each target running node based on the arrangement relationship and setting the model parameters of each target running node to model the lightweight model; wherein, when the target task is a periodic task, the target running nodes include at least periodic nodes.

[0008] In some feasible implementations, the lightweight model is specifically used to label real-time business data to obtain labeled result data. Before the step of building an intelligent question-answering system based on the large question-answering model and the lightweight model, the system further includes: building a knowledge base for the large question-answering model so that the large question-answering model can form answers based on the knowledge base; the knowledge base includes at least training data and reference documents provided by the user stored in vector format. After the step of building an intelligent question-answering system based on the large question-answering model and the lightweight model, the system further includes: after the large question-answering model forms answers based on the user's questions, merging the questions and answers into question-answer pairs and saving the question-answer pairs to the knowledge base so that the large question-answering model can form answers based on the user's questions. The knowledge base is optimized; the question-answer pair includes questions and answers from one or more question-answering rounds; and / or, after the lightweight model outputs the label result data, the label result data is saved to the knowledge base so that the large question-answering model can be optimized based on the knowledge base; and, for each lightweight model, its corresponding execution status is collected according to a first preset period to form a task list, and a first visualization interface is formed based on the task list to manage the lightweight model; the execution status includes at least the model status, the number of target tasks, the accuracy of the lightweight model's labeling, the time spent executing the target tasks, the cumulative number of target tasks executed, and the time of the last execution of the target tasks.

[0009] In some feasible approaches, after the steps of building an intelligent question-answering system based on a large question-answering model and a lightweight model, the following steps are also included: modifying the model parameters of the corresponding running nodes of the lightweight model based on the tag result data.

[0010] In some feasible approaches, after the steps of building an intelligent question-answering system based on a large question-answering model and a lightweight model, the following steps are also included: after deploying the intelligent question-answering system on the user's host, establishing a communication channel between the intelligent question-answering system and the host; acquiring real-time business data based on the communication channel, and preprocessing the real-time business data, wherein the preprocessing steps include at least storing the real-time business data in a target format.

[0011] Secondly, embodiments of this application also provide a security domain intelligent question-answering system based on a large model. The intelligent question-answering system is constructed based on the security domain intelligent question-answering system construction method and its various implementations provided in the first aspect. The intelligent question-answering system includes a question-answering large model and at least one lightweight model. The question-answering large model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data. The target tasks are published by the user.

[0012] In some feasible implementations, the intelligent question-answering system also includes a knowledge base, which is built based on historical business data and reference documents provided by the user. The question-answering model is specifically used for: retrieving the knowledge base based on the user's question, and / or using a search engine to retrieve multiple candidate documents; the steps of retrieving the knowledge base based on the user's question at least include vectorizing the question and performing vector matching on the reference documents based on the question; calculating a first similarity for each candidate document; selecting n candidate documents corresponding to the first similarity scores in descending order as relevant documents; forming an answer based on the relevant documents; and feeding the answer back to the user.

[0013] The first similarity score is calculated based on the following formula:

[0014] score = 0.2 * score fuzzy +0.1*score cosine +0.4*score BM25 +0.3*score Levenshtein ;

[0015] Where score represents the first similarity; score fuzzy The score represents the score of the fuzzy similarity algorithm between the question and the candidate documents. cosine The value in 'score' represents the cosine similarity score between the question and candidate documents after vectorization. BM25 The score represents the BM25 similarity score between the question and the candidate documents. Levenshtein The values ​​in denoted by represent the Levenstein distance similarity scores between the question and the candidate documents, with 0.2, 0.1, 0.4, and 0.3 representing weighting coefficients.

[0016] In some feasible methods, after selecting n candidate documents corresponding to the first similarity in descending order to form the answer and feeding the answer back to the user, the question-answering model is also used to: calculate accuracy, recall, and F1 score; wherein, the formula for calculating accuracy is: accuracy P = total number of relevant documents / total number of candidate documents * 100%; the formula for calculating recall is: recall R = total number of relevant documents / total number of reference documents * 100%; the formula for calculating F1 score is: P * R * 2 / (P + R); and the accuracy, recall, and F1 score are saved to the database according to the second preset period.

[0017] In some feasible implementations, a model invocation module is also included, specifically used for: responding to a target task published by the user of the intelligent question-answering system, determining the target business data and the target lightweight model used to execute the target task based on the type of the target task; invoking the target lightweight model to tag the real-time business data and obtain the tag result data; forming a second visualization interface based on the tag result data and displaying the second visualization interface to the user; and saving the tag result data to the database.

[0018] Thirdly, embodiments of this application also provide a device for constructing a security domain intelligent question-answering system based on a large model, comprising: an acquisition module for acquiring training data, the training data including at least historical business data in the security domain provided by the user of the intelligent question-answering system, and historical business data acquired from a third-party security platform through a first interface API; a first construction module for constructing a large question-answering model based on the training data, and constructing at least one lightweight model; wherein the large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, the target tasks being published by the user; and a second construction module for constructing an intelligent question-answering system based on the large question-answering model and the lightweight model.

[0019] As described above, this application provides a method for constructing an intelligent question-answering system in the security field and an intelligent question-answering system. The construction method includes: acquiring training data, which includes at least historical business data in the security field provided by the user of the intelligent question-answering system, and historical business data obtained from a third-party security platform through a first interface API; constructing a large question-answering model based on the training data, and constructing at least one lightweight model; wherein the large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, the target tasks being published by the user; and constructing an intelligent question-answering system based on the large question-answering model and the lightweight model. In this way, intelligent question-answering technology can be closely integrated with actual security needs, realizing question-answering interaction and task execution in security scenarios. Attached Figure Description

[0020] Figure 1 A flowchart illustrating the method for constructing a security domain intelligent question-answering system based on a large model, as provided in this application embodiment;

[0021] Figure 2 A schematic diagram of the lightweight modeling interface provided in the embodiments of this application;

[0022] Figure 3 A schematic diagram illustrating the process of constructing a lightweight model as provided in an embodiment of this application;

[0023] Figure 4 A schematic diagram of the first visual interface provided in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the structure of the intelligent question-answering system provided in the embodiments of this application;

[0025] Figure 6 A schematic diagram of the structure of the security domain intelligent question-answering system construction device based on a large model provided in this application embodiment. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of this application.

[0027] Before introducing the technical solutions of the embodiments of this application, the terminology involved in the embodiments of this application will be introduced by way of example.

[0028] Security field: Generally refers to the field of information security, which encompasses a range of measures, technologies, and practices in computer and network technology specifically designed to protect systems, networks, data, and information from unauthorized access, damage, tampering, or disclosure. It mainly includes the following key aspects:

[0029] (1) Network security: Ensure the security of network infrastructure, communications and data transmission, and prevent hacking, malware and network attacks.

[0030] (2) Information security: Protecting the integrity, availability and confidentiality of information, including measures such as data encryption, access control, data backup and recovery.

[0031] (3) Application security: Ensure the security of software applications during the design, development, deployment and maintenance process, and prevent the exploitation of vulnerabilities and security weaknesses.

[0032] (4) Physical security: Protect the physical environment such as computing devices, servers and data centers from physical damage, theft and destruction.

[0033] (5) Identity authentication and access management: Ensure that only authorized users and devices can access the system and data, including multi-factor authentication and access control policies.

[0034] (6) Risk management and compliance: Assess security risks and take appropriate measures to comply with regulations, standards and industry best practices to ensure the organization’s security compliance.

[0035] (7) Safety awareness and training: Enhance employees’ awareness and understanding of safety issues and strengthen their safety awareness and behavior in daily work.

[0036] In summary, the security field encompasses a wide range of technologies, strategies, and practices aimed at protecting information and technology resources from various threats and attacks, ensuring the continued secure operation and business continuity of an organization.

[0037] Enterprise informatization: Enterprise informatization refers to the process by which enterprises introduce and utilize information technology to improve business management efficiency, optimize resource allocation, enhance service quality, and strengthen competitiveness. The construction of enterprise informatization systems typically includes the development of application systems, such as Enterprise Management Information Systems (ERP), Customer Relationship Management Systems (CRM), or Supply Chain Management Systems (SCM).

[0038] Therefore, the operation and maintenance management of information systems is a crucial step in ensuring their normal operation. In actual operation and maintenance management, intelligent question-and-answer systems can be used to help improve the stability and efficiency of information systems. When facing problems in the use or configuration of application systems, operation and maintenance personnel can obtain real-time answers and guidance through intelligent question-and-answer systems. By analyzing problem descriptions or keywords, intelligent question-and-answer systems automatically match relevant knowledge bases or expert experience to provide effective solutions or suggestions.

[0039] However, when implementing a question-answering system based on a large language model, it is usually necessary to train the system with massive amounts of data so that the system can acquire extensive knowledge. In other words, the knowledge base used by the question-answering system when answering questions mainly comes from the training data.

[0040] However, if the training dataset lacks sufficient information in a particular domain, the question-answering system may provide inaccurate answers or even be unable to answer specific questions within that domain. For example, in highly specialized fields such as security, the system may lack sufficient in-depth expertise to provide accurate solutions. Therefore, question-answering systems built using existing methods often lack a strong connection to real-world application scenarios and are therefore less practical.

[0041] This application provides a method for constructing a security domain intelligent question-answering system based on a large model, which can build a vertical domain intelligent question-answering system and realize intelligent, humanized, secure, professional and localized intelligent question-answering capabilities.

[0042] Figure 1 This is a flowchart illustrating the method for constructing a security domain intelligent question-answering system based on a large model, as provided in an embodiment of this application.

[0043] like Figure 1 As shown, the method for constructing a security domain intelligent question-answering system based on a large model provided in this application includes the following steps S100-S300:

[0044] S100: Obtain training data, which includes at least historical business data in the security field provided by the user of the intelligent question answering system, as well as historical business data obtained from a third-party security platform through the first interface API.

[0045] Users of the intelligent question-answering system, such as financial institutions and cybersecurity companies, can obtain historical business data in the security field provided by the users in this application embodiment. Historical business data refers to information generated by users during security-related operations, or process data generated during the operation of third-party systems (such as the user's management system). It can cover problems and solutions in various scenarios, such as security incident handling, vulnerability management, and security policy enforcement. This application embodiment can also obtain historical business data from third-party security platforms through the first interface API. Third-party security platforms are external security platforms, which generally include network security monitoring services, threat intelligence centers, and security vulnerability databases. Integrating this data can help the intelligent question-answering system update and provide the latest security solutions in a timely manner, thereby enhancing the practicality and responsiveness of the intelligent question-answering system. More importantly, using historical business data as training data to build the intelligent question-answering system allows it to integrate with the security field, making it more suitable for professional security-related questions and answers.

[0046] S200: Construct a large question-answering model based on training data, and construct at least one lightweight model; wherein, the large question-answering model is used to generate answers to questions from users, and the lightweight model is used to execute target tasks based on real-time business data from users, the target tasks being published by users.

[0047] In this embodiment, the large question-answering model can be based on an open-source model, such as BERT (Bidirectional Encoder Representations from Transformers) or GPT (Generative Pre-trained Transformer) series models. Furthermore, this embodiment can utilize training data to perform customized training and fine-tuning of the open-source model, enabling the large question-answering model to better adapt to specific security domain problems and contexts. Correspondingly, a lightweight model is a small model with a lightweight architecture, such as a Convolutional Neural Network (CNN) or a Recurrent Neural Network (RNN). Deploying a trained lightweight model to a production environment ensures that it can monitor and process the target tasks issued by the user in real time, improving the efficiency of the user's operation and maintenance management.

[0048] In this embodiment, the target tasks may include credential stuffing detection, malicious domain name analysis, password strength identification, crawler IP identification, and machine behavior detection. In this embodiment, a corresponding lightweight model can be built for each type of target task.

[0049] Among them, the credential stuffing detection model can be used to detect and prevent hackers from attempting to log in using leaked username and password combinations (obtained from data breaches). Specifically, the credential stuffing detection model can analyze the patterns, frequency, and sources of login attempts, and identify abnormal login patterns through machine learning or rule engines, thereby effectively preventing credential stuffing attacks.

[0050] Malicious domain analysis models can be used to identify and block malicious domains, preventing users from accessing phishing websites, malware download links, command and control (C&C) attacks, and other malicious activity disguises. These models combine domain feature analysis, historical behavior data, and blacklists, using machine learning algorithms or rule-based detection techniques to classify and evaluate domains, providing accurate identification of malicious domains.

[0051] Password strength assessment models can be used to evaluate the strength of user-set passwords, reducing the risk of passwords being cracked or guessed. Specifically, they can analyze password complexity, length, character combinations, and compare them with commonly used password databases, providing feedback to users and recommending stronger password choices.

[0052] Web crawler IP identification models can be used to identify and defend against malicious web crawler programs accessing websites or APIs. Specifically, they can analyze characteristics such as access patterns, request frequency, and source IP addresses, and use machine learning or rule-based methods to distinguish between normal users and web crawlers, thereby implementing appropriate anti-crawler measures.

[0053] Some accounts can log into the management system via scripts, allowing them to execute scripts and interact with information. Unlike manually operated accounts, these accounts exhibit machine-like characteristics, enabling frequent, short-term operations; they are generally referred to as bot accounts. If unauthorized users create and use bot accounts without permission, these accounts can threaten the management system, potentially leading to communication disruptions or data leaks. Machine behavior detection models can be used to detect bot accounts.

[0054] It is understood that the foregoing content is merely an illustrative introduction to the target task. In practical applications, the target task and lightweight model can be determined based on the actual situation.

[0055] It is worth noting that the large question-answering model and lightweight design can work together to achieve a more efficient response mechanism and task execution process.

[0056] In this embodiment, the large question-answering model can be used to understand and generate natural language answers, processing questions raised by users. The lightweight model can be used to process real-time business data and execute specific tasks. When the answer generated by the large question-answering model to a user's question involves a target task (such as performing an operation or querying real-time data), the lightweight model can be invoked to complete the corresponding target task. For example, if a user asks, "Please help me check the current security status," the large question-answering model can recognize that this is a request to execute a task and can invoke the lightweight model to obtain real-time business data and return the result. In specific implementations, a standardized interface protocol can be defined, allowing the large question-answering model to initiate the lightweight model to execute specific tasks when needed via API calls.

[0057] Furthermore, the lightweight model can feed the processed results back to the large question-answering model, which can then integrate the feedback into the answer and send it back to the user. Based on this, the large question-answering model's responses can provide real-time updates on the latest operational status.

[0058] In some implementations, embodiments of this application may also establish a shared database or caching mechanism, enabling the large question-answering model and the lightweight model to share data and state information for better collaboration.

[0059] S300: An intelligent question-answering system built on a large question-answering model and a lightweight model.

[0060] In practical applications, intelligent question-answering systems can have at least two types of user interfaces. The first type is used for question-answering interaction, allowing users to ask questions, while the second type is used for user task interaction, allowing users to post tasks. By working collaboratively with a large-scale question-answering model and a lightweight model, the functionality and user experience of the intelligent question-answering system can be improved.

[0061] As described above, this application provides a method for constructing a security domain intelligent question-answering system based on a large model. The method includes: acquiring training data, which includes at least historical business data in the security domain provided by the user of the intelligent question-answering system, and historical business data obtained from a third-party security platform via a first interface API; constructing a large question-answering model based on the training data; and constructing at least one lightweight model. The large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, where the target tasks are published by the user. The intelligent question-answering system is then constructed based on the large question-answering model and the lightweight model. This allows for a close integration of intelligent question-answering technology with actual security needs, enabling question-answering interaction and task execution in security scenarios.

[0062] Figure 2 This is a schematic diagram of the lightweight modeling interface provided in an embodiment of this application.

[0063] Figure 3 This is a schematic diagram illustrating the process of constructing a lightweight model as provided in an embodiment of this application.

[0064] like Figure 2 and Figure 3 As shown, step S200 may include the following steps S201-S202.

[0065] S201: Based on the user's task requirements for the target task, determine the target running nodes included in the lightweight model and the arrangement relationship between each target running node; the task requirements should at least include the requirements for detection and analysis of business data, the requirements for prediction of numerical values, and the requirements for classification of business data.

[0066] In this embodiment, the running nodes are determined based on the actual execution process of the target task. Running nodes may include, for example, data source nodes, data preprocessing nodes, feature engineering nodes, statistical analysis nodes, machine learning nodes, deep learning nodes, natural language processing nodes, and custom script nodes. The arrangement of these target running nodes can be determined based on the actual situation. Figure 2 Only one arrangement relationship is shown as an example.

[0067] For example, a data source node refers to a node that provides raw data, such as data tables or other files in a database. A data preprocessing node can specifically perform operations such as cleaning and transformation on the raw data to make it meet the requirements of subsequent data analysis and modeling. A feature engineering node can be used to extract useful information (features) from the raw data to improve model performance. A statistical analysis node is used to perform statistical analysis on the data using statistical methods and tools. Deep learning nodes and machine learning nodes are built based on data modeling and prediction techniques and can be used to process and learn data to make predictions or decisions. Custom script nodes can be user-defined nodes or nodes defined based on user needs during the development of the intelligent question-answering system; specifically, they can include code or programs written according to specific needs. Specific needs may include, for example, database creation steps, which are not specifically limited in this embodiment.

[0068] S202: In the modeling interface, each target running node is arranged based on the arrangement relationship, and the model parameters of each target running node are set to model the lightweight model; wherein, when the target task is a periodic task, the target running nodes shall include at least periodic nodes.

[0069] The model parameters, such as network structure parameters and loss functions, can be determined based on actual conditions, and this application does not impose specific limitations on them. Periodic tasks support data mining and detection within a set specific time period until the results are returned.

[0070] Furthermore, in this embodiment, the intelligent question-answering system can be directly deployed in the user's production environment, specifically on various hosts of the user. Then, it can access the user's private data resources for intelligent question-answering. This reduces factual errors in the large question-answering model, enabling private and accurate question-answering and facilitating private domain interaction. Simultaneously, deploying the intelligent question-answering system in the user's local production environment significantly reduces the risk of data privacy leaks.

[0071] Furthermore, in this embodiment, the lightweight model is specifically used to label real-time business data to obtain labeled result data. This step can occur either during the model training phase or during the model application phase.

[0072] Specifically, after step S300, the following steps S401-S402 may also be included.

[0073] S401: After deploying the intelligent question-answering system on the user's host, establish a communication channel between the intelligent question-answering system and the host.

[0074] The specific deployment steps may refer to installing the intelligent question-answering system program on the host machine. The purpose of establishing a communication channel is to connect the user's private data to the intelligent question-answering system.

[0075] S402: Acquire real-time service data based on the communication channel and preprocess the real-time service data. The preprocessing steps include at least storing the real-time service data in the target format.

[0076] Real-time business data refers to the real-time data generated during the actual operation of the user's management system or other third-party systems. Business data can include enterprise user information, raw logs, alarm data, asset information, and permission information. A preprocessing step, for example, is to convert the real-time business data into an Excel spreadsheet. For instance, the header of the Excel spreadsheet can include IP address, operation time, and user account. Each column of the Excel spreadsheet can be considered a label. In other words, the preprocessing step organizes the user-provided data into tagged data. Furthermore, when the lightweight model executes the target task, the tagged data can be further tagged to obtain tagged result data. For example, a malicious domain name analysis model can tag the IP address column to obtain tagged result data. Thus, when the model outputs the task results, the user can determine the domains exhibiting malicious behavior based on the column names and specific IP addresses of the tagged columns.

[0077] In this embodiment of the application, the following step S500 may be included before step S300.

[0078] S500: Build a knowledge base for the question-answering model so that the question-answering model can generate answers based on the knowledge base; the knowledge base includes at least training data and reference documents provided by the user, stored in vector format.

[0079] Reference documents include, for example, standardized security standards or compliance documents used to ensure information security, network security, and physical security. These reference documents can be obtained via FTP or SFTP protocols and can also be stored in an Oracle database. It's understood that the knowledge base is the information source for the question-answering model to generate answers.

[0080] Storing reference documents as vectors involves dividing the document into data blocks. For example, preprocessing the document according to a preset character count can include removing irrelevant characters and splitting the document into multiple data blocks. Then, NLP techniques can be used to extract features from the preprocessed document. The purpose of feature extraction is to convert textual information into a numerical form that computers can understand for further processing and analysis. The extracted features are then converted into vector representations. Furthermore, the resulting vectors are stored in a database or a dedicated vector search engine. The database can be a vector database (such as Faiss, Annoy, etc.), and the stored vectors can be indexed. Vector indexes can be built based on tree structures (such as KD-trees, ball trees, etc.) or hash techniques (such as LSH).

[0081] It should be noted that the knowledge base can include not only training data and reference documents, but also knowledge graphs provided by users.

[0082] Step S300 may be followed by the following steps S601-S602.

[0083] S601: After the large question-answering model generates an answer based on the user's question, the question and answer are merged into a question-answer pair, and the question-answer pair is saved to the knowledge base so that the large question-answering model can be optimized based on the knowledge base; the question-answer pair includes questions and answers from one or more question-answering rounds; and / or, after the lightweight model outputs the label result data, the label result data is saved to the knowledge base so that the large question-answering model can be optimized based on the knowledge base.

[0084] In this way, based on the self-learning capability of the large question-answering model, it can continuously learn and optimize according to the new data fed back from the large question-answering model and / or the lightweight model, thereby further improving the quality of question-answering feedback and the accuracy of data analysis.

[0085] Furthermore, after step S601, the following step S602 may also be included: for each lightweight model, its corresponding execution status is collected according to a first preset period to form a task list, and a first visualization interface is formed based on the task list to manage the lightweight model.

[0086] The first preset period can be hourly, daily, or weekly, etc. The task list, also known as the task management list, is the first visual interface for the intelligent question-answering system's management side. Based on this first visual interface, the management side can quickly identify the status of the model executing the target task. Figure 4 The diagram above illustrates a first visual interface.

[0087] The execution status includes at least the model status, the number of target tasks, the labeling accuracy of the lightweight model, the time taken to execute target tasks, the cumulative number of target tasks executed, and the time of the last target task execution. Furthermore, the model status can include normal and abnormal. The number of target tasks can refer to the total number of target tasks assigned to or received by the lightweight model within a period. The labeling accuracy of the lightweight model can refer to the accuracy of the model in classifying, identifying, or labeling data when executing target tasks. The time taken to execute target tasks can refer to the time required for the lightweight model to complete a single or average target task. The cumulative number of target tasks executed refers to the total number of target tasks completed since the lightweight model was deployed. The time of the last target task execution can refer to the specific time when the lightweight model most recently executed a target task.

[0088] Understandably, the first visualization interface can be used to represent the usage of the lightweight model by multiple users. The intelligent question answering system management terminal can interact with the first visualization interface to perform batch operations on the task status in the task list, such as deleting data.

[0089] Furthermore, after step S300, the following step S603 may also be included: modifying the model parameters of the corresponding running node of the lightweight model based on the label result data.

[0090] In practical applications, users can determine the accuracy of the lightweight model based on the label results data. After receiving the accuracy feedback, they can adjust the model parameters and related indicators of the lightweight model based on the accuracy, which can help optimize the lightweight model.

[0091] This application also provides a security domain intelligent question answering system based on a large model, which can be constructed based on the construction method provided in the foregoing embodiments.

[0092] Figure 5 This is a schematic diagram of the structure of the intelligent question-answering system provided in an embodiment of this application.

[0093] like Figure 5 As shown, the intelligent question-answering system may include a large question-answering model 1001 and at least one lightweight model 1002. The large question-answering model 1001 is used to generate answers to questions from users, and the lightweight model 1002 is used to execute target tasks based on real-time business data from users, where the target tasks are published by the users. The intelligent question-answering system also includes a knowledge base, which is built based on historical business data and reference documents provided by users.

[0094] Furthermore, the question-and-answer model 1001 is specifically used to perform the following steps S701-S703.

[0095] S701; Based on the questions raised by the user, the knowledge base is searched, and / or a search engine is used to retrieve multiple candidate documents.

[0096] In this embodiment, the question-answering model 1001 can recall multiple candidate documents from various channels. The specific number of candidate documents recalled can be, for example, 30, but this embodiment does not impose a specific limitation. Different channels can refer to knowledge base channels and external network data (search engine channels).

[0097] Furthermore, the steps for retrieving knowledge from the knowledge base based on the questions raised by the user include at least vectorizing the questions and performing vector matching on the reference documents based on the questions. Specifically, the intelligent question answering system can first convert the question into a high-dimensional vector, which can be achieved through word embedding or semantic embedding techniques (such as BERT, GPT, etc.) in natural language processing. In this way, the semantic information of the question can be represented in vector form.

[0098] The intelligent question-answering system provided in this application embodiment can also utilize search engines or external network data sources for retrieval. Specifically, this can be accomplished by directly querying a search engine or calling a specific network data API. In this way, the intelligent question-answering system can obtain the latest information that may not be in the knowledge base, thereby increasing the diversity and breadth of candidate documents.

[0099] S702: For each candidate document, calculate the first similarity.

[0100] The first similarity score is calculated based on the following formula:

[0101] score = 0.2 * score fuzzy +0.1*score cosine +0.4*score BM25 +0.3*score Levenshtein ;

[0102] Where score represents the first similarity; score fuzzy The score represents the score of the fuzzy similarity algorithm between the question and the candidate documents. cosine The value in 'score' represents the cosine similarity score between the question and candidate documents after vectorization. BN25 The score represents the BM25 similarity score between the question and the candidate documents. Levenshtein The values ​​in denoted by represent the Levenstein distance similarity scores between the question and the candidate documents, with 0.2, 0.1, 0.4, and 0.3 representing weighting coefficients.

[0103] Specifically, the Fuzzy similarity algorithm can calculate the degree of fuzzy matching between two strings based on characters or tokens. In this embodiment, a fuzzy matching algorithm can be used to calculate the similarity. Cosine similarity is an indicator that measures how similar the directions of two vectors are. In intelligent question-answering systems, the cosine value between the two vectors corresponding to the question and the candidate document can be calculated. The closer the cosine value is to 1, the closer the directions of the two vectors are, i.e., the higher the semantic similarity. BM25 is an information retrieval algorithm based on a probabilistic retrieval model. It considers multiple factors such as term frequency (TF), inverse document frequency (IDF), and the position of words in the document to evaluate the relevance between the query and the document. The BM25 algorithm is widely used in search engines because it can effectively handle large-scale text data. Levenstein distance, also known as edit distance, refers to the minimum number of single-character edits required to transform one string into another. Editing refers to insertion, deletion, or replacement.

[0104] The weighting coefficients can be adjusted based on actual circumstances to enable the intelligent question-answering system to more comprehensively and accurately evaluate the matching degree between candidate documents and questions, thereby providing more accurate and useful answers. This application does not impose specific limitations on the weighting coefficients.

[0105] S703: Select n candidate documents corresponding to the first similarity scores in descending order as relevant documents, form an answer based on the relevant documents, and feed the answer back to the user.

[0106] In this embodiment, n can specifically be equal to 3, 4, or 5; this embodiment does not impose a specific limitation. This ensures that the documents retrieved from the knowledge base are highly consistent with the semantic content of the question.

[0107] The step of generating an answer based on relevant documentation can also be called prompt templating. Prompt templating specifically involves creating a set of fixed-format prompts that cover all the key information needed in a specific context. For example, when outputting the system's operational status, the template could include the system name, current status, key metrics (such as CPU usage, memory usage, disk space, etc.), alarm / error messages, and suggested actions. This is merely an example.

[0108] After step S703, the question-and-answer big model 1001 is also used to perform the following steps S704-S705.

[0109] S704: Calculate precision, recall, and F1 score; wherein, the formula for calculating precision is: Precision P = Total number of relevant documents / Total number of candidate documents * 100%; the formula for calculating recall is: Recall R = Total number of relevant documents / Total number of reference documents * 100%; the formula for calculating F1 score is: P * R * 2 / (P + R).

[0110] Understandably, accuracy, recall, and F1 score are important metrics for evaluating the quality of large question-answering models.

[0111] S705: Save the accuracy, recall, and F1 score to the database according to the second preset period.

[0112] The second preset period can be, for example, daily, weekly, or monthly. During this second preset period, the accuracy, recall, and F1 score can be considered self-monitoring results, which can be saved to a Syslog database. Syslog is a standard protocol used for transmitting log information. Monitoring systems or specialized tools (such as rsyslog or syslog-ng) can be configured to send important monitoring events (accuracy, recall, and F1 score) to a log server for centralized management.

[0113] In this embodiment, to evaluate the performance of the lightweight model, related metrics such as latency, throughput, resource utilization, and stability metrics can be calculated. Latency, also known as delay, specifically refers to the time required for the lightweight model to process a single request; each running node corresponds to one request. Throughput refers to the number of requests the lightweight model can process per unit of time. Resource utilization can include the utilization of resources such as CPU, memory, and GPU, used to evaluate the resource consumption of the lightweight model. Stability metrics include, for example, error rate and anomaly detection count. Furthermore, this embodiment can also save the values ​​of various metrics to a database using the Syslog protocol for long-term storage and subsequent query and analysis.

[0114] The intelligent question-answering system provided in this application embodiment may further include a calling module 1003, which is specifically used to execute the following steps S801-S803.

[0115] S801: In response to the target task issued by the user of the intelligent question answering system, determine the target business data and the target lightweight model used to execute the target task based on the type of the target task.

[0116] In some implementations, the target lightweight model can also be specified directly when the user publishes the target task.

[0117] S802: Call the target lightweight model to label real-time business data and obtain the label result data.

[0118] The intelligent question-answering system provided in this application can not only quickly identify and determine the target business data required to execute the target task based on the type of the target task, but also match the target lightweight model for executing the task. Specifically, it can parse the task description based on natural language processing (NLP) technology to determine the target lightweight model. For example, it can match the name of lightweight model 1002 with the task description to determine the target lightweight model 1.

[0119] S803: Create a second visualization interface based on the label result data and display the second visualization interface to the user; and save the label result data to the database.

[0120] In this embodiment of the application, the label result data can be saved to the database using the Syslog protocol.

[0121] As can be seen, this embodiment of the application can generate a second visual interface to present complex labeling results data to users in an intuitive and easy-to-understand form, enabling them to quickly understand the data status and business trends. Simultaneously, this embodiment of the application can also automatically save the labeling results data to a database, ensuring data traceability and security.

[0122] It is worth noting that the embodiments of this application can also save the tag result data to the knowledge base so that the question-answering big model 1001 can complete self-optimization.

[0123] In some implementations, the intelligent question-answering system provided in this application embodiment can also generate alarms for abnormal data. For example, when the abnormal value in the tag result data exceeds a preset threshold, alarm information can be generated and an alarm process can be triggered.

[0124] Figure 6 A schematic diagram of the structure of the security domain intelligent question-answering system construction device based on a large model provided in this application embodiment.

[0125] like Figure 6 As shown, this application provides an apparatus for constructing a security domain intelligent question-answering system based on a large model, comprising:

[0126] The acquisition module 2001 is used to acquire training data. The training data includes at least historical business data in the security field provided by the user of the intelligent question answering system, as well as historical business data acquired from a third-party security platform through the first interface API.

[0127] The first building module 2002 is used to build a large question-answering model based on training data, and to build at least one lightweight model; wherein, the large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, the target tasks being published by the user.

[0128] The second building module 2003 is used to build an intelligent question answering system based on the large question answering model and the lightweight question answering model.

[0129] In some implementations, the first construction module 2002 is specifically used to determine the target running nodes included in the lightweight model and the arrangement relationship between each target running node based on the user's task requirements for the target task; the task requirements include at least the requirements for detection and analysis of business data, the requirements for prediction of numerical values, and the requirements for classification of business data; in the modeling interface, each target running node is arranged based on the arrangement relationship, and the model parameters of each target running node are set to model the lightweight model; wherein, when the target task is a periodic task, the target running nodes include at least periodic nodes.

[0130] In some implementations, the lightweight model is specifically used to label real-time business data to obtain labeled result data; the first construction module 2002 is also used to: build a knowledge base for the question-answering model so that the question-answering model can form answers based on the knowledge base; the knowledge base includes at least training data and reference documents provided by the user stored in vector form.

[0131] The first construction module 2002 is further configured to: after the large question-answering model forms an answer based on the user's question, merge the question and answer into a question-answer pair and save the question-answer pair to the knowledge base so that the large question-answering model can be optimized based on the knowledge base; the question-answer pair includes questions and answers from one or more question-answering rounds; and / or, after the lightweight model outputs label result data, save the label result data to the knowledge base so that the large question-answering model can be optimized based on the knowledge base; and the second construction module 2003 is further configured to: for each lightweight model, collect its corresponding execution status according to a first preset period to form a task list, and form a first visualization interface based on the task list to manage the lightweight model; the execution status includes at least the model status, the number of target tasks, the accuracy of the lightweight model's labeling, the time spent executing the target tasks, the cumulative number of target tasks executed, and the time of the last execution of the target tasks.

[0132] In some implementations, the first building module 2002 is also used to modify the model parameters of the corresponding running node of the lightweight model based on the label result data.

[0133] In some implementations, the second building module 2003 is also used to establish a communication channel between the intelligent question-answering system and the host after the intelligent question-answering system is deployed on the user's host; to obtain real-time business data based on the communication channel, and to preprocess the real-time business data, wherein the preprocessing steps include at least storing the real-time business data in a target format.

[0134] In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, it may include some or all of the steps of the various embodiments of the method for constructing a security domain intelligent question-answering system based on a large model provided by the present invention. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0135] It is readily understood that, based on the several embodiments provided in this application, those skilled in the art can combine, split, or reorganize the embodiments of this application to obtain other embodiments, none of which exceed the protection scope of this application.

[0136] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for constructing a security domain intelligent question-answering system based on a large model, characterized in that, include: Acquire training data, which includes at least historical business data in the security field provided by the user of the intelligent question answering system, and historical business data obtained from a third-party security platform through the first interface API; Based on the training data, a large question-answering model is constructed, and at least one lightweight model is constructed; wherein, the large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, the target tasks being published by the user, and the target tasks including at least one of credential stuffing detection, malicious domain name analysis, password strength identification, crawler IP identification, and machine behavior detection; the real-time business data is generated during the operation of the user's third-party security platform, and the real-time business data includes at least one of enterprise user information, raw logs, alarm data, asset information, and permission information; A smart question-answering system is constructed based on the aforementioned large-scale question-answering model and the aforementioned lightweight model. After deploying the intelligent question-answering system on the user's host, a communication channel is established between the intelligent question-answering system and the host. The real-time service data is acquired based on the communication channel, and the real-time service data is preprocessed to obtain tag data. The preprocessing step includes at least storing the real-time service data in a target format. When the lightweight model performs the target task, the lightweight model is used to label the label data to obtain label result data; the intelligent question answering system also includes a second visualization interface, which is formed based on the label result data and is used to display the data to the user. The method further includes: after the lightweight model outputs the label result data, saving the label result data to a knowledge base so that the question-answering model can be optimized based on the knowledge base; and, for each lightweight model, a task list is formed by aggregating its corresponding execution status according to a first preset period, and a first visualization interface is formed based on the task list to manage the lightweight model; the execution status includes at least the model status, the number of target tasks, the labeling accuracy of the lightweight model, the time spent executing the target tasks, the cumulative number of target tasks executed, and the time of the last execution of the target tasks; Furthermore, the method further includes: modifying the model parameters of the corresponding running node of the lightweight model based on the label result data.

2. The method for constructing a security domain intelligent question-answering system based on a large model according to claim 1, characterized in that, The steps to build at least one lightweight model include: Based on the user's task requirements for the target task, the target running nodes included in the lightweight model and the arrangement relationship between each target running node are determined; the task requirements include at least the requirements for detection and analysis of business data, the requirements for prediction of numerical values, and the requirements for classification of business data. In the modeling interface, the target running nodes are arranged according to the arrangement relationship, and the model parameters of each target running node are set to model the lightweight model; wherein, when the target task is a periodic task, the target running nodes include at least periodic nodes.

3. The method for constructing a security domain intelligent question-answering system based on a large model according to claim 1, characterized in that, Before the step of constructing an intelligent question-answering system based on the large question-answering model and the lightweight model, the following steps are also included: A knowledge base is constructed for the question-answering model so that the question-answering model can generate answers based on the knowledge base; the knowledge base includes at least the training data and the reference documents provided by the user stored in vector format; After the step of constructing an intelligent question-answering system based on the large question-answering model and the lightweight model, the system further includes: After the question-and-answer model generates an answer based on the user's question, the question and answer are merged into a question-and-answer pair, and the question-and-answer pair is saved to the knowledge base so that the question-and-answer model can be optimized based on the knowledge base; the question-and-answer pair includes questions and answers from one or more question-and-answer rounds.

4. A security domain intelligent question-answering system based on a large model, characterized in that, The intelligent question-answering system is constructed based on the construction method described in any one of claims 1-3; The intelligent question-answering system includes a large question-answering model and at least one lightweight model. The large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data. The target tasks are published by the user. The real-time business data is generated during the operation of the user's third-party security platform. The intelligent question-answering system is also used to: establish a communication channel between the intelligent question-answering system and the host after the intelligent question-answering system is deployed on the user's host; The real-time service data is acquired based on the communication channel, and the real-time service data is preprocessed to obtain tag data. The preprocessing step includes at least storing the real-time service data in a target format. When the lightweight model performs the target task, the lightweight model is used to label the label data to obtain label result data; the intelligent question answering system also includes a second visualization interface, which is formed based on the label result data and is used to display the data to the user. The intelligent question-answering system is also used to: after the lightweight model outputs the tag result data, save the tag result data to the knowledge base, so that the large question-answering model can be optimized based on the knowledge base; Furthermore, for each of the lightweight models, its corresponding execution status is aggregated according to a first preset period to form a task list, and a first visualization interface is formed based on the task list to manage the lightweight models; the execution status includes at least the model status, the number of target tasks, the accuracy of the labeling of the lightweight model, the time spent executing the target tasks, the cumulative number of target tasks executed, and the time of the last execution of the target tasks; Furthermore, the intelligent question-answering system is also used to modify the model parameters of the corresponding running node of the lightweight model based on the tag result data.

5. The security domain intelligent question-answering system based on a large model according to claim 4, characterized in that, The intelligent question-answering system also includes a knowledge base, which is built based on historical business data and reference documents provided by users; The aforementioned question-answering model is specifically used for: The knowledge base is searched based on the question raised by the user, and / or a search engine is used to retrieve multiple candidate documents; the step of searching the knowledge base based on the question raised by the user includes at least vectorizing the question and performing vector matching on the reference documents based on the question. For each of the candidate documents, calculate the first similarity. Based on the first similarity scores in descending order, n candidate documents corresponding to the first similarity scores are selected as relevant documents. An answer is formed based on the relevant documents and then fed back to the user. The first similarity is calculated based on the following formula: ； in, This indicates the first similarity; The score represents the fuzzy similarity between the question and the candidate documents. The value in the vectorized form represents the cosine similarity score between the question and the candidate document. The value in 'm' represents the BM25 similarity score between the question and the candidate document. The values ​​in denoted by represent the Levenstein distance similarity scores between the question and the candidate document, with 0.2, 0.1, 0.4, and 0.3 representing weighting coefficients.

6. The security domain intelligent question-answering system based on a large model according to claim 5, characterized in that, After selecting n candidate documents corresponding to the first similarity scores in descending order to form an answer, and feeding the answer back to the user, the question-answering model is further used for: Calculate precision, recall, and F1 score; wherein, the formula for calculating precision is: Precision P = Total number of relevant documents / Total number of candidate documents. 100%; The formula for calculating the recall rate is: Recall Rate R = Total number of relevant documents / Total number of reference documents 100%; the formula for calculating the F1 value is: P R 2 / (P+R); According to the second preset period, the accuracy, the recall, and the F1 score are saved to the database.

7. The security domain intelligent question-answering system based on a large model according to claim 4, characterized in that, It also includes a model invocation module, specifically used for: In response to the target task issued by the user of the intelligent question answering system, target business data and a target lightweight model for executing the target task are determined based on the type of the target task. The target lightweight model is invoked to label the real-time business data, thereby obtaining the label result data; A second visualization interface is generated based on the label result data, and the second visualization interface is displayed to the user. In addition, the label result data is saved to the database.

8. A device for constructing a security domain intelligent question-answering system based on a large model, characterized in that, include: The acquisition module is used to acquire training data, which includes at least historical business data in the security field provided by the user of the intelligent question answering system, as well as historical business data acquired from a third-party security platform through the first interface API. The first construction module is used to construct a large question-answering model based on the training data, and to construct at least one lightweight model; wherein, the large question-answering model is used to generate answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, the target tasks being published by the user, and the target tasks including at least one of credential stuffing detection, malicious domain name analysis, password strength identification, crawler IP identification, and machine behavior detection; the real-time business data is generated during the operation of the user's third-party security platform, and the real-time business data includes at least one of enterprise user information, raw logs, alarm data, asset information, and permission information; The second construction module is used to build an intelligent question-answering system based on the large question-answering model and the lightweight model. The second building module is also used to: establish a communication channel between the intelligent question-answering system and the host after the intelligent question-answering system is deployed on the user's host; The real-time service data is acquired based on the communication channel, and the real-time service data is preprocessed to obtain tag data. The preprocessing step includes at least storing the real-time service data in a target format. When the lightweight model performs the target task, the lightweight model is used to label the label data to obtain label result data; the intelligent question answering system also includes a second visualization interface, which is formed based on the label result data and is used to display the data to the user. Furthermore, the second construction module is also configured to: after the lightweight model outputs the label result data, save the label result data to a knowledge base so that the question-answering model can be optimized based on the knowledge base; and, for each lightweight model, aggregate its corresponding execution status according to a first preset period to form a task list, and form a first visualization interface based on the task list to manage the lightweight model; the execution status includes at least the model status, the number of target tasks, the labeling accuracy of the lightweight model, the execution time of the target tasks, the cumulative number of target tasks executed, and the time of the last execution of the target tasks; Furthermore, the first construction module is also used to: modify the model parameters of the corresponding running node of the lightweight model based on the label result data.

Images