Construction method of intelligent question-answering system in security domain and intelligent question-answering system

Abstract

The embodiment of the invention provides a security domain intelligent question-answering system construction method and an intelligent question-answering system.The construction method comprises the steps that training data is obtained, and the training data at least comprises historical business data, provided by an intelligent question-answering system user, of a security domain and historical business data obtained from a third-party security platform through a first interface API; constructing a question and answer large model based on the training data, and constructing at least one lightweight model; wherein the large question and answer model is used for forming answers for questions of a user, the lightweight model is used for executing a target task based on real-time service data of the user, and the target task is issued by the user; and constructing the intelligent question-answering system based on the large question-answering model and the lightweight model. In this way, the intelligent question-answering technology can be closely combined with the actual safety requirement, and question-answering interaction and task execution in the safety scene are achieved.

Description

Technical Field

[0001] The present application relates to the field of intelligent question-answering technology, and in particular to a method for constructing an intelligent question-answering system in the security field and an intelligent question-answering system. Background Art

[0002] A large model generally refers to a deep learning model with a large number of parameters, a large number of neurons and a hierarchical structure, such as a large language model (LLM) for text data (also known as corpus). Large language models are trained on large data sets and can perform various natural language processing (NLP) tasks.

[0003] When implementing a question-answering system based on a large language model, it is usually necessary to train the question-answering system with massive data so that the system can acquire extensive knowledge. In other words, the knowledge base used by the question-answering system when answering mainly comes from the training data.

[0004] However, if the training data set does not contain enough information in a certain field, the question-answering system may give inaccurate answers or even fail to answer specific questions in this field. For example, in highly specialized fields such as security, the system may not be able to provide accurate answers due to a lack of in-depth expertise. It can be seen that the question-answering system built based on the existing question-answering system construction method lacks connection with real application scenarios and has poor practicality. Summary of the invention

[0005] The embodiments of the present application provide a method for constructing an intelligent question-and-answer system in the security field and an intelligent question-and-answer system to solve the problem that the question-and-answer system constructed by the existing question-and-answer system construction method lacks connection with the actual application scenario.

[0006] In a first aspect, an embodiment of the present application provides a method for constructing an intelligent question and answer system in the security field based on a big model, comprising: obtaining training data, the training data at least including historical business data in the security field provided by a user of the intelligent question and answer system, and historical business data obtained from a third-party security platform through a first interface API; constructing a question and answer big model based on the training data, and constructing at least one lightweight model; wherein the question and answer big model is used to form answers to the user's questions, and the lightweight model is used to perform target tasks based on the user's real-time business data, and the target tasks are issued by the user; constructing an intelligent question and answer system based on the question and answer big model and the lightweight model.

[0007] In some achievable methods, the step of constructing at least one lightweight model includes: based on the user's task requirements for the target task, determining the target operating nodes included in the lightweight model and the arrangement relationship between the target operating nodes; the task requirements include at least the detection and analysis requirements for business data, the prediction requirements for numerical values, and the classification requirements for business data; in the modeling interface, arranging the target operating nodes based on the arrangement relationship, and setting the model parameters of each target operating node to model the lightweight model; wherein, when the target task is a periodic task, the target operating nodes include at least periodic nodes.

[0008] In some achievable methods, the lightweight model is specifically used to label real-time business data to obtain label result data; before the step of building an intelligent question-answering system based on the question-answering large model and the lightweight model, it also includes: building a knowledge base for the question-answering large model so that the question-answering large model can form answers based on the knowledge base; the knowledge base at least includes training data and reference documents provided by the user stored in vector form; after the step of building an intelligent question-answering system based on the question-answering large model and the lightweight model, it also includes: after the question-answering large model forms an answer based on the user's question, the question and the answer are merged into a question-answer pair, and the question-answer pair is saved in the knowledge base, so that the question-answering large model can form an answer based on the user's question. The knowledge base is optimized; the question-answer pair includes questions and answers of one or more question-answer rounds; and / or, after the lightweight model outputs the label result data, the label result data is saved in the knowledge base, so that the question-answering large model is optimized based on the knowledge base; and, for each lightweight model, its corresponding execution status is collected according to the first preset period to form a task list, and a first visual interface is formed based on the task list to manage the lightweight model; the execution status includes at least the model status, the number of target tasks, the accuracy of lightweight model labeling, the time taken to execute the target task, the cumulative number of target tasks executed, and the time of the last execution of the target task.

[0009] In some achievable methods, after the step of constructing an intelligent question-answering system based on the large question-answering model and the lightweight model, it also includes: modifying the model parameters of the corresponding running nodes of the lightweight model based on the label result data.

[0010] In some feasible methods, after the step of building an intelligent question-and-answer system based on the large question-and-answer model and the lightweight model, it also includes: after the intelligent question-and-answer system is deployed on the user's host, a communication channel is established between the intelligent question-and-answer system and the host; real-time business data is obtained based on the communication channel, and the real-time business data is pre-processed, and the pre-processing step at least includes storing the real-time business data in a target format.

[0011] In the second aspect, an embodiment of the present application also provides an intelligent question and answer system in the security field based on a big model. The intelligent question and answer system is constructed based on the method for constructing an intelligent question and answer system in the security field based on a big model provided in the first aspect and its various implementation methods; the intelligent question and answer system includes a big question and answer model and at least one lightweight model. The big question and answer model is used to form answers to the user's questions, and the lightweight model is used to execute target tasks based on the user's real-time business data, and the target tasks are issued by the user.

[0012] In some achievable methods, the intelligent question-answering system further includes a knowledge base, which is constructed based on historical business data and reference documents provided by the user; the question-answering big model is specifically used for: searching the knowledge base based on questions raised by the user, and / or searching using a search engine to obtain multiple candidate documents; the step of searching the knowledge base based on the questions raised by the user at least includes vectorizing the questions and performing vector matching on the reference documents based on the questions; calculating a first similarity for each candidate document; selecting n candidate documents corresponding to the first similarity as relevant documents in descending order of the first similarity, forming an answer based on the relevant documents, and feeding back the answer to the user;

[0013] The first similarity is calculated based on the following formula:

[0014] score=0.2*score fuzzy +0.1*score cosine +0.4*score BM25 +0.3*score Levenshtein ;

[0015] Among them, score represents the first similarity; score fuzzy Indicates the score of the fuzzy similarity algorithm between the question and the candidate document, score cosine The score represents the cosine similarity calculation score of the question and candidate document after vectorization. BM25 The score represents the BM25 similarity algorithm score of the question and the candidate document. Levenshtein represents the score of the Levenshtein distance similarity algorithm between the question and the candidate document, and 0.2, 0.1, 0.4 and 0.3 represent the weighting coefficients.

[0016] In some feasible methods, after selecting n candidate documents corresponding to the first similarity in order from large to small to form an answer and feeding the answer back to the user, the question-answering model is also used to: calculate the accuracy, recall and F1 value; wherein, the formula for calculating the accuracy is: accuracy P = total number of relevant documents / total number of candidate documents*100%; the formula for calculating the recall is: recall R = total number of relevant documents / total number of reference documents*100%; the formula for calculating the F1 value is: P*R*2 / (P+R); according to the second preset period, the accuracy, recall and F1 value are saved to the database.

[0017] In some achievable methods, a model calling module is also included, which is specifically used to: respond to a target task issued by a user of the intelligent question and answer system, determine the target business data and a target lightweight model for executing the target task based on the type of the target task; call the target lightweight model so that the target lightweight model labels the real-time business data to obtain label result data; form a second visualization interface based on the label result data, and display the second visualization interface to the user; and save the label result data to a database.

[0018] On the third aspect, an embodiment of the present application also provides a device for constructing an intelligent question and answer system in the security field based on a big model, including: an acquisition module for acquiring training data, the training data at least includes historical business data in the security field provided by the user of the intelligent question and answer system, and historical business data obtained from a third-party security platform through a first interface API; a first construction module for constructing a question and answer big model based on the training data, and constructing at least one lightweight model; wherein the question and answer big model is used to form answers to the user's questions, and the lightweight model is used to perform target tasks based on the user's real-time business data, and the target tasks are issued by the user; a second construction module is used to construct an intelligent question and answer system based on the question and answer big model and the lightweight model.

[0019] From the above content, it can be seen that the embodiment of the present application provides a method for constructing an intelligent question-and-answer system in the security field and an intelligent question-and-answer system, and the construction method includes: obtaining training data, the training data at least includes historical business data in the security field provided by the user of the intelligent question-and-answer system, and historical business data obtained from a third-party security platform through a first interface API; constructing a large question-and-answer model based on the training data, and constructing at least one lightweight model; wherein the large question-and-answer model is used to form answers to the user's questions, and the lightweight model is used to perform target tasks based on the user's real-time business data, and the target tasks are issued by the user; constructing an intelligent question-and-answer system based on the large question-and-answer model and the lightweight model. In this way, the intelligent question-and-answer technology can be closely integrated with actual security needs to realize question-and-answer interaction and task execution in security scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for constructing a security field intelligent question-answering system based on a large model provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a lightweight model building interface provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a process for constructing a lightweight model provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of a first visual interface provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of the intelligent question-answering system provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the structure of a device for building an intelligent question-and-answer system in the security field based on a large model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0027] Before introducing the technical solutions of the embodiments of the present application, the terms involved in the embodiments of the present application are first introduced by way of example.

[0028] Security field: usually refers to the field of information security, which refers to a series of measures, technologies and practices in computer technology and network technology that specifically deal with protecting systems, networks, data and information from unauthorized access, destruction, tampering or leakage. It mainly includes the following important aspects:

[0029] (1) Cybersecurity: Ensure the security of network infrastructure, communications, and data transmission to prevent hacker intrusion, malware, and cyber attacks.

[0030] (2) Information security: protecting the integrity, availability and confidentiality of information, including measures such as data encryption, access control, data backup and recovery.

[0031] (3) Application security: Ensure the security of software applications during design, development, deployment, and maintenance to prevent the exploitation of vulnerabilities and security weaknesses.

[0032] (4) Physical security: Protecting the physical environment, such as computing equipment, servers, and data centers, from physical damage, theft, and destruction.

[0033] (5) Identity authentication and access management: Ensure that only authorized users and devices can access systems and data, including multi-factor authentication and access control policies.

[0034] (6) Risk Management and Compliance: Assess security risks and take appropriate measures to comply with regulations, standards, and industry best practices to ensure the organization's security compliance.

[0035] (7) Safety awareness and training: Improve employees’ awareness and understanding of safety issues and enhance their safety awareness and behavior in daily work.

[0036] In summary, the security field covers a wide range of technologies, strategies and practices aimed at protecting information and technology resources from various threats and attacks, ensuring the safe operation and continuous business of the organization.

[0037] Enterprise informatization: Enterprise informatization refers to the process of enterprises introducing and using information technology to improve business management efficiency, optimize resource allocation, improve service quality and enhance competitiveness. The process of enterprise informatization usually includes the construction of application systems, such as enterprise management information system (ERP), customer relationship management system (CRM) or supply chain management system (SCM).

[0038] Therefore, the operation and maintenance management of information systems is a key step to ensure their normal operation. In the actual operation and maintenance management process, the intelligent question-answering system can be used to help improve the stability and efficiency of the information system. When operation and maintenance personnel face problems in the use or configuration of application systems, they can use the intelligent question-answering system for real-time question answers and guidance. The intelligent question-answering system analyzes the problem description or keywords, automatically matches the relevant knowledge base or expert experience, and provides effective solutions or suggestions.

[0039] However, when implementing a question-answering system based on a large language model, it is usually necessary to train the question-answering system with massive data so that the system can acquire extensive knowledge. In other words, the knowledge base used by the question-answering system when answering mainly comes from the training data.

[0040] However, if the training data set does not contain enough information in a certain field, the question-answering system may give inaccurate answers or even fail to answer specific questions in this field. For example, in highly specialized fields such as security, the system may not be able to provide accurate answers due to a lack of in-depth expertise. It can be seen that the question-answering system built based on the existing question-answering system construction method lacks connection with real application scenarios and has poor practicality.

[0041] The embodiment of the present application provides a method for constructing an intelligent question-and-answer system in the security field based on a large model, which can construct an intelligent question-and-answer system in a vertical field and realize intelligent, humanized, security-specialized and localized intelligent question-and-answer capabilities.

[0042] Figure 1 A flowchart of a method for constructing a security field intelligent question-answering system based on a large model provided in an embodiment of the present application.

[0043] like Figure 1 As shown, the method for constructing a security field intelligent question-answering system based on a large model provided in an embodiment of the present application includes the following steps S100-S300:

[0044] S100: Obtain training data, where the training data at least includes historical business data in the security field provided by the user of the intelligent question-and-answer system, and historical business data obtained from a third-party security platform through a first interface API.

[0045] The user of the intelligent question-and-answer system, such as a financial institution, a network security company, etc., can obtain historical business data in the security field provided by the user of the intelligent question-and-answer system in the embodiment of the present application. The historical business data is the information generated by the user in security-related operations, and can also refer to the process data generated during the operation of a third-party system (such as the user's management system), which can cover problems and solutions in various scenarios, such as security incident handling, vulnerability management, and security policy enforcement. The embodiment of the present application can also obtain historical business data from a third-party security platform through the first interface API. The third-party security platform is an external security platform, which generally includes network security monitoring services, threat intelligence centers, and security vulnerability databases. Integrating these data can help the intelligent question-and-answer system to update in a timely manner and provide the latest security solutions, thereby enhancing the practicality and responsiveness of the intelligent question-and-answer system. More importantly, using historical business data as training data to build an intelligent question-and-answer system can integrate the intelligent question-and-answer system with the security field, making it more suitable for professional questions and answers in the security field.

[0046] S200: Build a large question-answering model based on the training data, and build at least one lightweight model; wherein the large question-answering model is used to form answers to the user's questions, and the lightweight model is used to perform target tasks based on the user's real-time business data, and the target tasks are issued by the user.

[0047] In an embodiment of the present application, the large question-answering model can be based on an open source model, such as BERT (Bidirectional Encoder Representations from Transformers) or GPT (Generative Pre-trained Transformer) series models. Furthermore, the embodiment of the present application can use training data to perform customized training and fine-tuning on the open source model, so that the large question-answering model can better adapt to specific security issues and contexts. Correspondingly, a lightweight model is a small model with a lightweight architecture, such as a convolutional neural network (CNN) or a recursive neural network (RNN). Deploying the trained lightweight model into a production environment can ensure that it can monitor and process the target tasks issued by the user in real time, thereby improving the efficiency of the user's operation and maintenance management.

[0048] In the embodiment of the present application, the target tasks may include database collision detection, malicious domain name analysis, password strength identification, crawler IP identification and machine behavior detection. In the embodiment of the present application, a corresponding lightweight model can be constructed for each type of target task.

[0049] Among them, the credential stuffing behavior detection model can be used to detect and prevent hackers from using leaked username and password combinations (obtained from data leaks) to attempt to log in. Specifically, the credential stuffing behavior detection model can analyze the pattern, frequency, and source of login attempts, and identify abnormal login patterns through machine learning or rule engines, thereby effectively preventing credential stuffing attacks.

[0050] Malicious domain name analysis models can be used to identify and block malicious domain names, prevent users from accessing phishing websites, malware download links, command and control (C&C), and malicious activity disguises, etc. Malicious domain name analysis models can combine domain name feature analysis, historical behavior data, and blacklists, and use machine learning algorithms or rule detection technology to classify and evaluate domain names to provide accurate malicious domain name identification.

[0051] The password strength recognition model can be used to evaluate the strength of passwords set by users to reduce the risk of passwords being cracked or guessed. Specifically, it can analyze the complexity, length, character combination of passwords and their comparison with common password databases, provide feedback to users and recommend stronger password options.

[0052] The crawler IP identification model can be used to identify and defend against malicious crawler programs accessing websites or APIs. Specifically, it can analyze features such as access patterns, request frequencies, and source IP addresses, and use machine learning or rule-based methods to distinguish between normal users and crawlers, thereby implementing appropriate anti-crawler measures.

[0053] Some accounts can log in to the management system through scripts, and then such accounts can execute scripts and realize information interaction. Different from manually operated accounts, such accounts have machine operation characteristics that can be operated frequently in a short period of time, and are generally called machine accounts. If some illegal users create and use machine accounts without permission, then the machine accounts will pose a threat to the management system and even cause communication obstruction or data leakage. Machine behavior detection models can be used to detect machine accounts.

[0054] It can be understood that the above content is only an exemplary introduction to the target task. In actual application, the target task and lightweight model can be determined based on actual conditions.

[0055] It is worth noting that the large question-answering model and the lightweight one can work together to achieve a more efficient response mechanism and task execution process.

[0056] In an embodiment of the present application, the question-and-answer big model can be used to understand and generate natural language answers and process questions raised by the user. The lightweight model can be used to process real-time business data and perform specific tasks. When the answer formed by the question-and-answer big model to the user's question involves a target task (such as performing an operation or querying real-time data), the lightweight model can be called to complete the corresponding target task. For example, the user asks "Please help me check the current security status". At this time, the question-and-answer big model can recognize that this is a request to perform a task, and can call the lightweight model to obtain real-time business data and return the result. In the specific implementation, a standardized interface protocol can be defined so that the question-and-answer big model can start the lightweight model to perform specific tasks when needed through API calls.

[0057] In addition, the lightweight model can feed back the processed results to the large question-answering model, and the large question-answering model can integrate the feedback results into the answer and feed it back to the user. Based on this, the answer of the large question-answering model can feedback the latest operating status in real time.

[0058] In some implementations, the embodiments of the present application may also establish a shared database or cache mechanism so that the large question-and-answer model and the lightweight model can share data and status information for better collaboration.

[0059] S300: Build an intelligent question-answering system based on a large question-answering model and a lightweight model.

[0060] In practical applications, the intelligent question-answering system can have at least two types of user interfaces. The first type of user interface is used for question-answering interaction so that users can ask questions, and the other type of user interface is used for user-task interaction so that users can publish tasks. The collaborative work of the large question-answering model and the lightweight model can improve the functionality and user experience of the intelligent question-answering system.

[0061] From the above content, it can be seen that the embodiment of the present application provides a method for constructing an intelligent question-and-answer system in the security field based on a large model, including: obtaining training data, the training data at least includes historical business data in the security field provided by the user of the intelligent question-and-answer system, and historical business data obtained from a third-party security platform through a first interface API; constructing a large question-and-answer model based on the training data, and constructing at least one lightweight model; wherein the large question-and-answer model is used to form answers to the user's questions, and the lightweight model is used to perform target tasks based on the user's real-time business data, and the target tasks are issued by the user; constructing an intelligent question-and-answer system based on the large question-and-answer model and the lightweight model. In this way, the intelligent question-and-answer technology can be closely integrated with actual security needs to achieve question-and-answer interaction and task execution in security scenarios.

[0062] Figure 2 A schematic diagram of a lightweight model building interface provided in an embodiment of the present application.

[0063] Figure 3 A schematic diagram of the process of constructing a lightweight model provided in an embodiment of the present application.

[0064] like Figure 2 and Figure 3 As shown, step S200 may include the following steps S201-S202.

[0065] S201: Based on the user's task requirements for the target task, determine the target operating nodes included in the lightweight model and the arrangement relationship between each target operating node; the task requirements at least include the detection and analysis requirements for business data, the prediction requirements for numerical values, and the classification requirements for business data.

[0066] In the embodiment of the present application, the running nodes are determined based on the actual execution process of the target task. The running nodes are, for example, data source nodes, data preprocessing nodes, feature engineering nodes, statistical analysis nodes, machine learning nodes, deep learning nodes, natural language processing nodes, and custom script nodes. How to arrange each target running node can be determined based on the actual situation. Figure 2 Only one arrangement relationship is shown as an example.

[0067] Exemplarily, a data source node refers to a node that provides raw data, such as files such as data tables in a database. A data preprocessing node may specifically be a node that performs operations such as cleaning and conversion on raw data so that the data meets the requirements of subsequent data analysis and modeling work. Feature engineering nodes can be used to extract useful information (features) from raw data to improve model performance. Statistical analysis nodes are used to perform statistical analysis on data using statistical methods and tools. Deep learning nodes and machine learning nodes are built based on data modeling and prediction techniques, and can be used to process and learn data to make predictions or decisions. A custom script node may be a node customized by the user, or a node defined based on the needs of the user when the intelligent question-and-answer system is developed, and may specifically include code or programs written according to specific needs. Specific needs, such as database building steps, are not specifically limited in the embodiments of the present application.

[0068] S202: In the modeling interface, each target running node is arranged based on an arrangement relationship, and model parameters of each target running node are set to model a lightweight model; wherein, when the target task is a periodic task, the target running node includes at least a periodic node.

[0069] The model parameters, such as the model network structure parameters and the loss function, can be determined based on the actual situation, and the present application embodiment does not specifically limit this. The periodic task supports mining and detecting business data within a set specific time period until the results are returned.

[0070] Furthermore, the embodiment of the present application can deploy the intelligent question-and-answer system directly in the production environment of the user, specifically on each host of the user, and then access the user's private data resources for intelligent question-and-answer. In this way, it is possible to reduce factual errors in the large question-and-answer model, achieve private and accurate question-and-answer of the user's questions, and realize private domain interaction. At the same time, deploying the intelligent question-and-answer system in the user's local production environment can greatly reduce the risk of data privacy leakage.

[0071] Further, in the embodiment of the present application, the lightweight model is specifically used to label the real-time business data to obtain label result data. This step can specifically occur in the model training stage or in the model application stage.

[0072] Specifically, after step S300, the following steps S401-S402 may also be included.

[0073] S401: After the intelligent question-answering system is deployed on the host of the user, a communication channel between the intelligent question-answering system and the host is established.

[0074] The specific deployment step may refer to installing the program of the intelligent question-answering system in the host. The purpose of establishing the communication channel is to connect the private data of the user to the intelligent question-answering system.

[0075] S402: Acquire real-time business data based on the communication channel and pre-process the real-time business data. The pre-processing step at least includes storing the real-time business data in a target format.

[0076] Among them, real-time business data refers to the real-time data generated during the actual operation of third-party systems such as the user's management system. Business data may include enterprise user information, original logs, alarm data, asset information, and permission information. The preprocessing step is, for example, to convert the real-time business data into the form of an Excel table. Exemplarily, the header of the Excel table may include an IP address, operation time, and user account, etc. Among them, each column of the Excel table can be called a label. In other words, the preprocessing step can organize the data provided by the user into label data. Furthermore, when the lightweight model performs the target task, the label data can be further labeled to obtain label result data. For example, the malicious domain name analysis model can label the IP address column to obtain label result data. In this way, when the model outputs the task result, the user can determine the domain name with malicious behavior based on the column name of the labeled column and the specific IP address.

[0077] In the embodiment of the present application, before step S300, the following step S500 may also be included.

[0078] S500: Build a knowledge base for the question-answering big model so that the question-answering big model can form answers based on the knowledge base; the knowledge base at least includes training data and reference documents provided by the user stored in vector form.

[0079] The reference document is, for example, a standardized security specification (Security Standards) or compliance document for ensuring information security, network security, physical security, etc. The reference document can be obtained based on the FTP or SFTP protocol, and the reference document can also be stored in an Oracle database. It can be understood that the knowledge base is the information source when the question-answering model forms an answer.

[0080] Storing reference documents in vector form means dividing the reference documents into data blocks, such as preprocessing the reference documents according to a preset number of characters. The preprocessing step may include removing irrelevant characters and segmenting the reference documents to obtain multiple data blocks. Afterwards, NLP technology can be used to extract features from the preprocessed reference documents. The purpose of feature extraction is to convert text information into a numerical form that can be understood by a computer for further processing and analysis. Afterwards, the extracted features are converted into vector representations. Further, the converted vectors are stored in a database or a special vector search engine. The database can be a vector database (such as Faiss, Annoy, etc.), and the stored vectors can be indexed. The vector index can be constructed based on a tree structure (such as a KD tree, a ball tree, etc.) or a hashing technique (such as LSH, etc.).

[0081] It should be noted that the knowledge base may include not only training data and reference documents, but also knowledge graphs provided by users.

[0082] The following steps S601-S602 may also be included after step S300.

[0083] S601: After the large question-answering model forms an answer based on the user's question, the question and the answer are merged into a question-answer pair, and the question-answer pair is saved in the knowledge base, so that the large question-answering model is optimized based on the knowledge base; the question-answer pair includes questions and answers of one or more question-answering rounds; and / or, after the lightweight model outputs the label result data, the label result data is saved in the knowledge base, so that the large question-answering model is optimized based on the knowledge base.

[0084] In this way, based on the self-learning ability of the large question-answering model, it is possible to continuously learn and optimize based on new data fed back by the large question-answering model and / or the lightweight model, thereby further improving the quality of question-answering feedback and the accuracy of data analysis.

[0085] Furthermore, the following step S602 may be included after step S601: for each lightweight model, the corresponding execution status is collected according to the first preset period to form a task list, and a first visual interface is formed based on the task list to manage the lightweight model.

[0086] The first preset period may be hourly, daily or weekly. The task list may also be called a task management list. The first visual interface is for the management end of the intelligent question-answering system. Based on the first visual interface, the management end can quickly identify the state of the model executing the target task. Figure 4 A schematic diagram of the first visualization interface is exemplarily shown in FIG.

[0087] The execution status includes at least the model status, the number of target tasks, the accuracy of lightweight model labeling, the time consumed to execute the target tasks, the cumulative number of target tasks executed, and the time of the last execution of the target tasks. Furthermore, the model status may include normal and abnormal. The number of target tasks may be the total number of target tasks assigned or received by the lightweight model within a cycle. The accuracy of lightweight model labeling may refer to the accuracy of the model in classifying, identifying or annotating data when executing the target tasks. The time consumed to execute the target tasks may refer to the time required for the lightweight model to complete a single or average target task. The cumulative number of target tasks executed refers to the total number of target tasks completed since the lightweight model was deployed. The time of the last execution of a target task may refer to the specific time when the lightweight model last executed the target task.

[0088] It can be understood that the first visualization interface can be used to represent the usage of the lightweight model by multiple users, and the intelligent question and answer system management end can interact with the first visualization interface to perform batch operations on the task status in the task list, such as deleting data.

[0089] Furthermore, after step S300, the following step S603 may be included: modifying the model parameters of the corresponding running nodes of the lightweight model based on the label result data.

[0090] In practical applications, users can determine the accuracy of the lightweight model based on the label result data. After receiving feedback on the accuracy, they can adjust the model parameters and related indicators of the lightweight model based on the accuracy, thereby assisting in the optimization of the lightweight model.

[0091] An embodiment of the present application also provides a security field intelligent question and answer system based on a large model. The intelligent question and answer system can be constructed based on the construction method provided in the aforementioned embodiment.

[0092] Figure 5 A schematic diagram of the structure of the intelligent question-answering system provided in an embodiment of the present application.

[0093] like Figure 5 As shown, the intelligent question-answering system may include a large question-answering model 1001 and at least one lightweight model 1002. The large question-answering model 1001 is used to form answers to the user's questions, and the lightweight model 1002 is used to perform target tasks based on the user's real-time business data, and the target tasks are issued by the user. The intelligent question-answering system also includes a knowledge base, which is built based on historical business data and reference documents provided by the user.

[0094] Furthermore, the question-answer model 1001 is specifically used to execute the following steps S701-S703.

[0095] S701: Search the knowledge base based on the question raised by the user, and / or use a search engine to search and obtain multiple candidate documents.

[0096] In the embodiment of the present application, the question-answering model 1001 can recall multiple candidate documents from multiple different channels. The specific number of candidate documents recalled can be equal to 30, for example, and the embodiment of the present application does not specifically limit this. Different channels can refer to knowledge base channels and external network data (search engine channels).

[0097] Furthermore, the step of searching the knowledge base based on the question raised by the user at least includes vectorizing the question and matching the reference document with the vector based on the question. Specifically, the intelligent question answering system can first convert the question into a high-dimensional vector, which can be achieved through word embedding or semantic embedding technology (such as BERT, GPT, etc.) in natural language processing technology. In this way, the semantic information of the question can be represented in the form of a vector.

[0098] The intelligent question-answering system provided in the embodiment of the present application can also use search engines or external network data sources for retrieval, which can be done by directly querying the search engine or calling a specific network data API. In this way, the intelligent question-answering system can obtain the latest information that may not be in the knowledge base, thereby increasing the diversity and breadth of candidate documents.

[0099] S702: For each candidate document, calculate a first similarity.

[0100] The first similarity is calculated based on the following formula:

[0101] score=0.2*score fuzzy +0.1*score cosine +0.4*score BM25 +0.3*score Levenshtein ;

[0102] Among them, score represents the first similarity; score fuzzy Indicates the score of the fuzzy similarity algorithm between the question and the candidate document, score cosine The score represents the cosine similarity calculation score of the question and candidate document after vectorization. BN25 The score represents the BM25 similarity algorithm score of the question and the candidate document. Levenshtein represents the score of the Levenshtein distance similarity algorithm between the question and the candidate document, and 0.2, 0.1, 0.4 and 0.3 represent the weighting coefficients.

[0103] Specifically, the fuzzy similarity algorithm can calculate the degree of fuzzy matching between two strings based on characters or word units, and the fuzzy matching algorithm can be used to calculate the similarity in the embodiment of the present application. Cosine similarity is an indicator that measures the degree of similarity between the directions of two vectors. In the intelligent question-answering system, the cosine value between the two vectors corresponding to the question and the candidate document can be calculated. The closer the cosine value is to 1, the closer the directions of the two vectors are, that is, the higher the semantic similarity. BM25 is an information retrieval algorithm based on a probabilistic retrieval model, which takes into account multiple factors such as term frequency (TF), inverse document frequency (IDF), and the position of the word in the document to evaluate the relevance between the query and the document. The BM25 algorithm is widely used in search engines because it can effectively process large-scale text data. The Levenshtein distance is also called the edit distance, which refers to the minimum number of single-character edits required to convert one string to another, and editing refers to insertion, deletion, or replacement.

[0104] The weighting coefficient can be adjusted based on the actual situation so that the intelligent question answering system can more comprehensively and accurately evaluate the matching degree between the candidate document and the question, thereby providing a more accurate and useful answer. The embodiment of the present application does not specifically limit the weighting coefficient.

[0105] S703: Select n candidate documents corresponding to the first similarities as relevant documents in descending order of the first similarities, form answers based on the relevant documents, and feed back the answers to the user.

[0106] In the embodiment of the present application, n can be specifically equal to 3, 4 or 5, and the embodiment of the present application does not make any specific limitation on this. In this way, it can be ensured that the documents recalled from the knowledge base are highly consistent with the semantic content of the question.

[0107] The step of forming answers based on relevant documents can also be called the step of prompt templating. Prompt templating specifically includes creating a set of prompts in fixed formats that cover all the key information needed in a specific situation. For example, when outputting the operating status of a management system, the template can include the system name, current status, main indicators (such as CPU usage, memory usage, disk space, etc.), warning / error information, and recommended operations. This is only an example introduction.

[0108] After step S703, the question-answer model 1001 is also used to execute the following steps S704-S705.

[0109] S704: Calculate the accuracy, recall and F1 value; wherein, the formula for calculating the accuracy is: accuracy P = total number of relevant documents / total number of candidate documents*100%; the formula for calculating the recall is: recall R = total number of relevant documents / total number of reference documents*100%; the formula for calculating the F1 value is: P*R*2 / (P+R).

[0110] It is understandable that accuracy, recall and F1 value are important indicators for evaluating the quality of large question-answering models.

[0111] S705: According to the second preset period, the precision, recall and F1 value are saved in the database.

[0112] The second preset period is, for example, daily, weekly or monthly. In the second preset period, the accuracy, recall and F1 value can be called self-monitoring results, which can be saved in the Syslog database. Syslog is a standard protocol for transmitting log information. The monitoring system can be configured or a special tool (such as rsyslog, syslog-ng) can be used to send important monitoring events (accuracy, recall and F1 value) to the log server for centralized management.

[0113] In an embodiment of the present application, in order to evaluate the performance of the lightweight model, indicators related to the lightweight model, such as delay time, throughput, resource utilization and stability indicators, etc., can also be calculated. Among them, the delay time is also called delay, which specifically refers to the event required for the lightweight model to process a single request, and each running node corresponds to a request. Throughput time refers to the number of requests that a lightweight model can process per unit time. Resource utilization may include the utilization of resources such as CPU, memory and GPU, which is used to evaluate the resource consumption of the lightweight model. Stability indicators include, for example, error rate and anomaly detection count. Furthermore, in an embodiment of the present application, the values ​​of various indicators can also be saved in a database through the Syslog protocol for long-term storage and subsequent query analysis.

[0114] The intelligent question-answering system provided in the embodiment of the present application may further include a calling module 1003, and the calling module 1003 is specifically used to execute the following steps S801-S803.

[0115] S801: In response to a target task issued by a user of the intelligent question-answering system, target business data and a target lightweight model for executing the target task are determined based on the type of the target task.

[0116] In some implementations, the target lightweight model may also be directly specified when the user publishes the target task.

[0117] S802: Calling a target lightweight model so that the target lightweight model labels the real-time business data to obtain label result data.

[0118] The intelligent question-answering system provided in the embodiment of the present application can not only quickly identify and determine the target business data required to perform the target task based on the type of the target task, but also match the target lightweight model to perform the task. Specifically, the task description can be parsed based on natural language processing (NLP) technology to determine the target lightweight model. For example, the name of the lightweight model 1002 is matched with the task description to determine the target lightweight model 1.

[0119] S803: forming a second visualization interface based on the label result data, and displaying the second visualization interface to the user; and saving the label result data to a database.

[0120] In the embodiment of the present application, the label result data can be saved in the database through the Syslog protocol.

[0121] It can be seen that the embodiment of the present application can generate a second visual interface to present complex label result data to the user in an intuitive and easy-to-understand form, so that the user can quickly understand the data status and business trends. At the same time, the embodiment of the present application can also automatically save the label result data to the database to ensure the traceability and security of the data.

[0122] It is worth noting that the embodiment of the present application can also save the label result data into the knowledge base so that the question and answer model 1001 can complete self-optimization.

[0123] In some implementations, the intelligent question-and-answer system provided in the embodiments of the present application can also generate alarms for abnormal data. For example, when the abnormal value in the label result data exceeds a preset threshold, an alarm message can be generated and an alarm process can be triggered.

[0124] Figure 6 A schematic diagram of the structure of a device for constructing a security field intelligent question-answering system based on a large model provided in an embodiment of the present application.

[0125] like Figure 6 As shown, the embodiment of the present application provides a device for constructing an intelligent question-answering system in the security field based on a large model, including:

[0126] The acquisition module 2001 is used to acquire training data, and the training data at least includes historical business data in the security field provided by the user of the intelligent question-answering system, and historical business data acquired from a third-party security platform through the first interface API.

[0127] The first construction module 2002 is used to construct a large question-answering model based on training data, and to construct at least one lightweight model; wherein the large question-answering model is used to form answers to the user's questions, and the lightweight model is used to perform target tasks based on the user's real-time business data, and the target tasks are issued by the user.

[0128] The second construction module 2003 is used to construct an intelligent question-answering system based on the large question-answering model and the lightweight model.

[0129] In some implementations, the first construction module 2002 is specifically used to determine the target operating nodes included in the lightweight model and the arrangement relationship between each target operating node based on the user's task requirements for the target task; the task requirements at least include the detection and analysis requirements for business data, the prediction requirements for numerical values, and the classification requirements for business data; in the modeling interface, the target operating nodes are arranged based on the arrangement relationship, and the model parameters of each target operating node are set to model the lightweight model; wherein, when the target task is a periodic task, the target operating nodes at least include periodic nodes.

[0130] In some implementations, the lightweight model is specifically used to label real-time business data to obtain label result data; the first construction module 2002 is also used to: build a knowledge base for the question-answering big model, so that the question-answering big model can form answers based on the knowledge base; the knowledge base at least includes training data and reference documents provided by the user stored in vector form.

[0131] The first construction module 2002 is also used for: after the question-answering large model forms an answer based on the user's question, merging the question and the answer into a question-answer pair, and saving the question-answer pair into the knowledge base, so that the question-answering large model is optimized based on the knowledge base; the question-answer pair includes questions and answers of one or more question-answering rounds; and / or, after the lightweight model outputs the label result data, saving the label result data into the knowledge base, so that the question-answering large model is optimized based on the knowledge base; and, the second construction module 2003 is also used for: for each lightweight model, collecting its corresponding execution status according to the first preset period to form a task list, and forming a first visual interface based on the task list to manage the lightweight model; the execution status includes at least the model status, the number of target tasks, the accuracy of lightweight model labeling, the time consumed in executing the target tasks, the cumulative number of target tasks executed, and the time of the last execution of the target tasks.

[0132] In some implementations, the first construction module 2002 is further configured to modify model parameters of a corresponding running node of the lightweight model based on the label result data.

[0133] In some implementations, the second construction module 2003 is also used to establish a communication channel between the intelligent question and answer system and the host after the intelligent question and answer system is deployed on the user's host; obtain real-time business data based on the communication channel, and pre-process the real-time business data, and the pre-processing step at least includes storing the real-time business data in a target format.

[0134] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the method for constructing a security field intelligent question-answering system based on a large model provided by the present invention. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0135] It is easy to understand that those skilled in the art can combine, split, reorganize, etc. the embodiments of the present application to obtain other embodiments based on the several embodiments provided in the present application, and these embodiments do not exceed the protection scope of the present application.

[0136] The above specific implementation methods further explain in detail the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above are only specific implementation methods of the embodiments of the present application and are not used to limit the protection scope of the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. A method for constructing an intelligent question-answering system in the security field based on a large model, characterized in that: include: Acquire training data, the training data including at least historical business data in the security field provided by the user of the intelligent question-answering system, and historical business data acquired from a third-party security platform through a first interface API; Building a large question-answering model based on the training data, and building at least one lightweight model; wherein the large question-answering model is used to form an answer to the user's question, and the lightweight model is used to perform a target task based on the user's real-time business data, and the target task is issued by the user; An intelligent question-answering system is constructed based on the large question-answering model and the lightweight model.

2. The method for constructing a security field intelligent question-answering system based on a large model according to claim 1 is characterized in that: The steps of constructing at least one lightweight model include: Based on the task requirements of the user for the target task, determine the target operation nodes included in the lightweight model and the arrangement relationship between each of the target operation nodes; the task requirements at least include the detection and analysis requirements for business data, the prediction requirements for values, and the classification requirements for business data; In the modeling interface, each of the target operation nodes is arranged based on the arrangement relationship, and the model parameters of each of the target operation nodes are set to model the lightweight model; wherein, when the target task is a periodic task, the target operation nodes include at least periodic nodes.

3. The method for constructing a security field intelligent question-answering system based on a large model according to claim 1 is characterized in that: The lightweight model is specifically used to label the real-time business data to obtain label result data; Before the step of building an intelligent question-answering system based on the large question-answering model and the lightweight model, the method further includes: Constructing a knowledge base for the question-answering big model so that the question-answering big model can form answers based on the knowledge base; the knowledge base at least includes the training data and reference documents provided by the user stored in vector form; After the step of building an intelligent question-answering system based on the large question-answering model and the lightweight model, the method further includes: After the question-answering big model forms an answer based on the question of the user, the question and the answer are combined into a question-answering pair, and the question-answering pair is saved in the knowledge base, so that the question-answering big model is optimized based on the knowledge base; the question-answering pair includes questions and answers of one or more question-answering rounds; and / or, After the lightweight model outputs the label result data, the label result data is saved in the knowledge base, so that the question-answering large model is optimized based on the knowledge base; and, for each of the lightweight models, its corresponding execution status is collected according to a first preset period to form a task list, and a first visual interface is formed based on the task list to manage the lightweight model; the execution status includes at least the model status, the number of target tasks, the accuracy of labeling by the lightweight model, the time consumed in executing the target tasks, the cumulative number of executions of the target tasks, and the time of the last execution of the target tasks.

4. The method for constructing a security field intelligent question-answering system based on a large model according to claim 3 is characterized in that: After the step of building an intelligent question-answering system based on the large question-answering model and the lightweight model, the method further includes: The model parameters of the corresponding running nodes of the lightweight model are modified based on the label result data.

5. The method for constructing a security field intelligent question-answering system based on a large model according to claim 1 is characterized in that: After the step of building an intelligent question-answering system based on the large question-answering model and the lightweight model, the method further includes: After the intelligent question-answering system is deployed on the host of the user, a communication channel between the intelligent question-answering system and the host is established; The real-time service data is acquired based on the communication channel, and the real-time service data is preprocessed, wherein the preprocessing step at least includes storing the real-time service data in a target format.

6. A security field intelligent question answering system based on a large model, characterized in that: The intelligent question-answering system is constructed based on the construction method described in any one of claims 1 to 5; The intelligent question-and-answer system includes a large question-and-answer model and at least one lightweight model. The large question-and-answer model is used to form answers to questions from users, and the lightweight model is used to execute target tasks based on the real-time business data of the user, and the target tasks are issued by the user.

7. The security domain intelligent question-answering system based on a large model according to claim 6 is characterized in that: The intelligent question-answering system also includes a knowledge base, which is constructed based on historical business data and reference documents provided by the user; The question-answering model is specifically used for: The knowledge base is searched based on the question raised by the user, and / or a search engine is used to search to obtain a plurality of candidate documents; the step of searching the knowledge base based on the question raised by the user at least includes vectorizing the question and performing vector matching on the reference document based on the question; For each of the candidate documents, calculating a first similarity; In descending order of the first similarities, n candidate documents corresponding to the first similarities are selected as relevant documents, an answer is formed based on the relevant documents, and the answer is fed back to the user; The first similarity is calculated based on the following formula: score=0.2*score fuzzy +0.1*score cosine +0.4*score BM25 +0.3*score Levenshtein ; Wherein, score represents the first similarity; score fuzzy Indicates the score of the fuzzy similarity algorithm between the question and the candidate document, score cosine The score represents the score of the cosine similarity calculation between the question and the candidate document after vectorization. BM25 The score represents the BM25 similarity algorithm score between the question and the candidate document. Levenshtein represents the score of the Levenshtein distance similarity algorithm between the question and the candidate document, and 0.2, 0.1, 0.4 and 0.3 represent weighting coefficients.

8. The security domain intelligent question-answering system based on a large model according to claim 7 is characterized in that: After the step of selecting n candidate documents corresponding to the first similarities in descending order to form an answer and feeding the answer back to the user, the question-answering model is further used to: Calculate the accuracy, recall and F1 value; wherein the formula for calculating the accuracy is: accuracy P = the total number of relevant documents / the total number of candidate documents*100%; the formula for calculating the recall is: recall R = the total number of relevant documents / the total number of reference documents*100%; the formula for calculating the F1 value is: P*R*2 / (P+R); According to a second preset period, the accuracy, the recall rate and the F1 value are saved in a database.

9. The security domain intelligent question-answering system based on a large model according to claim 6 is characterized in that: It also includes a model calling module, which is specifically used to: In response to a target task issued by a user of the intelligent question-answering system, determining target business data and a target lightweight model for executing the target task based on the type of the target task; Calling the target lightweight model so that the target lightweight model labels the real-time business data to obtain label result data; forming a second visualization interface based on the label result data, and displaying the second visualization interface to the user; And, the label result data is saved in a database.

10. A device for constructing an intelligent question-answering system in the security field based on a large model, characterized in that: include: An acquisition module, used to acquire training data, wherein the training data at least includes historical business data in the security field provided by the user of the intelligent question-answering system, and historical business data acquired from a third-party security platform through a first interface API; A first construction module is used to construct a large question-answering model based on the training data, and to construct at least one lightweight model; wherein the large question-answering model is used to form an answer to the user's question, and the lightweight model is used to perform a target task based on the user's real-time business data, and the target task is issued by the user; The second construction module is used to build an intelligent question-answering system based on the large question-answering model and the lightweight model.

Images