A method, apparatus, and electronic device for retrieving threat intelligence.

Abstract

The present application provides a threat intelligence retrieval question and answer method, device, electronic device and storage medium, wherein the method includes: obtaining a pre-processed threat intelligence query statement; performing entity extraction on the threat intelligence query statement to obtain the entity of the threat intelligence query statement; performing structured processing on the entity to obtain a first text data; performing unstructured processing on the threat intelligence query statement to obtain a second text data; fusing the first text data and the second text data to obtain the question and answer result of the threat intelligence query statement. The implementation of the present application can realize the retrieval question and answer of threat intelligence, improve the professionalism of the threat intelligence question and answer, not impose restrictions on the user's search, improve the quality of the answer, not rely too much on the understanding of the user input, and can maximize the accuracy and scalability of the answer.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and more specifically, to a threat intelligence retrieval question-answering method, device, electronic device, and storage medium. Background Art

[0002] Cybersecurity has become a key factor in enterprise development. Understanding threat intelligence plays a vital role in enterprise security operations. However, in existing technologies, question-answering tools based on threat intelligence have many problems.

[0003] For example, search question-answering based on a third-party search engine usually involves integrating the searched web page results directly into the prompt to answer the question. Or sorting and filtering the data to achieve general domain question-answering. This implementation method usually cannot give satisfactory answers to questions that require deep domain knowledge, and lacks professionalism in answering domain knowledge. Existing generative retrieval question-answering systems have deficiencies in achieving interaction with underlying data, which limits the ability to obtain data that meets user needs, thereby affecting the quality of answer generation. There are also deficiencies in effectively combining databases and knowledge base documents, which is particularly evident in user question understanding and data scheduling. Existing technologies rely too much on understanding user input to achieve scheduling of different data sources. Intent-based methods require a large number of intent categories to be defined in advance, and are difficult to achieve expectations in terms of accuracy and scalability. Summary of the invention

[0004] The purpose of this application is to provide a threat intelligence retrieval question and answer method, device, electronic device and storage medium, which can realize the retrieval question and answer of threat intelligence, improve the professionalism of threat intelligence question and answer, do not restrict the user's search, improve the quality of the answer, do not rely too much on the understanding of the user input, and can maximize the accuracy and scalability of the answer.

[0005] In a first aspect, an embodiment of the present application provides a threat intelligence retrieval question-answering method, the method comprising:

[0006] Obtain preprocessed threat intelligence query statements;

[0007] Performing entity extraction on the threat intelligence query statement to obtain an entity of the threat intelligence query statement;

[0008] Performing structural processing on the entity to obtain first text data;

[0009] Performing unstructured processing on the threat intelligence query statement to obtain second text data;

[0010] The first text data and the second text data are merged to obtain a question and answer result of the threat intelligence query statement.

[0011] In the above implementation process, by parsing, expanding and aggregating questions based on threat intelligence query statements, structured databases and knowledge base documents, it is possible to realize retrieval and answer questions about threat intelligence, improve the professionalism of threat intelligence questions and answers, not impose limits on user retrieval, improve the quality of answers, not rely too much on the understanding of user input, and maximize the accuracy and scalability of answers.

[0012] Furthermore, the step of extracting entities from the threat intelligence query statement to obtain entities of the threat intelligence query statement includes:

[0013] Perform entity extraction on the threat intelligence query statement according to a pre-built first large model to obtain a first entity set;

[0014] Perform keyword matching on the threat intelligence query statement according to a predefined entity word list to obtain a second entity set;

[0015] Taking a union of the first entity set and the second entity set to obtain the entity.

[0016] In the above implementation process, two types of entity extraction are performed on the structured data respectively. Different extraction methods are used for different entity states, which can achieve deep analysis and accurate recall of entities, without restricting user retrieval and improving the quality of answers.

[0017] Furthermore, the step of performing structural processing on the entity to obtain the first text data includes:

[0018] Obtaining descriptive text data corresponding to the entity relationship between the entities;

[0019] Obtaining single entity text data corresponding to each of the entities;

[0020] The descriptive text data and the single entity text data are concatenated to obtain the first text data.

[0021] In the above implementation process, the entity relationships between multiple entities and individual entities are structured accordingly and spliced ​​together, which can maximize the improvement of structured data and achieve better question and answer understanding.

[0022] Furthermore, the step of obtaining descriptive text data corresponding to the entity relationship between the entities includes:

[0023] Obtaining entity relationships between the entities;

[0024] The link relationship between entities is extracted from the entity relationship according to a pre-built link prediction model to obtain the descriptive text.

[0025] In the above implementation process, the extraction of descriptive text is realized according to the link relationship between different entities, which can expand the interactivity between entities and provide more possibilities for answer generation.

[0026] Furthermore, the step of obtaining the single entity text data corresponding to each of the entities includes:

[0027] Performing data pair matching on each of the entities to obtain a structured query statement corresponding to each of the entities;

[0028] Optimizing the structured query statement according to the pre-built second largest model to obtain a target structured query statement;

[0029] The target structured query statement is matched in a structured database to obtain the single entity text data.

[0030] In the above implementation process, the corresponding structured query statement is obtained according to the conversion data of a single entity, so that a single entity can be converted quickly and accurately, improving data utilization efficiency and reducing the limitations of single entity intent recognition.

[0031] Furthermore, the step of performing unstructured processing on the threat intelligence query statement to obtain second text data includes:

[0032] Expanding the threat intelligence query statement according to the pre-constructed prompt words to obtain an expanded threat intelligence query statement;

[0033] Performing vectorization processing on the expanded threat intelligence query statement to obtain vectorized text data;

[0034] Vectorize the pre-built knowledge base documents to obtain segmented text data;

[0035] Interactively processing the vectorized text data and the segmented text data to obtain third text data;

[0036] The third text data is accurately sorted to obtain the second text data.

[0037] In the above implementation process, the threat intelligence query statement is expanded and vectorized, and the knowledge base document is vectorized at the same time, which can expand the data sample range and improve the scalability and practicality of the second text data.

[0038] Furthermore, the step of accurately sorting the third text data to obtain the second text data includes:

[0039] The third text data is sequentially subjected to inverse sorting fusion processing and custom vector scoring weighting processing to obtain fourth text data;

[0040] The data meeting the requirements in the fourth text data are selected according to a preset threshold value to obtain the second text data.

[0041] In the above implementation process, the third text data is sorted and fused in reverse order, and the text data with high weight is selected for further screening, which can improve the availability and accuracy of structured data and realize precise retrieval and question answering.

[0042] Furthermore, the step of fusing the first text data and the second text data to obtain a question-and-answer result of the threat intelligence query statement includes:

[0043] Sequentially concatenating the first text data and the second text data to obtain concatenated text data;

[0044] Adjusting the concatenated text data to obtain question-answer text data;

[0045] The question and answer text data is input into a pre-built third model to retrieve the question and answer, and obtain the question and answer result.

[0046] In the above implementation process, splicing structured data and unstructured data can improve the accuracy and weight of the question and answer results, making the question and answer results more scalable.

[0047] Furthermore, the step of adjusting the concatenated text data to obtain question-answer text data includes:

[0048] Obtaining the length value of the concatenated text data;

[0049] Determine whether the length of the concatenated text data is greater than a preset length threshold;

[0050] If yes, mapping the concatenated text data according to the first template to obtain the question and answer text data;

[0051] If not, the concatenated text data is mapped according to the second template to obtain the question and answer text data.

[0052] In the above implementation process, the concatenated text data is mapped according to different mapping templates to achieve expansion of different data, making the question and answer text data more accurate.

[0053] In a second aspect, an embodiment of the present application further provides a threat intelligence retrieval question-answering device, the device comprising:

[0054] The acquisition module is used to obtain the pre-processed threat intelligence query statement;

[0055] An extraction module, used to extract entities from the threat intelligence query statement to obtain entities of the threat intelligence query statement;

[0056] A structured processing module, used for performing structured processing on the entity to obtain first text data;

[0057] An unstructured processing module, used to perform unstructured processing on the threat intelligence query statement to obtain second text data;

[0058] A fusion module is used to fuse the first text data and the second text data to obtain a question and answer result of the threat intelligence query statement.

[0059] In the above implementation process, by parsing, expanding and aggregating questions based on threat intelligence query statements, structured databases and knowledge base documents, it is possible to realize retrieval and answer questions about threat intelligence, improve the professionalism of threat intelligence questions and answers, not impose limits on user retrieval, improve the quality of answers, not rely too much on the understanding of user input, and maximize the accuracy and scalability of answers.

[0060] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0061] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0062] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method as described in any one of the first aspects.

[0063] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0064] And it can be implemented according to the contents of the specification. The following is a detailed description of the preferred embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the range values. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 A flowchart of a threat intelligence retrieval question-answering method provided in an embodiment of the present application;

[0067] Figure 2 A schematic diagram of the structure of a threat intelligence retrieval and question-answering device provided in an embodiment of the present application;

[0068] Figure 3 A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0070] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0071] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0072] Embodiment 1

[0073] Figure 1 FIG. 1 is a flow chart of a threat intelligence retrieval question-answering method provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0074] S1, obtain the pre-processed threat intelligence query statement;

[0075] S2, extracting entities from the threat intelligence query statement to obtain entities of the threat intelligence query statement;

[0076] S3, performing structural processing on the entity to obtain first text data;

[0077] S4, performing unstructured processing on the threat intelligence query statement to obtain second text data;

[0078] S5, fusing the first text data and the second text data to obtain a question and answer result of the threat intelligence query statement.

[0079] In the above implementation process, by parsing, expanding and aggregating questions based on threat intelligence query statements, structured databases and knowledge base documents, it is possible to realize retrieval and answer questions about threat intelligence, improve the professionalism of threat intelligence questions and answers, not impose limits on user retrieval, improve the quality of answers, not rely too much on the understanding of user input, and maximize the accuracy and scalability of answers.

[0080] The method proposed in this application dynamically recommends a small number of sample examples for each question (the user's threat intelligence query statement) to optimize the selection of data calling tools. In the tool function preprocessing, a detailed description of its function is given for each tool. In the question and answer result analysis process, tool selection strategy examples for similar question and answer results are recalled from the database based on the semantic similarity of the question and answer results, and then given to the prompt text. By replacing intent recognition with tool selection, the frequency of rule maintenance and the long-term nature of classification model training iterations in traditional methods are effectively reduced, and the problem of gradually declining performance as the number of categories increases is also solved.

[0081] In S1, unstructured threat intelligence query statements input by users are preprocessed, such as generating question-answer pair data, deeply analyzing document content, and vectorizing documents. This can implement a series of optimization processes such as error correction and statement rewriting on the original query data to ensure the accuracy and fluency of threat intelligence query statements, and realize dynamic selection and task diversion of subsequent data calls.

[0082] The preprocessing of this application includes first splitting the text of the threat intelligence query statement into paragraphs, dividing a document into several paragraphs of equal length, and vectorizing each paragraph based on a semantic model to facilitate subsequent data matching.

[0083] Furthermore, S2 includes:

[0084] Perform entity extraction on the threat intelligence query statement according to the pre-built first model to obtain a first entity set;

[0085] Perform keyword matching on the threat intelligence query statement according to a predefined entity word list to obtain a second entity set;

[0086] The first entity set and the second entity set are taken as a union to obtain an entity.

[0087] In the above implementation process, two types of entity extraction are performed on the structured data respectively. Different extraction methods are used for different entity states, which can achieve deep analysis and accurate recall of entities, without restricting user retrieval and improving the quality of answers.

[0088] In the data selection process of the structured database, the threat intelligence query statements queried by the user are deeply analyzed, including entity extraction and entity relationship extraction. The entity extraction method mainly considers the first large model after fine-tuning based on the entity extraction task.

[0089] In addition, we also consider extracting entities based on a predefined list of entity words using a keyword matching method.

[0090] Furthermore, S3 includes:

[0091] Obtain descriptive text data corresponding to entity relationships between entities;

[0092] Get the single entity text data corresponding to each entity;

[0093] The descriptive text data and the single entity text data are concatenated to obtain first text data.

[0094] In the above implementation process, the entity relationships between multiple entities and individual entities are structured accordingly and spliced ​​together, which can maximize the improvement of structured data and achieve better question and answer understanding.

[0095] Furthermore, the step of obtaining descriptive text data corresponding to the entity relationship between entities includes:

[0096] Get entity relationships between entities;

[0097] The link relationships between entities are extracted based on the pre-built link prediction model to obtain descriptive text.

[0098] In the above implementation process, the extraction of descriptive text is realized according to the link relationship between different entities, which can expand the interactivity between entities and provide more possibilities for answer generation.

[0099] For the determination of direct relationships between different entities, a graph association algorithm is used to realize the association of data between entities. This application trains a link prediction model for predicting the relationship between entities by aggregating entities (i.e., the features of other entities within the k-hop range) based on a random walk method on a heterogeneous graph, and predicts the connection relationship between different pairs of entities extracted on the graph based on the trained link prediction model, thereby realizing the extraction of relationships between multiple entities. The edge relationship description field in the auxiliary structured database is associated based on the edge relationship to supplement the edge relationship with better explanatory description text.

[0100] Furthermore, the step of obtaining the single entity text data corresponding to each entity includes:

[0101] Match each entity with data pairs to obtain the structured query statement corresponding to each entity;

[0102] Optimizing the structured query statement according to the pre-built second largest model to obtain a target structured query statement;

[0103] Match the target structured query statement in the structured database to obtain single entity text data.

[0104] In the above implementation process, the corresponding structured query statement is obtained according to the conversion data of a single entity, so that a single entity can be converted quickly and accurately, improving data utilization efficiency and reducing the limitations of single entity intent recognition.

[0105] In order to further improve the data recall capability, this application generates high-quality QA question-answer pairs (data pairs) based on the content in the threat intelligence query statement, and on this basis, similar difficulties are expanded for each transformed data pair, and a specified number of answers are generated to ensure that the questions can be recalled from the questions, and then the answers are associated to improve data recall.

[0106] For a single entity, the SQL query statement is generated through the dynamic and adaptive text2sql technology to achieve accurate extraction of the answer. For different single entities, similar text-sql data pairs are recalled from the structured database based on semantic similarity. During the user query conversion to SQL, the recalled similar text-sql data pairs are filled into the prompt text to achieve more accurate SQL conversion and more accurate data recall.

[0107] Furthermore, S4 includes:

[0108] The threat intelligence query statement is expanded according to the pre-built prompt words to obtain an expanded threat intelligence query statement;

[0109] Perform vectorization processing on the expanded threat intelligence query statement to obtain vectorized text data;

[0110] Vectorize the pre-built knowledge base documents to obtain segmented text data;

[0111] Interactively processing the vectorized text data and the segmented text data to obtain third text data;

[0112] The third text data is accurately sorted to obtain the second text data.

[0113] In the above implementation process, the threat intelligence query statement is expanded and vectorized, and the knowledge base document is vectorized at the same time, which can expand the data sample range and improve the scalability and practicality of the second text data.

[0114] In the data aggregation stage, this application integrates unstructured data. Since there is an expansion of threat intelligence query statements in the initial question-and-answer analysis process, user queries are expanded to generate multiple similar and different threat intelligence query statements.

[0115] Furthermore, the step of accurately sorting the third text data to obtain the second text data includes:

[0116] The third text data is sequentially subjected to inverse sorting fusion processing and custom vector scoring weighting processing to obtain fourth text data;

[0117] According to a preset threshold, data meeting the requirements in the fourth text data is selected to obtain the second text data.

[0118] In the above implementation process, the third text data is sorted and fused in reverse order, and the text data with high weight is selected for further screening, which can improve the availability and accuracy of structured data and realize precise retrieval and question answering.

[0119] The present application performs data recall in a multi-channel manner, and implements overall sorting of the structured data recalled from the knowledge base documents by the third text data using a fusion of inverse sorting and a custom vector scoring weighting method, and determines a suitable threshold through overall analysis of the data, thereby ensuring that more accurate data can be retained.

[0120] Furthermore, S5 includes:

[0121] Sequentially concatenating the first text data and the second text data to obtain concatenated text data;

[0122] Adjust the concatenated text data to obtain question-answer text data;

[0123] The question and answer text data is input into the pre-built third model to retrieve the question and answer and obtain the question and answer results.

[0124] In the above implementation process, splicing structured data and unstructured data can improve the accuracy and weight of the question and answer results, making the question and answer results more scalable.

[0125] In the data splicing stage, structured data and unstructured data need to be spliced ​​to construct complete prompt text filling data (question and answer text data).

[0126] Put higher-quality structured data in front, splice unstructured data after it, and give suggestive titles to each type of data to help the third model better understand the meaning of the given data.

[0127] Furthermore, the step of adjusting the concatenated text data to obtain question-answer text data includes:

[0128] Get the length value of the concatenated text data;

[0129] Determine whether the length of the concatenated text data is greater than a preset length threshold;

[0130] If yes, mapping the concatenated text data according to the first template to obtain question and answer text data;

[0131] If not, the concatenated text data is mapped according to the second template to obtain question and answer text data.

[0132] In the above implementation process, the concatenated text data is mapped according to different mapping templates to achieve expansion of different data, making the question and answer text data more accurate.

[0133] This application considers dynamically calculating the length of the spliced ​​text data to adjust the context window size and template of the question and answer text data. If the length value of the spliced ​​text data is greater than the preset length threshold, only the spliced ​​text data is considered for interpretation, because for such questions, the context scenario is given to the user and no additional recall data is required. The prompt text for such questions only needs to give a simple data interpretation type (first template) prompt text. When the length value of the spliced ​​text data is less than or equal to the specified threshold, consider recalling and sorting structured data and unstructured data, and select prompt text containing spliced ​​text data and context filling, and smoothly dynamically adjust the length of the question and answer text data filling content according to the length of the spliced ​​text data (the longer the spliced ​​text data, the shorter the question and answer result, because the total length that can be input by the third largest model is limited), thereby realizing the construction of the question and answer result.

[0134] Optionally, the first large model, the second large model, and the third large model involved in the embodiments of the present application are all deep learning models pre-built using existing technologies.

[0135] This application improves the accuracy of query understanding and the quality of answers by correcting query statement errors, splitting, and rewriting problems. It uses a large language model to dynamically select data and recommend examples to optimize tool selection, improve stability, and simplify maintenance. In addition, by initially diverting questions, unnecessary tool calls are avoided, improving the efficiency of task allocation.

[0136] By processing knowledge base documents and using dynamic text2sql methods, data recall accuracy and information retrieval capabilities are improved. Combining reciprocal sorting fusion, custom vector scoring weighting, Markdown format structured data processing, and the fusion of professional search engines and structured database data, data aggregation of unstructured knowledge base documents, improved question and answer accuracy, and effective question and answer for professional fields and traditional questions are achieved.

[0137] Embodiment 2

[0138] In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, a threat intelligence retrieval question-answering device is provided below, such as Figure 2 As shown, the device comprises:

[0139] Acquisition module 1, used to obtain the pre-processed threat intelligence query statement;

[0140] Extraction module 2, used to extract entities from threat intelligence query statements to obtain entities of threat intelligence query statements;

[0141] A structured processing module 3, used for performing structured processing on the entity to obtain first text data;

[0142] The unstructured processing module 4 is used to perform unstructured processing on the threat intelligence query statement to obtain second text data;

[0143] The fusion module 5 is used to fuse the first text data and the second text data to obtain the question and answer result of the threat intelligence query statement.

[0144] In the above implementation process, by parsing, expanding and aggregating questions based on threat intelligence query statements, structured databases and knowledge base documents, it is possible to realize retrieval and answer questions about threat intelligence, improve the professionalism of threat intelligence questions and answers, not impose limits on user retrieval, improve the quality of answers, not rely too much on the understanding of user input, and maximize the accuracy and scalability of answers.

[0145] Furthermore, the extraction module 2 is also used for:

[0146] Perform entity extraction on the threat intelligence query statement according to the pre-built first model to obtain a first entity set;

[0147] Perform keyword matching on the threat intelligence query statement according to a predefined entity word list to obtain a second entity set;

[0148] The first entity set and the second entity set are taken as a union to obtain an entity.

[0149] In the above implementation process, two types of entity extraction are performed on the structured data respectively. Different extraction methods are used for different entity states, which can achieve deep analysis and accurate recall of entities, without restricting user retrieval and improving the quality of answers.

[0150] Furthermore, the structured processing module 3 is also used for:

[0151] Obtain descriptive text data corresponding to entity relationships between entities;

[0152] Get the single entity text data corresponding to each entity;

[0153] The descriptive text data and the single entity text data are concatenated to obtain first text data.

[0154] In the above implementation process, the entity relationships between multiple entities and individual entities are structured accordingly and spliced ​​together, which can maximize the improvement of structured data and achieve better question and answer understanding.

[0155] Furthermore, the structured processing module 3 is also used for:

[0156] Get entity relationships between entities;

[0157] The link relationships between entities are extracted based on the pre-built link prediction model to obtain descriptive text.

[0158] In the above implementation process, the extraction of descriptive text is realized according to the link relationship between different entities, which can expand the interactivity between entities and provide more possibilities for answer generation.

[0159] Furthermore, the structured processing module 3 is also used for:

[0160] Match each entity with data pairs to obtain the structured query statement corresponding to each entity;

[0161] Optimizing the structured query statement according to the pre-built second largest model to obtain a target structured query statement;

[0162] Match the target structured query statement in the structured database to obtain single entity text data.

[0163] In the above implementation process, the corresponding structured query statement is obtained according to the conversion data of a single entity, so that a single entity can be converted quickly and accurately, improving data utilization efficiency and reducing the limitations of single entity intent recognition.

[0164] Furthermore, the unstructured processing module 4 is also used for:

[0165] The threat intelligence query statement is expanded according to the pre-built prompt words to obtain an expanded threat intelligence query statement;

[0166] Perform vectorization processing on the expanded threat intelligence query statement to obtain vectorized text data;

[0167] Vectorize the pre-built knowledge base documents to obtain segmented text data;

[0168] Interactively processing the vectorized text data and the segmented text data to obtain third text data;

[0169] The third text data is accurately sorted to obtain the second text data.

[0170] In the above implementation process, the threat intelligence query statement is expanded and vectorized, and the knowledge base document is vectorized at the same time, which can expand the data sample range and improve the scalability and practicality of the second text data.

[0171] Furthermore, the unstructured processing module 4 is also used for:

[0172] The third text data is sequentially subjected to inverse sorting fusion processing and custom vector scoring weighting processing to obtain fourth text data;

[0173] According to a preset threshold, data meeting the requirements in the fourth text data is selected to obtain the second text data.

[0174] In the above implementation process, the third text data is sorted and fused in reverse order, and the text data with high weight is selected for further screening, which can improve the availability and accuracy of structured data and realize precise retrieval and question answering.

[0175] Furthermore, the fusion module 5 is also used for:

[0176] Sequentially concatenating the first text data and the second text data to obtain concatenated text data;

[0177] Adjust the concatenated text data to obtain question-answer text data;

[0178] The question and answer text data is input into the pre-built third model to retrieve the question and answer and obtain the question and answer results.

[0179] In the above implementation process, splicing structured data and unstructured data can improve the accuracy and weight of the question and answer results, making the question and answer results more scalable.

[0180] Furthermore, the fusion module 5 is also used for:

[0181] Get the length value of the concatenated text data;

[0182] Determine whether the length of the concatenated text data is greater than a preset length threshold;

[0183] If yes, mapping the concatenated text data according to the first template to obtain question and answer text data;

[0184] If not, the concatenated text data is mapped according to the second template to obtain question and answer text data.

[0185] In the above implementation process, the concatenated text data is mapped according to different mapping templates to achieve expansion of different data, making the question and answer text data more accurate.

[0186] The threat intelligence retrieval question-answering device described above can implement the method of the above embodiment 1. The options in the above embodiment 1 are also applicable to this embodiment and will not be described in detail here.

[0187] The rest of the contents of the embodiments of the present application can refer to the contents of the above-mentioned embodiment 1, and will not be repeated in this embodiment.

[0188] Embodiment 3

[0189] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the threat intelligence retrieval and question-answering method of embodiment one.

[0190] Optionally, the above-mentioned electronic device may be a server.

[0191] See also Figure 3 , Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include a processor 31, a communication interface 32, a memory 33 and at least one communication bus 34. The communication bus 34 is used to realize direct connection and communication between these components. The communication interface 32 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The processor 31 may be an integrated circuit chip with signal processing capabilities.

[0192] The processor 31 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor 31 can also be any conventional processor, etc.

[0193] The memory 33 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory 33 stores computer readable instructions. When the computer readable instructions are executed by the processor 31, the device can execute the above-mentioned Figure 1 The method embodiment involves various steps.

[0194] Optionally, the electronic device may further include a storage controller and an input / output unit. The memory 33, the storage controller, the processor 31, the peripheral interface, and the input / output unit are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components may be electrically connected to each other via a communication bus 34. The processor 31 is used to execute executable modules stored in the memory 33, such as software function modules or computer programs included in the device.

[0195] The input and output unit is used to provide users with the task creation and to create an optional time period or preset execution time for the task to enable interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and a keyboard.

[0196] Understandably, Figure 3 The structure shown is for illustration only. The electronic device may also include Figure 3 More or fewer components as shown, or with Figure 3 Different configurations shown. Figure 3 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0197] In addition, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the threat intelligence retrieval and question-answering method of embodiment one.

[0198] The embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the method described in the method embodiment.

[0199] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart, can be implemented with a dedicated hardware-based device that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0200] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0201] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0202] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0203] The above is only a specific implementation of the present application, but the protection range of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical range disclosed in the present application, which should be included in the protection range of the present application. Therefore, the protection range of the present application should be based on the protection range of the claims.

[0204] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A threat intelligence retrieval question-answering method, characterized in that: The method comprises: Obtain preprocessed threat intelligence query statements; Performing entity extraction on the threat intelligence query statement to obtain an entity of the threat intelligence query statement; Performing structural processing on the entity to obtain first text data; Performing unstructured processing on the threat intelligence query statement to obtain second text data; The first text data and the second text data are merged to obtain a question and answer result of the threat intelligence query statement; The step of performing structural processing on the entity to obtain the first text data includes: Obtaining descriptive text data corresponding to the entity relationship between the entities; Obtaining single entity text data corresponding to each of the entities; Concatenating the descriptive text data and the single entity text data to obtain the first text data; The step of obtaining descriptive text data corresponding to the entity relationship between the entities includes: Obtaining entity relationships between the entities; Extracting link relationships between entities from the entity relationships according to a pre-built link prediction model to obtain the descriptive text; The step of obtaining the single entity text data corresponding to each entity comprises: Performing data pair matching on each of the entities to obtain a structured query statement corresponding to each of the entities; Optimizing the structured query statement according to the pre-built second largest model to obtain a target structured query statement; Matching the target structured query statement in a structured database to obtain the single entity text data; The step of performing unstructured processing on the threat intelligence query statement to obtain second text data includes: Expanding the threat intelligence query statement according to the pre-constructed prompt words to obtain an expanded threat intelligence query statement; Performing vectorization processing on the expanded threat intelligence query statement to obtain vectorized text data; Vectorize the pre-built knowledge base documents to obtain segmented text data; Interactively processing the vectorized text data and the segmented text data to obtain third text data; Accurately sorting the third text data to obtain the second text data; The step of accurately sorting the third text data to obtain the second text data comprises: The third text data is sequentially subjected to inverse sorting fusion processing and custom vector scoring weighting processing to obtain fourth text data; The data meeting the requirements in the fourth text data are selected according to a preset threshold value to obtain the second text data.

2. The threat intelligence retrieval question-answering method according to claim 1 is characterized in that: The step of extracting entities from the threat intelligence query statement to obtain entities of the threat intelligence query statement includes: Perform entity extraction on the threat intelligence query statement according to a pre-built first large model to obtain a first entity set; Perform keyword matching on the threat intelligence query statement according to a predefined entity word list to obtain a second entity set; Taking a union of the first entity set and the second entity set to obtain the entity.

3. The threat intelligence retrieval question-answering method according to claim 1 is characterized in that: The step of fusing the first text data and the second text data to obtain a question-and-answer result of the threat intelligence query statement includes: Sequentially concatenating the first text data and the second text data to obtain concatenated text data; Adjusting the concatenated text data to obtain question-answer text data; The question and answer text data is input into a pre-built third model to retrieve the question and answer, and obtain the question and answer result.

4. The threat intelligence retrieval question-answering method according to claim 3 is characterized in that: The step of adjusting the spliced ​​text data to obtain question-answer text data includes: Obtaining the length value of the concatenated text data; Determine whether the length of the concatenated text data is greater than a preset length threshold; If yes, mapping the concatenated text data according to the first template to obtain the question and answer text data; If not, the concatenated text data is mapped according to the second template to obtain the question and answer text data.

5. A threat intelligence retrieval question-answering device, characterized in that: The device comprises: The acquisition module is used to obtain the pre-processed threat intelligence query statement; An extraction module, used to extract entities from the threat intelligence query statement to obtain entities of the threat intelligence query statement; A structured processing module, used for performing structured processing on the entity to obtain first text data; An unstructured processing module, used to perform unstructured processing on the threat intelligence query statement to obtain second text data; A fusion module, used to fuse the first text data and the second text data to obtain a question and answer result of the threat intelligence query statement; The structured processing module is also used for: Obtaining descriptive text data corresponding to the entity relationship between the entities; Obtaining single entity text data corresponding to each of the entities; Concatenating the descriptive text data and the single entity text data to obtain the first text data; Obtaining entity relationships between the entities; Extracting link relationships between entities from the entity relationships according to a pre-built link prediction model to obtain the descriptive text; Performing data pair matching on each of the entities to obtain a structured query statement corresponding to each of the entities; Optimizing the structured query statement according to the pre-built second largest model to obtain a target structured query statement; Matching the target structured query statement in a structured database to obtain the single entity text data; The unstructured processing module is further used for: Expanding the threat intelligence query statement according to the pre-constructed prompt words to obtain an expanded threat intelligence query statement; Performing vectorization processing on the expanded threat intelligence query statement to obtain vectorized text data; Vectorize the pre-built knowledge base documents to obtain segmented text data; Interactively processing the vectorized text data and the segmented text data to obtain third text data; Accurately sorting the third text data to obtain the second text data; The third text data is sequentially subjected to inverse sorting fusion processing and custom vector scoring weighting processing to obtain fourth text data; The data meeting the requirements in the fourth text data are selected according to a preset threshold value to obtain the second text data.

6. An electronic device, characterized in that: It includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the threat intelligence retrieval question-answering method according to any one of claims 1 to 4.

7. A storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the threat intelligence retrieval question-answering method as described in any one of claims 1 to 4.

Images