A method and system for identifying and processing foreign network attack threat intelligence

Abstract

The application provides a kind of identification and processing method and system for overseas network attack threat intelligence, specifically applied in text processing field, comprising: collecting new page, obtaining multiple structured text data using multiple analysis engines;After merging, obtain new intelligence records;Using intelligence quality evaluation algorithm to score new intelligence records, get the score results;Add new intelligence records and corresponding score results to intelligence database;Further generate daily briefing and update search engine index.

Description

Technical Field

[0001] This application relates to the field of text processing, and more specifically, to a method and system for identifying and processing threat intelligence related to overseas cyberattacks. Background Technology

[0002] With the rapid development of information technology and the transformation of people's production and lifestyle concepts, their dependence on the internet has also increased. However, this has also brought about hidden dangers in network security. Common network attack techniques include DoS attacks, system vulnerabilities, password cracking, and unauthorized intrusions, which seriously affect network environment security and user security.

[0003] Therefore, how to efficiently and accurately identify cyberattack threat intelligence is an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for identifying and processing intelligence on cyberattack threats from overseas. It introduces multiple analysis engines and intelligence quality evaluation algorithms to construct an intelligence database, and uses the database content to generate daily briefings and update search engine indexes. This enables timely and effective identification of cyberattack threats from overseas, thereby improving network security. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, a method for identifying and processing threat intelligence related to overseas cyberattacks is provided, comprising:

[0006] Step S1: Collect multiple newly added pages from any website during a preset time period each day;

[0007] Step S2: Initialize the text queue;

[0008] Step S3: Extract the text data from the newly added page sequentially, and cache the text data sequentially into the text queue;

[0009] Step S4: Extract text data from the text queue sequentially, and obtain standardized text data through preprocessing;

[0010] Step S5: Use multiple analysis engines to identify the standardized text data to obtain multiple types of structured text data;

[0011] Step S6: Merge the various structured text data to obtain new intelligence records; add the new intelligence records to the intelligence database in sequence;

[0012] Step S7: Read the newly added intelligence records in the intelligence database in sequence, and use the intelligence quality evaluation algorithm to score the newly added intelligence records to obtain the scoring results;

[0013] Step S8: Based on the correspondence between the newly added intelligence records and the scoring results, the scoring results are added to the intelligence database in sequence.

[0014] Optionally, step S1: collecting multiple newly added pages from any website during a preset time period each day, specifically includes:

[0015] The system reads the directory address of any website from the configuration file during a preset time period each day.

[0016] Based on the directory address, crawl the corresponding page addresses of the website;

[0017] Match the multiple page addresses with the intelligence database;

[0018] If a match is found, the page corresponding to that address is removed; if no match is found, the page corresponding to that address is a new page.

[0019] Optionally, the process of obtaining standardized text data through preprocessing specifically includes:

[0020] The traditional Chinese characters in the text data in the text queue are converted to simplified Chinese characters, and special symbols are filtered out to obtain the preliminary text content;

[0021] The preliminary text content is segmented into sentences and words to obtain word segmentation results;

[0022] The word segmentation results are then processed by synonym replacement to obtain standardized text content.

[0023] Optionally, step S5: using multiple analysis engines to identify the standardized text data to obtain multiple types of structured text data, specifically including:

[0024] The first analysis engine, the second analysis engine, and the third analysis engine were constructed using regular expressions, heuristic algorithms, and machine learning algorithms, respectively.

[0025] The standardized text data is identified using the first analysis engine, the second analysis engine, and the third analysis engine, respectively, to obtain first structured text data, second structured text data, and third structured text data.

[0026] Optionally, step S6: merging the various structured data to obtain new intelligence records specifically includes:

[0027] The first structured text data, the second structured text data, and the third structured text data are merged to obtain the new intelligence record.

[0028] Optionally, the third analytics engine may be constructed using machine learning algorithms, including:

[0029] The training set includes a professional terminology database, a regional terminology database, a regional idiom database, and a regional popular word database.

[0030] Machine learning algorithms are used to train and optimize the third analysis engine.

[0031] Optionally, the intelligence quality evaluation algorithm includes:

[0032] The training set includes a professional terminology database, a regional terminology database, a regional idiom database, and a regional popular word database.

[0033] The parameters of the intelligence quality evaluation algorithm are optimized.

[0034] Optionally, the method further includes:

[0035] Read the newly added intelligence records and corresponding scoring results from the intelligence database for the current day;

[0036] According to the preset template, the newly added intelligence records and the scoring results are converted into daily briefings;

[0037] The daily briefing will be published via webpage attachment or email.

[0038] Optionally, the method further includes:

[0039] The search engine reads newly added intelligence records and corresponding rating results from the intelligence database for the current day;

[0040] Update the search engine index.

[0041] In another aspect of the present invention, a system for identifying and processing intelligence on cyberattack threats from overseas is provided, comprising:

[0042] The page collection module is used to collect multiple newly added pages from any website during a preset time period each day.

[0043] The queue initialization module is used to initialize the text queue;

[0044] The caching module is used to sequentially extract text data from the newly added page and cache the text data sequentially into the text queue;

[0045] The data preprocessing module is used to extract text data from the text queue sequentially and obtain standardized text data through preprocessing.

[0046] The data structuring module is used to identify the standardized text data using multiple analysis engines to obtain various structured text data.

[0047] The database operation module is used to merge the various structured text data to obtain new intelligence records; and to add the new intelligence records to the intelligence database in sequence.

[0048] The scoring module is used to sequentially read the newly added intelligence records in the intelligence database, score the newly added intelligence records using an intelligence quality evaluation algorithm, and obtain the scoring results.

[0049] The database operation module is further configured to add the scoring results to the intelligence database sequentially according to the correspondence between the newly added intelligence records and the scoring results.

[0050] Optionally, the page acquisition module is specifically used for:

[0051] The system reads the directory address of any website from the configuration file during a preset time period each day.

[0052] Based on the directory address, crawl the corresponding page addresses of the website;

[0053] Match the multiple page addresses with the intelligence database;

[0054] If a match is found, the page corresponding to that address is removed; if no match is found, the page corresponding to that address is a new page.

[0055] Optionally, the process of obtaining standardized text data through preprocessing specifically includes:

[0056] The traditional Chinese characters in the text data in the text queue are converted to simplified Chinese characters, and special symbols are filtered out to obtain the preliminary text content;

[0057] The preliminary text content is segmented into sentences and words to obtain word segmentation results;

[0058] The word segmentation results are then processed by synonym replacement to obtain standardized text content.

[0059] Optionally, the data structuring module is specifically used for:

[0060] The first analysis engine, the second analysis engine, and the third analysis engine were constructed using regular expressions, heuristic algorithms, and machine learning algorithms, respectively.

[0061] The standardized text data is identified using the first analysis engine, the second analysis engine, and the third analysis engine, respectively, to obtain first structured text data, second structured text data, and third structured text data.

[0062] Optionally, the database operation module is specifically used for:

[0063] The first structured text data, the second structured text data, and the third structured text data are merged to obtain the new intelligence record.

[0064] Optionally, the third analytics engine may be constructed using machine learning algorithms, including:

[0065] The training set includes a professional terminology database, a regional terminology database, a regional idiom database, and a regional popular word database.

[0066] Machine learning algorithms are used to train and optimize the third analysis engine.

[0067] Optionally, the intelligence quality evaluation algorithm includes:

[0068] The training set includes a professional terminology database, a regional terminology database, a regional idiom database, and a regional popular word database.

[0069] The parameters of the intelligence quality evaluation algorithm are optimized.

[0070] Optionally, the system also includes a briefing generation module, used to read the newly added intelligence records and corresponding scoring results in the intelligence database for the day; convert the newly added intelligence records and the scoring results into a daily briefing according to a preset template; and publish the daily briefing in the form of a web page attachment or email.

[0071] Optionally, the system also includes a search engine update module, used by the search engine to read newly added intelligence records and corresponding rating results from the intelligence database on the same day; and to update the search engine index.

[0072] Beneficial effects:

[0073] (1) In this invention, newly added pages are collected daily, and new intelligence records are constructed through text queues, standardized and structured text processing methods; an intelligence quality evaluation algorithm with sentiment analysis capabilities is proposed to score the new intelligence records and obtain the scoring results; the new intelligence records and the corresponding scoring results are added to the intelligence database. This improves the timeliness and accuracy of threat intelligence identification.

[0074] (2) By introducing multiple algorithms to construct an analysis engine, the accuracy of the intelligence records obtained by merging the results is higher. At the same time, a professional terminology database, a regional terminology database, a regional idiom database, and a regional popular word database are used as training sets, and machine learning algorithms are used to train and optimize the third analysis engine. The resulting analysis engine has the ability to adapt to local language expression habits and can improve the performance of data extraction and transformation.

[0075] (3) Daily briefings are generated based on the daily updated intelligence database, and the search engine index is updated accordingly. This enables timely and effective identification of overseas cyberattack threats, thereby improving network security. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a flowchart illustrating a method for identifying and processing threat intelligence from overseas cyberattacks, as provided in an embodiment of this application.

[0078] Figure 2 This is a flowchart illustrating the method for constructing new intelligence records provided in an embodiment of this application;

[0079] Figure 3 This is a schematic diagram of the structure of a system for identifying and processing threat intelligence from overseas cyberattacks, provided in an embodiment of this application. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0081] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0082] This application provides a method and system for identifying and processing overseas cyberattack threat intelligence, including: collecting newly added pages and obtaining various structured text data using multiple analysis engines; merging the data to obtain new intelligence records; scoring the new intelligence records using an intelligence quality evaluation algorithm to obtain scoring results; adding the new intelligence records and corresponding scoring results to an intelligence database; and then generating a daily briefing and updating the search engine index. This enables timely and effective identification of overseas cyberattack threat intelligence, improving network security.

[0083] This method and system for identifying and processing cyberattack threat intelligence from overseas can be integrated into electronic devices, such as terminals or servers. The terminal can be a laptop or a personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.

[0084] It is understood that the method and system for identifying and processing threat intelligence from overseas networks, as described in this embodiment, can be executed on a terminal, on a server, or jointly by both. The above examples should not be construed as limiting this application.

[0085] Example 1

[0086] Figure 1 The diagram illustrates a flowchart of the method for identifying and processing overseas cyberattack threat intelligence provided in an embodiment of this application. Please refer to it. Figure 1 Specifically, it includes the following steps:

[0087] Step S110: Collect multiple newly added pages from any website during a preset time period each day.

[0088] Specifically, the crawler program is started at a preset time each day; the directory address of any website is read from the configuration file; multiple page addresses corresponding to the website are crawled based on the directory address; the multiple page addresses are matched with the intelligence database; if a match is found, the page corresponding to that page address is removed; if no match is found, the page corresponding to that page address is a new page.

[0089] The intelligence database is used to store historical addresses, intelligence text data, and the scores corresponding to the intelligence text data.

[0090] Step S120: Initialize the text queue.

[0091] Step S130: Cache the text data sequentially into the text queue.

[0092] Specifically, the text data in the newly added page is extracted sequentially, and the extracted text data is cached in the text queue. This step is repeated until all text data has been extracted and cached, then step S140 is executed.

[0093] Step S140: Obtain standardized text data through preprocessing.

[0094] Specifically, text data in the text queue is extracted sequentially, and steps S140-S160 are performed on the extracted text data until all text data in the queue has been processed, then step S170 is performed.

[0095] The preprocessing includes: converting traditional Chinese characters in the text data of the text queue into simplified Chinese characters and filtering out special symbols to obtain preliminary text content; segmenting the preliminary text content into sentences and words to obtain word segmentation results; and performing synonym replacement processing on the word segmentation results to obtain standardized text content.

[0096] Step S150: Use multiple analysis engines to identify the standardized text data to obtain multiple types of structured text data.

[0097] Step S160: Merge the various structured text data to obtain new intelligence records; add the new intelligence records to the intelligence database in sequence.

[0098] Step S170: Use an intelligence quality evaluation algorithm to score the newly added intelligence records and obtain the score results.

[0099] Specifically, the newly added intelligence records in the intelligence database are read sequentially, and steps S170-S180 are executed until the scoring results corresponding to all newly added intelligence records are added to the intelligence database.

[0100] In one implementation, step S170 may specifically include the following steps:

[0101] Step S171: Match any word in the newly added intelligence record with the regional idiom database or regional popular word database. If a match is found, convert the word into the corresponding regional terminology or professional terminology. Based on the conversion result of any word, convert the newly added intelligence record into a candidate intelligence record.

[0102] The regional idiom database includes regional idioms and their corresponding regional terms or professional terms, while the regional popular word database includes regional popular words and their corresponding regional terms or professional terms.

[0103] Step S172: Match any word in the candidate intelligence record with a professional terminology database or a regional terminology database, and determine the first score of the candidate intelligence record based on the matching result.

[0104] Optionally, the first score of the candidate intelligence record is calculated according to the following formula:

[0105]

[0106] Where n represents the total number of words in the newly added intelligence record, w i,j It is the weight of the vocabulary j corresponding to the i-th word in the newly added intelligence record, c i This represents the number of times the i-th word appears in the historical intelligence record.

[0107] Step S173: Input the newly added intelligence record and the candidate intelligence record into the sentiment analysis model respectively to obtain the first sentiment analysis result and the second sentiment analysis result.

[0108] Step S174: Determine the second score based on the first sentiment analysis result and the second sentiment analysis result.

[0109] Alternatively, the second score can be calculated using the following formula:

[0110] s2 = sen1 × sen2

[0111] in,

[0112]

[0113] Step S175: Obtain the scoring result based on the first score and the second score.

[0114] Alternatively, the scoring result can be obtained according to the following formula:

[0115] S=α×s1+β×s2

[0116] Where s1 represents the first score, s2 represents the second score, and α and β represent the weight parameters.

[0117] The resulting intelligence quality assessment algorithm possesses sentiment analysis capabilities, improving the accuracy of threat intelligence scoring.

[0118] Step S180: Based on the correspondence between the newly added intelligence records and the scoring results, the scoring results are added to the intelligence database in sequence.

[0119] Furthermore, the method also includes: reading newly added intelligence records and corresponding scoring results from the intelligence database for the day; converting the newly added intelligence records and scoring results into a daily briefing according to a preset template; and publishing the daily briefing via webpage attachment or email.

[0120] Furthermore, the method also includes: the search engine reading newly added intelligence records and corresponding rating results from the intelligence database on the same day; and updating the search engine index.

[0121] This implementation method enables timely and effective identification of overseas cyberattack threats, thereby enhancing network security.

[0122] Example 2

[0123] Figure 2 The flowchart illustrating the method for constructing new intelligence records provided in this application is shown below. Please refer to it. Figure 2 Specifically, it includes the following steps:

[0124] Step S210: Construct a first analysis engine using regular expressions; construct a second analysis engine using heuristic algorithms; construct a third analysis engine using machine learning algorithms.

[0125] Among them, the analysis engine is used to convert the text data of threat intelligence from natural language descriptions into structured data.

[0126] In one implementation, a professional term library, a regional term library, a regional idiom library, and a regional buzzword library can be used as a training set; the third analysis engine is trained and optimized using machine learning algorithms.

[0127] Among them, professional terms refer to common professional terms in scenarios such as computer networks and network attack and defense; regional terms refer to the situation where different terms are used for the same concept in different regions due to English translation problems and historical reasons. For example, the English word "server" has two Chinese names, "server" and "servo"; regional idioms refer to unique idioms formed locally due to dialects and expression habits. For example, in Taipei, China, text messages are called "brief messages"; regional buzzwords refer to popular words triggered by hot issues in the local area's Internet.

[0128] The analysis engine obtained thereby has the ability to adapt to local language expression habits and can improve the performance of data extraction and conversion.

[0129] Step S220: Identify the standardized text data to obtain the first structured text data, the second structured text data, and the third structured text data.

[0130] Step S230: Merge the first structured text data, the second structured text data, and the third structured text data to obtain a new intelligence record.

[0131] Furthermore, the new intelligence record can be added to the intelligence database.

[0132] By introducing multiple algorithms to construct an analysis engine in this implementation manner and merging the results, the accuracy of the obtained intelligence record is higher, greatly reducing the recognition error caused by language differences.

[0133] Embodiment III

[0134] To implement the above method embodiments, this embodiment also provides a recognition and processing system for threat intelligence of overseas cyberattacks, as Figure 3 shown, the system includes:

[0135] A page collection module 310, configured to collect multiple new pages of any website at a preset time period every day.

[0136] The queue initialization module 320 is used to initialize the text queue.

[0137] The caching module 330 is used to sequentially extract text data from the newly added page and cache the text data sequentially into the text queue.

[0138] The data preprocessing module 340 is used to extract text data from the text queue sequentially and obtain standardized text data through preprocessing.

[0139] The data structuring module 350 is used to identify the standardized text data using multiple analysis engines to obtain multiple types of structured text data.

[0140] The database operation module 360 ​​is used to merge the various structured text data to obtain new intelligence records; and to add the new intelligence records to the intelligence database in sequence.

[0141] The scoring module 370 is used to sequentially read the newly added intelligence records in the intelligence database, score the newly added intelligence records using an intelligence quality evaluation algorithm, and obtain a scoring result.

[0142] The database operation module 360 ​​is further configured to add the scoring results to the intelligence database sequentially according to the correspondence between the newly added intelligence records and the scoring results.

[0143] Optionally, the page collection module 310 is specifically used for: reading the directory address of any website from the configuration file during a preset time period each day; crawling multiple page addresses corresponding to the website based on the directory address; matching the multiple page addresses with the intelligence database; if a match is found, removing the page corresponding to that page address; if no match is found, the page corresponding to that page address is a newly added page.

[0144] Optionally, the step of obtaining standardized text data through preprocessing specifically includes: converting traditional Chinese characters in the text data in the text queue into simplified Chinese characters and filtering out special symbols to obtain preliminary text content; segmenting the preliminary text content into sentences and words to obtain word segmentation results; and performing synonym replacement processing on the word segmentation results to obtain standardized text content.

[0145] Optionally, the data structuring module 350 is specifically used to: construct a first analysis engine, a second analysis engine, and a third analysis engine using regular expressions, heuristic algorithms, and machine learning algorithms, respectively; and use the first analysis engine, the second analysis engine, and the third analysis engine to identify the standardized text data, thereby obtaining first structured text data, second structured text data, and third structured text data.

[0146] Optionally, the database operation module 360 ​​is specifically used to: merge the first structured text data, the second structured text data, and the third structured text data to obtain a new intelligence record.

[0147] Optionally, constructing a third analysis engine using machine learning algorithms includes: using a professional terminology database, a regional terminology database, a regional idiom database, and a regional popular word database as training sets; and using machine learning algorithms to train and optimize the third analysis engine.

[0148] Optionally, the intelligence quality evaluation algorithm includes: using a professional terminology database, a regional terminology database, a regional idiom database, and a regional popular word database as a training set; and optimizing the parameters of the intelligence quality evaluation algorithm.

[0149] Optionally, the system also includes a briefing generation module 380, which reads the newly added intelligence records and corresponding scoring results in the intelligence database for the day; converts the newly added intelligence records and the scoring results into a daily briefing according to a preset template; and publishes the daily briefing in the form of a web page attachment or email.

[0150] Optionally, the system also includes a search engine update module 390, which is used for the search engine to read newly added intelligence records and corresponding rating results in the intelligence database on the same day; and update the search engine index.

[0151] Therefore, the system can identify overseas cyberattack threats in a timely and effective manner, thereby improving network security.

[0152] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the modules / units / subunits / components in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0153] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0156] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0158] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for identification and processing of threat intelligence for foreign network attack, characterized in that, The method comprises the following steps: Step S1: collecting a plurality of new pages of any website in a preset period of time every day; Step S2: initializing a text queue; Step S3: extracting text data in the new pages in sequence and caching the text data into the text queue in sequence; Step S4: extracting text data in the text queue in sequence and obtaining standardized text data through preprocessing; Step S5: identifying the standardized text data by using a plurality of analysis engines respectively to obtain a plurality of structured text data; Step S6: merging the plurality of structured text data to obtain new intelligence records, and adding the new intelligence records into an intelligence database in sequence; Step S7: reading the new intelligence records in the intelligence database in sequence, scoring the new intelligence records by using an intelligence quality evaluation algorithm to obtain a scoring result; The method comprises the following steps: matching any word in the new intelligence record with a regional idiom library or a regional popular word library, if matched, converting the word into a corresponding regional term or professional term; converting the new intelligence record into a candidate intelligence record according to the conversion result of any word; matching any word in the candidate intelligence record with a professional term library or a regional term library, and determining a first score of the candidate intelligence record according to the matching result; inputting the new intelligence record and the candidate intelligence record into an emotion analysis model respectively to obtain a first emotion analysis result and a second emotion analysis result; determining a second score according to the first emotion analysis result and the second emotion analysis result; obtaining a scoring result according to the first score and the second score; Step S8: adding the scoring result into the intelligence database in sequence according to the corresponding relationship between the new intelligence record and the scoring result. 2.The method for identification and processing of threat intelligence against foreign network attack according to claim 1, characterized in that, The step S1: collecting a plurality of new pages of any website in a preset period of time every day, specifically comprises the following steps: reading a directory address of any website from a configuration file in a preset period of time every day; based on the directory address, crawling a plurality of page addresses corresponding to the website; matching the plurality of page addresses with an intelligence database; if matched, the page corresponding to the page address is excluded; if not matched, the page corresponding to the page address is a new page. 3.The method for identification and processing of threat intelligence against foreign network attack according to claim 1, characterized in that, The standardized text data is obtained through preprocessing, specifically comprising the following steps: converting traditional Chinese characters in the text data in the text queue into simplified Chinese characters and filtering special symbols to obtain preliminary text content; performing sentence segmentation and word segmentation on the preliminary text content to obtain a word segmentation result; performing synonym replacement processing on the word segmentation result to obtain standardized text content. 4.The method of identifying and processing threat intelligence for foreign network attacks according to claim 1, wherein, The step S5: identifying the standardized text data by using a plurality of analysis engines respectively to obtain a plurality of structured text data, specifically comprising the following steps: constructing a first analysis engine, a second analysis engine and a third analysis engine by using regular expressions, heuristic algorithms and machine learning algorithms respectively; identifying the standardized text data by using the first analysis engine, the second analysis engine and the third analysis engine respectively to obtain first structured text data, second structured text data and third structured text data. 5.The method for identification and processing of threat intelligence against foreign network attack according to claim 4, characterized in that, The step S6 of merging the plurality of structured data to obtain the new intelligence record, specifically comprising: Merging the first structured text data, the second structured text data and the third structured text data to obtain the new intelligence record. 6.The method for identification and processing of threat intelligence against foreign network attack according to claim 4, characterized in that, The third analysis engine is constructed by using a machine learning algorithm, comprising: Taking the professional term library, the regional term library, the regional idiom library and the regional popular word library as a training set; Training and optimizing the third analysis engine by using a machine learning algorithm. 7.The method for identification and processing of threat intelligence against foreign network attacks according to claim 1, characterized in that, The intelligence quality evaluation algorithm comprises: Taking the professional term library, the regional term library, the regional idiom library and the regional popular word library as a training set; Optimizing the parameters of the intelligence quality evaluation algorithm. 8.The method for identification and processing of threat intelligence against foreign network attack according to claim 1, characterized in that, The method further comprises: Reading the new intelligence record and the corresponding scoring result in the intelligence database on the same day; According to the preset template, the new intelligence record and the scoring result are converted into a daily briefing; The daily briefing is published in the form of a web page attachment or an email. 9.The method for identification and processing of threat intelligence against foreign network attacks according to claim 8, characterized in that, The method further comprises: The search engine reads the new intelligence record and the corresponding scoring result in the intelligence database on the same day; Updating the search engine.

10. An identification and processing system for foreign network attack threat intelligence, characterized by, Comprise: A page collection module for collecting a plurality of new pages of any website in a daily preset time period; A queue initialization module for initializing a text queue; A cache module for sequentially extracting text data in the new pages and sequentially caching the text data into the text queue; A data preprocessing module for sequentially extracting text data in the text queue and obtaining standardized text data through preprocessing; A data structuring module for identifying the standardized text data by using a plurality of analysis engines to obtain a plurality of structured text data; A database operation module for merging the plurality of structured text data to obtain a new intelligence record; and sequentially adding the new intelligence record to an intelligence database; A scoring module for sequentially reading the new intelligence record in the intelligence database, scoring the new intelligence record by using an intelligence quality evaluation algorithm to obtain a scoring result; comprising: matching any word in the new intelligence record with a regional idiom library or a regional popular word library, if matched, converting the word into a corresponding regional term or professional term; converting the new intelligence record into a candidate intelligence record according to the conversion result of any word; matching any word in the candidate intelligence record with a professional term library or a regional term library, and determining a first score of the candidate intelligence record according to the matching result; inputting the new intelligence record and the candidate intelligence record into an emotion analysis model to obtain a first emotion analysis result and a second emotion analysis result; determining a second score according to the first emotion analysis result and the second emotion analysis result; and obtaining a scoring result according to the first score and the second score; The database operation module is further used to sequentially add the scoring result to the intelligence database according to the corresponding relationship between the new intelligence record and the scoring result.

Images