A network security threat information early warning method and system based on big data

Abstract

This invention relates to the field of data processing technology, specifically to a method and system for early warning of network security threats based on big data. The method includes: identifying co-working users of the current user based on key information words and key information word vectors of the current user and other users in past predetermined time periods, and the work execution times of the current user and other users in past predetermined time periods; determining the data flow model of the current user based on the traffic data packets of the current user and co-working users in each past predetermined time period; inputting the traffic data packets generated by the current user during the current time period into the data flow model to obtain the predicted fields of the current user's traffic data packets in the current time period; comparing the predicted fields with the network flow transmission protocol fields corresponding to the traffic data packets; and determining whether to issue a network security early warning based on the attribute comparison results. This invention improves the reliability of network security early warning.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method and system for early warning of cybersecurity threats based on big data. Background Technology

[0002] With the deepening of digital transformation, cyberspace has become a critical infrastructure for the operation of modern society, and the scale, complexity, and concealment of cybersecurity threats have also increased dramatically. While it is difficult to completely eliminate threats at their source, early warning and prevention are possible. Cybersecurity can be broadly divided into device system security, data content security, and operational behavior security, with operational behavior security being particularly critical. When users use networks and devices, transmitting data or downloading malware through mobile devices may actively or passively leak large amounts of data, triggering security threats. Traditional security threat early warning methods often rely on collecting, processing, and analyzing massive amounts of heterogeneous data generated from users' past actions to build data flow models, thereby identifying abnormal and harmful data flows from among numerous data streams. This is particularly problematic in local area networks (LANs) of organizations such as enterprises and schools, where users frequently interact with each other, and data transmission and reception are subject to access restrictions, resulting in each user having their own specific data interaction logic. Furthermore, LAN users need to obtain data from external networks, further complicating the data interaction process. If a user encounters a threat attack or persistent threat during data interaction, traditional methods based on modeling historical data flow behavior of a single user will overlook the security threats encountered during the user's interaction. Therefore, the reliability of network security early warnings is relatively low. Summary of the Invention

[0003] To address the technical problem of low reliability in network security early warning, the present invention aims to provide a network security threat information early warning method and system based on big data.

[0004] To solve the above technical problems, the specific technical solution adopted is as follows: In a first aspect, embodiments of the present invention provide a method for early warning of network security threats based on big data, comprising: acquiring key information words and corresponding key information word vectors from traffic data packets of each user in the past predetermined time period; determining co-working users of the current user based on the key information words and key information word vectors of the current user and other users in the past predetermined time period, and the work commencement time of the current user and other users in the past predetermined time period; determining the data flow model of the current user based on the traffic data packets of the current user and co-working users in each past predetermined time period; inputting the traffic data packets generated by the current user during work in the current time period into the data flow model to obtain the prediction field of the traffic data packets of the current user in the current time period; comparing the prediction field with the network flow transmission protocol field corresponding to the traffic data packets; and determining whether to issue network security early warning information based on the attribute comparison result.

[0005] Preferably, determining the current user's co-workers based on the key information words and key information word vectors of the current user and other users in the past predetermined time period, and the work execution time of the current user and other users in the past predetermined time period, includes: determining the information word similarity between the current user's key information words and the key information words of other users in the past predetermined time period; determining the work responsibility difference index between the current user and other users in the past predetermined time period based on the information word similarity and the number of key information words of the current user in the past predetermined time period; arranging all key information word vectors of the current user and other users in the past predetermined time period according to their frequency of occurrence to obtain the browsing information vector sets of the current user and other users respectively; arranging the browsing information vector sets of the current user and other users in all past predetermined time periods to obtain the information browsing time sequence of the current user and other users respectively; matching the information browsing time sequence of the current user and other users to obtain the matching work period of the current user and other users; and determining the current user's co-workers based on the work responsibility difference index between the current user and other users in the past predetermined time period, the work execution time of the current user in each past predetermined time period, and the work execution time of other users in the matching work period.

[0006] Preferably, determining the word similarity between the current user's key information words in the past predetermined time period and the key information words of other users in the past predetermined time period includes: determining the key information word vector corresponding to the current user's key information words in the past predetermined time period, and the cosine similarity between the key information word vectors of other users' key information words in the past predetermined time period; and obtaining the word similarity after normalizing the cosine similarity by performing maximum and minimum value normalization.

[0007] Preferably, determining the difference in responsibilities between the current user and other users during the past predetermined time period based on the information word similarity and the number of key information words of the current user during the past predetermined time period includes: selecting the key information words of other users corresponding to the maximum value in the information word similarity as the matching information words of the current user's current key information words; determining the short-term information consistency between the current user and other users during the past predetermined time period based on the information word similarity between the current user's key information words and their corresponding matching information words, and the number of key information words of the current user during the past predetermined time period; and determining the difference in responsibilities between the current user and other users during the past predetermined time period based on the information word similarity between the current user's key information words and their corresponding matching information words, the frequency of occurrence of each key information word of the current user during the past predetermined time period, the frequency of occurrence of matching information words during the past predetermined time period, and the short-term information consistency.

[0008] Preferably, the method for determining the difference in responsibilities between the current user and other users over a predetermined time period based on the word similarity between the current user's keywords and their corresponding matching keywords, the frequency of occurrence of each of the current user's keywords over a predetermined time period, the frequency of occurrence of matching keywords over a predetermined time period, and short-term information consistency includes: determining the absolute value of the first difference between the frequency of occurrence of each of the current user's keywords and the frequency of occurrence of matching keywords over a predetermined time period, and determining the first product between the first difference and the information similarity; superimposing the word similarity between the current user's keywords and their corresponding matching keywords to obtain a superimposed similarity, and determining the second product between the superimposed similarity and short-term information consistency; and determining the difference in responsibilities based on the absolute value of the first difference, the first product, and the second product.

[0009] Preferably, determining the current user's co-workers based on the difference in work responsibilities between the current user and other users in past predetermined time periods during the matched work period, the current user's work execution time in each past predetermined time period, and the work execution time of other users during the matched work period includes: determining the same work indicator for the current user and other users who were responsible for the same work in the past predetermined time periods based on the difference in work responsibilities between the current user and other users in past predetermined time periods during the matched work period, the current user's work execution time in past predetermined time periods, and the work execution time of other users during the matched work period; and identifying other users whose same work indicator is greater than or equal to a first threshold as the current user's co-workers.

[0010] Preferably, determining the same work indicator for the current user and other users in the same work period during the matching work period, based on the difference index of the work responsibilities between the current user and other users in the past predetermined time period, the work execution time of the current user in the past predetermined time period, and the work execution time of other users in the matching work period, includes: calculating the absolute value of a second difference between the work execution time of the current user in the past predetermined time period and the work execution time of other users in the matching work period; performing negative correlation normalization on the absolute value of the second difference to obtain a normalized value; and determining the same work indicator for the current user and other users in the same work period based on the normalized value and the difference index of the work responsibilities.

[0011] Preferably, determining the data flow model of the current user based on the traffic data packets of the current user and co-working users in each past predetermined time period includes: taking the payload content in the traffic data packets of the current user and co-working users in each past predetermined time period as input, taking the network flow transmission protocol field of the traffic data packets as output, taking the co-working index between the current user and co-working users as the attention weight of the traffic data packets, training the data flow model to be trained, and obtaining the data flow model.

[0012] Preferably, determining whether to issue a network security warning based on the attribute comparison results includes: comparing the predicted field with the network streaming protocol field corresponding to the traffic data packet; when the proportion of different attributes in the predicted field and the network streaming protocol field corresponding to the traffic data packet exceeds a second threshold in the attribute comparison results, it is determined that a data leak has occurred, and a network security warning is triggered.

[0013] Secondly, embodiments of the present invention provide a network security threat information early warning system based on big data, comprising: a processor and a memory; wherein the memory is used to store a computer program that can run on the processor; the processor is used to execute the program stored in the memory to implement the steps of the network security threat information early warning method based on big data mentioned in the first aspect.

[0014] This invention extracts key information words and vectors from each user's traffic data packets, identifies co-working users by combining the work deployment time, and merges the traffic data of the current user with that of co-working users to construct a data flow model. This model covers user interaction scenarios, avoids missing potential threats during the interaction process, and solves the limitations of traditional early warning perspectives. Furthermore, this invention uses data from co-working user groups as the basis for modeling, closely aligning with actual user data interaction logic. It can dynamically adapt to complex data interaction scenarios within a local area network, making it more consistent with the actual network environment than traditional single models and reducing early warning deviations caused by insufficient environmental adaptation. In addition, this invention obtains the predicted fields of the current user's traffic data packets through the model and compares them with network flow transmission protocol field attributes for double verification to determine anomalies. The model built based on multi-user associated data provides more accurate predictions, and the attribute comparison stage further filters out false positives, significantly reducing missed detections due to ignoring interaction threats, significantly improving the reliability of network security early warnings, and better protecting operational security. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a big data-based network security threat information early warning method provided in one embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a big data-based network security threat information early warning system provided in one embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a big data-based network security threat information early warning method and system proposed according to the present invention. In the following description, different "one embodiment" or "one embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details a specific scheme for a big data-based network security threat information early warning method provided by the present invention.

[0020] Example 1: Please see Figure 1 The diagram illustrates a flowchart of a big data-based network security threat information early warning method according to an embodiment of the present invention, including: Step S101: Obtain key information words and corresponding key information word vectors from the traffic data packets of each user within the past predetermined time period.

[0021] Specifically, this embodiment of the invention first acquires the traffic data packets of each user within a predetermined time period. The predetermined time period can be a period of the past month divided into 5-minute intervals. When acquiring the traffic data packets, firstly, corresponding switches are installed in various offices of the organization (company), and the data traffic information of users on the ports of these switches is copied to the monitoring port. This ensures that each user in the local area network corresponds to a port ID, and their data traffic can be monitored through the monitoring port. Furthermore, traffic monitoring technology (such as command-line tools) is used... It captures numerous traffic data packets transmitted by each user from the monitoring port and performs deep packet inspection. Extract the payload content from each data packet, and simultaneously perform deep flow inspection. This analysis derives the network streaming protocol fields (such as Internet Protocol (IP), port, timestamp, etc.) from each data packet. Specifically, it monitors all traffic data packets from all users within the local area network over the past month, including the packet payload content, network streaming protocol fields, and corresponding time information at each moment. During data acquisition, a traffic data packet is obtained by segmenting the data into a predetermined time period every five minutes, and then encoded according to the user and time of the data packet's origin.

[0022] More specifically, within an organization, to ensure the confidentiality of internal data, permissions are often assigned to each user, specifying corresponding file access permissions. However, there is no corresponding allocation for information outside the local area network (LAN). Furthermore, considering that the information needed within the same organization tends to be of a similar scope, meaning that the content and direction of data interaction among numerous users within the LAN exhibit a certain convergence, under normal circumstances, users with the same permission level will have a consistent data flow direction. Additionally, if two users have similar work content, the content of this data with a consistent flow direction will also be consistent. Therefore, when a new data flow appears, if its flow direction differs significantly from many other data flows with similar content, then this data flow may be abnormal. Based on this, the degree of abnormality of various other data flows can be analyzed. When a user browses information, the original information needs to be cached locally via the network. This process generates a certain data flow, which is then packaged into corresponding data packets. If the data packets generated by users browsing information within two different time periods correspond to similar information, it indicates that the information browsed by users during those two time periods is also similar. This allows for the assessment of the similarity of information browsed by users in two different time periods.

[0023] Therefore, the embodiments of the present invention first address the user. (Taking any user as an example) in the past time period Text information is extracted from the payload content of traffic data packets (taking any time period as an example), and then... The word segmentation tool segments the text information and removes meaningless words using a Chinese stop word list to obtain the user's... In the past period Keywords generated from browsing information (The same word appearing multiple times is recorded as the same key information word, but the number of times it appears is recorded as the frequency). Then, the bag-of-words model is used. Build users within the local area network In the past period Key information words in the corresponding traffic data packets The corresponding word vectors serve as key information word vectors. .

[0024] Step S102: Based on the key information words and key information word vectors of the current user and other users in the past predetermined time period, and the work commencement time of the current user and other users in the past predetermined time period, determine the co-working users of the current user.

[0025] Specifically, an organization determines its main business direction at the initial stage of its development. This means that the data streams generated by numerous users within the organization's local area network (LAN) when searching for information will generally lean towards this business direction. However, for the sake of organizational efficiency, i.e., the idea of ​​division of labor and collaboration, the organization divides its users into different departments, each responsible for a specific business direction. This makes each department more specialized, and the content they are responsible for will become differentiated. Consequently, when users performing the same task collect relevant reference information or generate related information, the similarity of this information will be relatively high. This results in a relatively stable similarity in the data stream content of users in the same department within the LAN who are responsible for the same task. Therefore, users performing the same task as the target user can be screened based on the similarity of their data stream content.

[0026] More specifically, according to another embodiment of the present invention, determining the co-workers of the current user based on the key information words and key information word vectors of the current user and other users in the past predetermined time period, and the work commencement time of the current user and other users in the past predetermined time period includes: determining the information word similarity between the key information words of the current user in the past predetermined time period and the key information words of other users in the past predetermined time period; determining the work responsibility difference index between the current user and other users in the past predetermined time period based on the information word similarity and the number of key information words of the current user in the past predetermined time period; arranging all key information word vectors of the current user and other users in the past predetermined time period according to their frequency of occurrence to obtain the browsing information vector sets of the current user and other users respectively; arranging the browsing information vector sets of the current user and other users in all past predetermined time periods to obtain the information browsing sequence of the current user and other users respectively; matching the information browsing sequence of the current user and other users to obtain the matching work period of the current user and other users; and determining the co-workers of the current user based on the work responsibility difference index between the current user and other users in the past predetermined time period, the work commencement time of the current user in each past predetermined time period, and the work commencement time of other users in the matching work period.

[0027] Specifically, in determining the word similarity between the key information words of the current user in a predetermined time period and the key information words of other users in a predetermined time period, according to another embodiment of the present invention, firstly, the key information word vector corresponding to the key information words of the current user in a predetermined time period is determined, and the cosine similarity between the key information word vectors of the key information words of other users in a predetermined time period is calculated; then, the cosine similarity is normalized by a minimum-maximum value to obtain the word similarity. It should be noted that in this embodiment, when calculating the cosine similarity between vectors, if the two vectors have different dimensions, zero-padding is performed on the vector with the smaller dimension to make the two vectors have the same dimension, i.e., the two vectors have the same length.

[0028] More specifically, when determining the responsibility difference index according to another embodiment of the present invention, the determination of the responsibility difference index between the current user and other users in the past predetermined time period based on the information word similarity and the number of key information words of the current user in the past predetermined time period includes: selecting the key information words of other users corresponding to the maximum value in the information word similarity as the matching information words of the current user's current key information words; determining the short-term information consistency between the current user and other users in the past predetermined time period based on the information word similarity between the current user's key information words and their corresponding matching information words, and the number of key information words of the current user in the past predetermined time period; and determining the responsibility difference index between the current user and other users in the past predetermined time period based on the information word similarity between the current user's key information words and their corresponding matching information words, the frequency of occurrence of each key information word of the current user in the past predetermined time period, the frequency of occurrence of matching information words in the past predetermined time period, and the short-term information consistency.

[0029] Specifically, in this embodiment of the invention, the current user is first calculated. In the past period The corresponding first data packet in the traffic data packet Key information words Other users In the past period The similarity between all key information words is calculated, and the key information word corresponding to the maximum similarity is selected as the current user's key information word. Key information words The matching information words. Similarly, the current user can be filtered out. In the past period All keywords in the traffic data packets on other users In the past period The corresponding matching information words in the text. Then calculate the user. In the past period All key information words in the traffic data packets and other users In the past period The similarity between corresponding matching information words is used, and their average value is taken as the similarity of information viewed by two users in the corresponding time period, denoted as user. In the past period Other users In the past period The short-term information consistency between traffic data packets generated during browsing is calculated using the following formula in this embodiment of the invention. In the past period Other users In the past period Short-term information consistency: In the above formula, Indicates user In the past period Other users In the past period The consistency of short-term information. Indicates user In the past period The first in the upflow data packet Key information words Other users In the past period Matching information words in the traffic data packets The similarity of information words between them; Indicates user In the past period The total number of key information words contained in the upstream traffic data packet.

[0030] More specifically, considering that many users in a department complete the same project (or task) under the coordination of the entire organization and department, the specific work assigned to each person will be adjusted according to the requirements of the entire project and the individual's specific work level. This will result in different focuses when users in the same department learn about work-related information. This leads to different emphases when users in the same department browse information related to the same project. However, in general, it is still the same project content. That is to say, the frequency of the corresponding keywords when users learn about relevant information is similar. Therefore, by combining the difference in the frequency of corresponding matching information words in the data packets of two users with their keyword similarity, we can obtain the similarity between the data packets generated by the two users browsing information in the corresponding time period, and obtain the work difference index of the two users in the corresponding time period. Therefore, according to another embodiment of the present invention, determining the responsibility difference index between the current user and other users in the past predetermined time period based on the information word similarity between the current user's key information words and their corresponding matching information words, the frequency of occurrence of each key information word of the current user in the past predetermined time period, the frequency of occurrence of the matching information words in the past predetermined time period, and short-term information consistency includes: determining the absolute value of the first difference between the frequency of occurrence of each key information word of the current user in the past predetermined time period and the frequency of occurrence of the matching information words in the past predetermined time period, and determining the first product between the first difference and the information similarity; superimposing the information word similarity between the current user's key information words and their corresponding matching information words to obtain a superimposed similarity, and determining the second product between the superimposed similarity and the short-term information consistency; and determining the responsibility difference index based on the absolute value of the first difference, the first product, and the second product.

[0031] Specifically, embodiments of the present invention are based on the target user. In the past period Keywords when browsing information on the internet In the past period Matching keywords used when browsing information online to obtain target users In the past period The numerous keywords correspond to users In the past period Numerous matching information words. Therefore, embodiments of the present invention use the following formula to calculate the target user. In the past period With users In the past period The responsible job variance index : In the above formula, Indicates target user In the past period With users In the past period The performance difference index in the responsible work. Indicates user In the past period The first in the upflow data packet Key information words With users In the past period Matching information words in the traffic data packets The similarity of information words between them. Indicates user In the past period The first in the upflow data packet Key information words The frequency of occurrence is the percentage of the total frequency of all information keywords within a predetermined time period. Indicates user In the past period Matching information words in the traffic data packets The frequency of occurrence is the percentage of the total frequency of all information keywords within a predetermined time period. Used to take the absolute value. Used for positive correlation normalization. Indicates user In the past period The total number of key information words contained in the upstream traffic data packet. Indicates user In the past period Other users In the past period Short-term information consistency between traffic data packets generated during browsing.

[0032] In this way, we can obtain the work difference index between the target user and other users in different scheduled time periods. The larger the work difference index, the greater the difference in the work content between the two users in the corresponding scheduled time periods.

[0033] More specifically, the previous analysis established the short-term information consistency between any two users at a given time. Based on this analysis, the information viewed by two users in the same department over a longer period should also be highly consistent. However, since work notifications are issued in batches, there is a time difference when two users view the same information. Therefore, it is necessary to first align the viewing times of the same information between the two users based on the differences in their information viewing. Thus, this embodiment of the invention first targets the user... In the past period The keyword vectors corresponding to numerous information keywords in the traffic data packets obtained from browsing information are arranged according to their frequency of occurrence to form a vector sequence, denoted as the browsing information vector set. Then target users The browsing information vector sets corresponding to all traffic data packets within the past month (the preset data collection period) are arranged chronologically to form the target user. The time sequence of information viewed over the past month is denoted as the corresponding information browsing time sequence. The browsing sequence of this information Each element in the dataset corresponds to a vector set of numerous keywords generated when a user browses information within a predetermined time period. Similarly, vector sets for other users can be constructed. Information browsing time sequence formed by browsing information over the past month Then, dynamic time warping is applied. The algorithm targets the user. Information browsing time sequence formed by browsing information over the past month With users Information browsing time sequence formed by browsing information over the past month Matching is performed between elements. During matching, the information browsing sequence is considered. Information browsing sequence The difference in responsible work between any two elements (browsing information vector sets) within a predetermined time period is used as a distance metric between the two elements (browsing information vector sets).

[0034] At this point, the target users have been acquired. With users The work content for each scheduled time period over the past month was matched. Similarly, the matching results of the target user's work content with other users for each scheduled time period over the past month can be obtained, i.e., the user's... Target users past period Matching work hours This can effectively avoid differences in the timing of information collection between two users due to differences in work efficiency and allocation.

[0035] More specifically, within a department, since users are always responsible for the same projects, the information they collect and browse in past periods is quite similar. Furthermore, since projects are uniformly arranged by the company, the time spent collecting the same information is roughly similar. Therefore, combined with the consistency of two users' work content in past periods, it can be concluded that two users are likely responsible for the same work within the same department. Thus, according to another embodiment of the present invention, based on the difference index of the current user's and other users' responsible work in past predetermined time periods during the matched work period, the current user's work execution time in each past predetermined time period, and the work execution time of other users in the matched work period, determining the current user's co-workers includes: determining the same work index of the current user and other users responsible for the same work in past predetermined time periods based on the difference index of the current user's and other users' responsible work in past predetermined time periods during the matched work period, the current user's work execution time in past predetermined time periods, and the work execution time of other users in the matched work period; and identifying other users whose same work index is greater than or equal to a first threshold as the current user's co-workers.

[0036] Specifically, the first threshold can be set according to the actual situation; in this embodiment of the invention, it is set to 0.7. When determining the same work indicator, according to another embodiment of the invention, the absolute value of the second difference between the current user's work commencement time in a predetermined time period and the work commencement time of other users in the matching work period is calculated; the absolute value of the second difference is negatively correlated and normalized to obtain a normalized value; based on the normalized value and the work difference indicator, the same work indicator between the current user and other users who were responsible for the same work in a predetermined time period is determined. In this embodiment of the invention, the following formula is used to calculate the target user... With users Same work targets for the same task during each scheduled time period in the past month. : In the above formula, Indicates target user With users The same work targets for the same task during each scheduled time period in the past month. Indicates target user In the past period In and with users Corresponding matching work period The differences in the responsibilities of the two job duties are indicated by the following indicators. Indicates target user In the past period The specific timeframe for carrying out the work (i.e., within a month). Indicates user During the matching work period The time required for the work to commence. Used to take the absolute value; Used for negative correlation normalization; This indicates the total number of scheduled work periods for the target user within the past month. This represents the normalization function.

[0037] A similar operation can be performed to calculate the same work metrics for the target user and other users who performed the same tasks over the past month. Therefore, a threshold can be set (preset to 0.7). When the same work metrics for the target user are greater than or equal to 0.7, the two users can be approximated as having similar job content, and thus recorded as users with the same job content. In this way, numerous users with the same job content can be identified as the target user.

[0038] Step S103: Determine the data flow model of the current user based on the traffic data packets of the current user and other users in the same work group during each predetermined time period in the past.

[0039] Specifically, to ensure that internal data is not leaked, an organization often divides users into access levels based on their job titles. This results in users with the same job title having similar access levels within the local area network. Consequently, the data flow generated by users at the same access level when handling related tasks has a certain similarity in its flow structure. At the same time, since their job titles are similar, the work content they are responsible for is also similar. Therefore, the possible data flow structure of users at work can be constructed based on the data flow of many users with the same work content.

[0040] More specifically, according to another embodiment of the present invention, determining the data flow model of the current user based on the traffic data packets of the current user and co-working users in each past predetermined time period includes: taking the payload content in the traffic data packets of the current user and co-working users in each past predetermined time period as input, taking the network flow transport protocol field of the traffic data packets as output, taking the co-working index between the current user and co-working users as the attention weight of the traffic data packets, training the data flow model to be trained, and obtaining the data flow model.

[0041] Specifically, this embodiment of the invention uses the payload content corresponding to the traffic data packets of users with the same work content as the target user (including the target user) in various predetermined time periods in the past as input, and the network stream transmission protocol fields of the corresponding traffic data packets (such as source IP, port, timestamp (24-hour format within a day)) as output to train a convolutional neural network structure that predicts the transmission direction based on the data payload content. The co-work index between each user with the same work content and the target user is introduced as the attention weight for each group of traffic data packet input-output pairs. Thus, the data flow structure for the target user is trained, allowing analysis of its data flow direction based on the data flow content generated by the target user's work within a predetermined time period.

[0042] Step S104: Input the traffic data packets generated by the current user during the current time period into the data flow model to obtain the prediction field of the traffic data packets of the current user during the current time period. Compare the prediction field with the network flow transmission protocol field corresponding to the traffic data packets. Determine whether to issue network security warning information based on the attribute comparison result.

[0043] Specifically, based on the similarity of data stream content generated by numerous users within the local area network during their work, and combined with the direction of their data streams, the normal data stream content and direction structure of the target user during their work are constructed. Then, the traffic data packet content (i.e., the payload content of the data packets) generated by the target user during the current time period can be input into the trained data stream model to derive the possible direction of the target user's traffic data packets during the current time period, i.e., the predicted network flow transmission protocol field (denoted as the prediction field).

[0044] More specifically, according to another embodiment of the present invention, determining whether to issue a network security warning based on the attribute comparison result includes: comparing the predicted field with the network streaming protocol field corresponding to the traffic data packet; when the proportion of different attributes in the predicted field and the network streaming protocol field corresponding to the traffic data packet exceeds a second threshold in the attribute comparison result, it is determined that a data leak has occurred, and a network security warning is triggered.

[0045] Specifically, in this embodiment of the invention, by comparing the predicted field obtained from the prediction with the network flow transmission protocol field actually corresponding to the traffic data packet, when the proportion of different attribute values ​​between the two exceeds a second threshold (preset to 40%), that is, the predicted field contains multiple attributes (attributes refer to source IP, port, timestamp, etc.), and when the proportion of the number of different attributes in the predicted field and the network flow transmission protocol field actually corresponding to the traffic data packet is greater than the second threshold, it is considered that the data flow generated by the target user's work in the current period is abnormal, which does not conform to the current data flow pattern in the department, and may leak relevant confidential data in the company or be infected with external malicious data. It is necessary to issue a network data security warning in a timely manner and cut off its external network access permission to prevent greater threats from arising.

[0046] This invention extracts key information words and vectors from each user's traffic data packets, identifies co-working users by combining the work deployment time, and merges the traffic data of the current user with that of co-working users to construct a data flow model. This model covers user interaction scenarios, avoids missing potential threats during the interaction process, and solves the limitations of traditional early warning perspectives. Furthermore, this invention uses data from co-working user groups as the basis for modeling, closely aligning with actual user data interaction logic. It can dynamically adapt to complex data interaction scenarios within a local area network, making it more consistent with the actual network environment than traditional single models and reducing early warning deviations caused by insufficient environmental adaptation. In addition, this invention obtains the predicted fields of the current user's traffic data packets through the model and compares them with network flow transmission protocol field attributes for double verification to determine anomalies. The model built based on multi-user associated data provides more accurate predictions, and the attribute comparison stage further filters out false positives, significantly reducing missed detections due to ignoring interaction threats, significantly improving the reliability of network security early warnings, and better protecting operational security.

[0047] Furthermore, this invention first compares the target user's data stream with the data streams of other users to obtain a reference data stream. Then, based on the user and its reference, it constructs a data stream model of the user in cyberspace. Finally, it compares the target user's actual data stream with its model data stream to complete a threat warning for their data security. This effectively avoids the problem of traditional algorithms relying solely on individual user data behavior to detect dangerous data, ignoring malicious operations or passively generated data threats. It also improves the overall network security warning efficiency within a local area network.

[0048] Example 2: Corresponding to the big data-based cybersecurity threat information early warning method provided in the above embodiments, based on the same technical concept, this invention also provides a big data-based cybersecurity threat information early warning system, which is used to execute the above-described big data-based cybersecurity threat information early warning method. Figure 2 To illustrate another structural diagram of a big data-based network security threat information early warning system according to various embodiments of the present invention, as shown below. Figure 2 As shown at the hardware level, the big data-based cybersecurity threat information early warning system includes a processor, and optionally, an internal bus, a network interface, and memory. The memory may include RAM, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, this big data-based cybersecurity threat information early warning system may also include other hardware required for its operation.

[0049] The processor, network interface, and memory can be interconnected via an internal bus, which can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, this diagram uses only a single bidirectional arrow, but it does not imply that there is only one bus or one type of bus.

[0050] Memory is used to store programs. Specifically, programs can include program code, which includes computer operation commands. Memory can include main memory and non-volatile memory, and it provides instructions and data to the processor.

[0051] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a device at the logical level that is assigned to a specific user. The processor executes the program stored in memory and specifically performs the following: Figure 1 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods in the preceding method embodiments, and will not be repeated here.

[0052] It should be noted that the big data-based network security threat information early warning system and the big data-based network security threat information early warning method provided in this embodiment of the invention are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned big data-based network security threat information early warning method, and has the same or similar beneficial effects. Repeated parts will not be described again.

[0053] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0054] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for early warning of cybersecurity threats based on big data, characterized in that, The big data-based cybersecurity threat information early warning method includes: Obtain key information words and corresponding key information word vectors from the traffic data packets of each user within a predetermined time period in the past; Based on the key information words and key information word vectors of the current user and other users in the past predetermined time period, and the work commencement time of the current user and other users in the past predetermined time period, determine the co-working users of the current user; The data flow model of the current user is determined based on the traffic data packets of the current user and the users working in the same field during each predetermined time period in the past. The traffic data packets generated by the current user during the current time period are input into the data flow model to obtain the prediction field of the traffic data packets of the current user during the current time period. The prediction field is compared with the network flow transmission protocol field corresponding to the traffic data packets. Based on the attribute comparison result, it is determined whether to issue network security warning information.

2. The method for early warning of cybersecurity threats based on big data according to claim 1, characterized in that, The step of determining the co-working users of the current user based on key information words and key information word vectors of the current user and other users in the past predetermined time period, and the work commencement time of the current user and other users in the past predetermined time period, includes: Determine the similarity of keywords between the current user and other users during the same past time period; Based on the information word similarity and the number of key information words of the current user in the past predetermined time period, determine the difference index of the responsible work between the current user and other users in the past predetermined time period; Arrange all key information word vectors of the current user and other users in the past predetermined time period according to their frequency of occurrence to obtain the browsing information vector sets of the current user and other users respectively; Arrange the browsing information vector sets of the current user and the other users in all past predetermined time periods to obtain the information browsing time sequence of the current user and the other users respectively; The information browsing time sequence of the current user and other users is matched to obtain the matching working time period of the current user and other users; Based on the difference index of the responsible work between the current user and other users in the past predetermined time period during the matched work period, the work start time of the current user in each of the past predetermined time periods, and the work start time of other users in the matched work period, the co-workers of the current user are determined.

3. The method for early warning of network security threats based on big data according to claim 2, characterized in that, The determination of the similarity between the key information words of the current user in the past predetermined time period and the key information words of other users in the past predetermined time period includes: Determine the key information word vectors corresponding to the key information words of the current user in the past predetermined time period, and the cosine similarity between the key information word vectors of the key information words of other users in the past predetermined time period; The cosine similarity is normalized by minima to obtain the information word similarity.

4. The method for early warning of cybersecurity threats based on big data according to claim 2, characterized in that, The method of determining the difference in responsibilities between the current user and other users during the past predetermined time period based on the similarity of the information words and the number of key information words of the current user during the past predetermined time period includes: Select the key information words of other users corresponding to the maximum value of the information word similarity as the matching information words of the current key information words of the current user; Based on the similarity between the current user's key information words and their corresponding matching information words, as well as the number of key information words of the current user in the past predetermined time period, the short-term information consistency between the current user and other users in the past predetermined time period is determined. Based on the similarity between the current user's key information words and their corresponding matching information words, the frequency of occurrence of each key information word of the current user in the past predetermined time period, the frequency of occurrence of the matching information words in the past predetermined time period, and the short-term information consistency, the difference index of the responsible work between the current user and other users in the past predetermined time period is determined.

5. The method for early warning of network security threats based on big data according to claim 4, characterized in that, The method of determining the difference indicators of the responsibilities of the current user and other users in the past predetermined time period based on the information word similarity between the current user's key information words and their corresponding matching information words, the frequency of occurrence of each key information word of the current user in the past predetermined time period, the frequency of occurrence of the matching information words in the past predetermined time period, and the short-term information consistency includes: Determine the absolute value of a first difference between the frequency of occurrence of each key information word of the current user in the past predetermined time period and the frequency of occurrence of the matching information word in the past predetermined time period, and determine a first product between the first difference and the information similarity. The similarity between the key information words of the current user and their corresponding matching information words is superimposed to obtain the superimposed similarity, and a second product between the superimposed similarity and the short-term information consistency is determined. The responsible work difference index is determined based on the absolute value of the first difference, the first product, and the second product.

6. The method for early warning of network security threats based on big data according to claim 2, characterized in that, The method of determining the co-working users of the current user based on the difference in the responsible work between the current user and other users in the past predetermined time periods, the work commencement time of the current user in each of the past predetermined time periods, and the work commencement time of other users in the matched work period includes: Based on the difference index of the work responsibilities of the current user and other users in the past predetermined time period during the matched work period, the work start time of the current user in the past predetermined time period, and the work start time of other users in the matched work period, the same work index of the current user and other users who are responsible for the same work in the past predetermined time period is determined. Other users whose work performance indicators are greater than or equal to the first threshold are identified as the current user's work users.

7. The method for early warning of network security threats based on big data according to claim 6, characterized in that, The determination of the same work indicators between the current user and other users in the past predetermined time period based on the difference in the work responsibilities of the current user and other users in the matched work period, the work commencement time of the current user in the past predetermined time period, and the work commencement time of other users in the matched work period, includes: Calculate the absolute value of a second difference between the current user's work start time in the past predetermined time period and the other users' work start times in the matched work period; The absolute value of the second difference is negatively correlated and normalized to obtain a normalized value. Based on the normalized value and the job performance difference index, determine the same job performance index of the current user and other users who were responsible for the same job in the past predetermined time period.

8. The method for early warning of network security threats based on big data according to claim 1, characterized in that, The step of determining the data flow model of the current user based on the traffic data packets of the current user and the users working in the same industry during each predetermined past time period includes: The data flow model is trained by taking the payload content of the traffic data packets of the current user and the co-working user in each past predetermined time period as input, taking the network stream transport protocol field of the traffic data packets as output, and taking the co-working index between the current user and the co-working user as the attention weight of the traffic data packets.

9. The method for early warning of network security threats based on big data according to claim 1, characterized in that, The step of determining whether to issue a network security warning based on the attribute comparison results includes: Compare the predicted field with the network stream transport protocol field corresponding to the traffic data packet; When the proportion of different attributes in the predicted field and the network flow transmission protocol field corresponding to the traffic data packet exceeds the second threshold in the attribute comparison results, a data leak is determined and a network security warning is triggered.

10. A network security threat information early warning system based on big data, characterized in that, include: Processor and memory; wherein the memory is used to store computer programs that can run on the processor; A processor is used to execute a program stored in memory to implement the steps of the big data-based network security threat information early warning method as described in any one of claims 1-9.

Images