A network behavior analysis method based on big data

Abstract

The present invention discloses a network behavior analysis method based on big data, which relates to the field of big data processing technology. The present invention utilizes a server to obtain user behavior data, identifies users, and generates useful trajectory segments based on user access; performs frequent sequence mining to obtain user frequent behavior sequences, uses frequent behavior sequences as user features, distinguishes users' abnormal behaviors from normal behaviors, abstracts the obtained abnormal behavior data into labels, constructs a user portrait model, establishes an abnormal behavior pattern feature library, matches the user behavior sequence to be tested with the abnormal behavior pattern feature library, determines whether the user is an abnormal user based on whether the match is successful, and extracts abnormal features of the user behavior data accordingly. The present invention completes the judgment and extraction of abnormal features of user behavior data, improves the accuracy of abnormal detection of user behavior data, and rates the detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of big data processing, and in particular relates to a network behavior analysis method based on big data. Background Art

[0002] With the advent of the information age, the amount of data has exploded. Extracting valuable information from these massive datasets has become crucial. Big data processing involves collecting, storing, processing, and analyzing large datasets. Big data analytics leverages statistical and machine learning algorithms to extract patterns, trends, and associations from this data, helping to identify user behavior patterns.

[0003] The rapid development of machine learning and artificial intelligence technologies has provided strong support for network behavior analysis. Machine learning algorithms can learn from large amounts of data and discover patterns and correlations, thereby predicting users' next actions. The application of artificial intelligence technology has made network behavior analysis more intelligent and automated, enabling a deeper understanding of user behavior and needs.

[0004] As the internet enters the era of big data, user behavior is bringing about a series of changes and reshaping of companies' products and services. The most significant change is that all user behavior is now traceable and analyzable. Companies maintain vast amounts of raw data and various business data, providing a true record of their operations. How to more effectively utilize this data for analysis and evaluation has become a challenge for companies facing this ever-increasing volume of data. Summary of the Invention

[0005] In order to overcome the shortcomings and deficiencies of the above-mentioned prior art, the purpose of the present invention is to provide a network behavior analysis method based on big data.

[0006] The present invention adopts the following technical solutions:

[0007] A network behavior analysis method based on big data, the process is as follows:

[0008] Step 1: Obtain user behavior data, identify users, and generate useful trajectory segments based on user access; user data includes user identity, access date, access path, access content, request results, and browsing time;

[0009] Step 2: After dividing the useful trajectory segments into different regions, perform frequent sequence mining, output the user's frequent behavior sequence and its support, use the frequent behavior sequence as the user feature, and assign weights;

[0010] Step 3: Distinguish the user's abnormal behavior from normal behavior and abstract the obtained abnormal behavior data into labels;

[0011] Step 4: Build a user portrait model;

[0012] Step 5: Establish an abnormal behavior pattern feature library, match the user behavior sequence to be tested with the abnormal behavior pattern feature library, and determine whether it is an abnormal user;

[0013] Step 6: Extract abnormal behavior features of abnormal users, quantify the accuracy of extracted abnormal behaviors of users, and complete the analysis of abnormal behavior features of abnormal users.

[0014] Preferably, step one is as follows:

[0015] Obtain user behavior data through log files generated by the web server;

[0016] The process of user identification based on user behavior data is as follows:

[0017] When the IP addresses are different, they represent multiple different users;

[0018] When the IP address is the same and the user uses the same operating system and browser, it represents the same user;

[0019] When the IP address is the same but the user uses different operating systems and browsers, it represents multiple different users;

[0020] When a user requests to access a page that is not related to any page they have visited before, it means that a new user has appeared;

[0021] When a user requests a page that has never been visited before and there is no reference trace in the log record, it means that a new user has appeared;

[0022] Decompose the user's visit into trace segments, which can be expressed as follows:

[0023] R= <U,{(P i ,T i ),…,(P k ,T k )}>;

[0024] Where R represents the set sequence of pages requested by the user within a period of time; U represents the user ID; P represents the identifier of the page requested by the user; T represents time;

[0025] Set the timeout threshold between adjacent user requests to θ, and determine whether T i -T i-1 ≤Q…1≤i<<Q. If it is satisfied, the trajectory is marked as a useful trajectory segment.

[0026] Preferably, step 2 is as follows:

[0027] Sort the useful trajectory segments in chronological order to obtain the user behavior data sequence Y and build a user behavior sequence database. Each user behavior node has a corresponding position in the sequence. The user behavior sequence represents a user's operation within a certain timestamp.

[0028] User behavior data sequence Y = (y1, y2, y3, y4, ..., ym), where m is a positive integer. For each behavior xm in the sequence, create a corresponding point and add the corresponding behavior identifier, timestamp, and page ID. Use the timestamp to sort the created points to obtain the user behavior data sequence point set X = x1, x2, x3, x4, ..., xm), where m is a positive integer.

[0029] Take any point xm in the space as the center of the circle and set the radius to r to form a circular area. The set of all points in this circular area is marked as the neighborhood B. r (xm),

[0030] B r (xm)={xn∈D│dist(xm,xn)≤r};

[0031] Among them, dist(xm,xn) represents the distance between xm and xn;

[0032] The minimum number of samples in the neighborhood is marked as MinPts;

[0033] Randomly select a point xm from the data set X and judge |B r (xm)| is greater than or equal to MinPts, when|B r (xm)|≥MinPts, then xm is determined to be a core object and added to the core set Y;

[0034] Randomly select a core object xn from the core set Y, and add all points where its density is reachable to the new set C1 to form the first cluster;

[0035] Continue to access the next point in the data set X and repeat the above steps until all points in the data set are processed to obtain the cluster Cm. The points not included in the cluster are marked as noise and deleted.

[0036] The cluster Cm includes user basic attributes, position attributes, business attributes, time attributes and spatial attributes. Among them, basic attributes include user ID and IP address; position attributes include user's position level; business attributes are user's business functions; time attributes are the time segmentation of user access; spatial attributes are the regional segmentation of user access.

[0037] The Apriori algorithm is used to mine frequent behavior sequences on the data after region segmentation, and the user behavior sequences and their support are output. The support threshold is set to ρ, and the behavior sequences below the support threshold ρ are eliminated. The behavior sequences above the support threshold ρ and their support are retained and marked as frequent behavior sequences.

[0038] Frequent behavior sequences are used as user features. Each user feature represents a specific user behavior pattern, including multiple user behavior steps; a weight is assigned to each user feature.

[0039] As a preference, step three is specifically as follows:

[0040] Establish a normal behavior model to capture the user's normal behavior patterns;

[0041] Set detection thresholds for abnormal behavior, including upper and lower limits for operation frequency and abnormal ranges for access duration, and classify behaviors that deviate from the detection thresholds as abnormal behavior;

[0042] Label abnormal behaviors.

[0043] As a preference, step five is specifically as follows:

[0044] Utilize the established user portrait to extract the user's abnormal behavior pattern features and build an abnormal behavior pattern feature library, where the behavior pattern features include behavior frequency, behavior sequence, and behavior time interval;

[0045] The user behavior sequence to be tested is matched with the abnormal behavior pattern feature library. Whether the match is successful is used to determine whether the user is an abnormal user, and based on this, the abnormal features of the user behavior data are extracted. The specific process is as follows:

[0046] Use TF-IDF to convert the user behavior sequence to be tested into a feature vector A = [a1, a2, a3, ..., ai], and convert the abnormal feature sequence in the abnormal behavior pattern feature library into a feature vector B = [b1, b2, b3, ..., bj];

[0047] The distance between vector A and vector B is calculated by dynamic time warping (DTW). Vector B is used as the reference sequence B and vector A as the comparison sequence A. The distance is calculated using the formula:

[0048] D(i,j)=d(i,j)+min[D(i-1,j),D(i,j-1),D(i-1,j-1)];

[0049] d(i,j)=1-cos(ai,bj);

[0050] Among them, d(i,j) is the cosine distance between ai and bj; D(i,j) is the sum of the distances between the first i coordinates of the comparison sequence A and the first j coordinates of the reference sequence B; when the D(i,j) value is less than the preset threshold, it means that the smaller the distance between vector A and vector B, the more similar the user behavior sequence to be tested is to the abnormal feature sequence in the abnormal behavior pattern feature library; when the D(i,j) value is greater than the set user behavior abnormal threshold, it means that the distance between vector A and vector B is greater, and the user behavior sequence to be tested is judged to be an abnormal sequence, and the user to be tested is marked as an abnormal user, completing the identification of the abnormal user.

[0051] As a preference, step six is ​​specifically as follows:

[0052] The specific quantification formula is as follows:

[0053]

[0054] Where a represents the amount of abnormal features detected; A represents the total amount of abnormal features in the data;

[0055] When the value of F is greater than or equal to the maximum value of the set accuracy threshold range δ1, the accuracy of extracting abnormal features of user behavior data is determined to be level one;

[0056] When the value of F is less than the maximum value δ1 of the accuracy threshold range and greater than or equal to the minimum value δ2 of the accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level 2;

[0057] When the value of F is less than the minimum value δ2 of the set accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level three.

[0058] As a preference, density reachability is defined as follows: if xn is in the neighborhood of xm and xm is a core object, then xn is density-reachable by xm, and if there are a1, a2, ..., a n , where a1=xm,a n =xn, and a i+1 By a i If the density is directly reachable, then xn can be reached through the density of xm.

[0059] Preferably, support is defined as follows: the frequency of a certain user behavior sequence appearing in the data set.

[0060] Preferably, in labeling abnormal behaviors, the number of labels is greater than or equal to 1.

[0061] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0062] 1. The present invention obtains user behavior data through the server, which can record users' operations on the platform in real time and comprehensively, thereby generating more accurate user trajectory segments. Furthermore, through frequent sequence mining, it can discover users' frequent behavior sequences. These sequences, as user features, can better represent users' actual behavior patterns. When distinguishing between abnormal and normal user behaviors, the accuracy will be greatly improved, which helps to more accurately identify abnormal users.

[0063] 2. Based on the user portrait model and abnormal behavior pattern feature library, this invention can provide a deeper understanding of users' behavioral preferences and needs. For abnormal behavior, the platform can respond promptly, such as reminding users to pay attention to account security and taking necessary risk prevention and control measures, thereby protecting users' rights and interests.

[0064] 3. By establishing a library of abnormal behavior pattern signatures, the present invention enables the platform to rapidly match the user's behavior sequence under test with the abnormal behavior patterns in the library. Once a match is successful, the user is identified as abnormal, and appropriate risk prevention and control measures are implemented. This automated matching and judgment process greatly improves the efficiency and accuracy of risk prevention and control, helping to reduce potential safety hazards and economic losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 The flowchart of the network behavior analysis method based on big data of the present invention is shown. DETAILED DESCRIPTION

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0068] Example 1

[0069] See Figure 1 As shown, the network behavior analysis method based on big data in this embodiment has the following process:

[0070] Step 1: Obtain user behavior data through the log files generated by the web server, identify users, and generate useful trajectory segments based on user access; these log files record the activities of the web server, including user requests and server responses; user data includes but is not limited to user identity, access date, access path, access content, request results, and browsing time.

[0071] The process of user identification based on user behavior data is as follows:

[0072] When the IP addresses are different, they represent multiple different users;

[0073] When the IP address is the same and the user uses the same operating system and browser, it represents the same user;

[0074] When the IP address is the same but the user uses different operating systems and browsers, it represents multiple different users;

[0075] When a user requests to access a page that is not related to any page they have visited before, it means that a new user has appeared;

[0076] When a user requests a page that has never been visited before and there is no reference trace in the log record, it means that a new user has appeared.

[0077] Decompose the user's visit into trace segments, which can be expressed as follows:

[0078] R= <U,{(P i ,T i ),…,(P k ,T k )}>;

[0079] Among them, R represents the set sequence of pages requested by the user within a period of time; U represents the user identifier; P represents the identifier of the page requested by the user; and T represents time.

[0080] Set the timeout threshold between adjacent user requests to θ, and determine whether T i -T i-1 ≤Q…1≤i<<Q. If it is satisfied, the trajectory is marked as a useful trajectory segment.

[0081] Step 2: After dividing the useful trajectory segments into different regions, perform frequent sequence mining, output the user's frequent behavior sequence and its support, use the frequent behavior sequence as the user feature, and assign weights.

[0082] The useful trajectory segments are sorted in chronological order to obtain the user behavior data sequence Y, and a user behavior sequence database is constructed. Each user behavior node has its corresponding position in the sequence, and the user behavior sequence represents the operation of a user within a certain timestamp.

[0083] User behavior data sequence Y = (y1, y2, y3, y4, ..., ym), where m is a positive integer. For each behavior xm in the sequence, create a corresponding point and add the corresponding behavior identifier, timestamp, and page ID to preserve the context of each point. Use the timestamp to sort the created points to obtain the user behavior data sequence point set X = (x1, x2, x3, x4, ..., xm), where m is a positive integer.

[0084] Take any point xm in the space as the center of the circle and set the radius to r to form a circular area. The set of all points in this circular area is marked as the neighborhood B. r (xm),

[0085] B r (xm)={xn∈D│dist(xm,xn)≤r};

[0086] Among them, dist(xm,xn) represents the distance between xm and xn.

[0087] The minimum number of samples in the neighborhood is marked as MinPts;

[0088] Randomly select a point xm from the data set X and judge |B r (xm)| is greater than or equal to MinPts, when|B r (xm)|≥MinPts, then xm is determined to be a core object and added to the core set Y;

[0089] A core object xn is randomly selected from the core set Y, and all points that are density-reachable to it are added to the new set C1 to form the first cluster. In this embodiment, density-reachable is defined as follows: if xn is in the neighborhood of xm, and xm is a core object, then xn is directly density-reachable by xm, and if there are a1, a2, ..., a n , where a1=xm,a n =xn, and a i+1 By a i Density is directly accessible, then xn is reachable by xm density;

[0090] Continue to access the next point in the data set X and repeat the above steps until all points in the data set are processed to obtain the cluster Cm. The points not included in the cluster are marked as noise and deleted.

[0091] The cluster Cm includes user basic attributes, position attributes, business attributes, time attributes and spatial attributes; among them, basic attributes include user ID and IP address; position attributes include the user's position level, such as administrator, ordinary user, etc.; business attributes are the business functions of the user, such as operation user, audit user, etc.; time attributes are the time segmentation of user access; spatial attributes are the regional segmentation of user access.

[0092] The Apriori algorithm is used to mine frequent behavior sequences from the segmented data, outputting user behavior sequences and their support. A support threshold is set to ρ, and behavior sequences below the support threshold are removed. Behavior sequences above the support threshold are retained, along with their support, and marked as frequent behavior sequences. The support represents the frequency with which a user behavior sequence appears in the dataset.

[0093] Frequent behavior sequences are used as user features. Each user feature represents a specific user behavior pattern, including multiple user behavior steps. A weight is assigned to each user feature to reflect its importance in user behavior.

[0094] Step 3: Distinguish the user's abnormal behavior from normal behavior, and abstract the obtained abnormal behavior data into labels.

[0095] Establish a normal behavior model to capture the user's normal behavior patterns;

[0096] Set detection thresholds for abnormal behavior, including but not limited to upper and lower limits for operation frequency and abnormal ranges for access duration, and classify behaviors that deviate from the detection thresholds as abnormal behavior;

[0097] Label abnormal behaviors.

[0098] Step 4: Build a user portrait model.

[0099] Step 5: Establish an abnormal behavior pattern feature library, match the user behavior sequence to be tested with the abnormal behavior pattern feature library, and determine whether it is an abnormal user.

[0100] The user profile established in the above steps is used to extract the user's abnormal behavior pattern characteristics and build an abnormal behavior pattern feature library, where the behavior pattern characteristics include but are not limited to behavior frequency, behavior sequence and behavior time interval.

[0101] The user behavior sequence to be tested is matched with the abnormal behavior pattern feature library. Whether the match is successful is used to determine whether the user is an abnormal user, and based on this, the abnormal features of the user behavior data are extracted. The specific process is as follows:

[0102] Use TF-IDF to convert the user behavior sequence to be tested into a feature vector A = [a1, a2, a3, ..., ai], and convert the abnormal feature sequence in the abnormal behavior pattern feature library into a feature vector B = [b1, b2, b3, ..., bj];

[0103] The distance between vector A and vector B is calculated by dynamic time warping (DTW). Vector B is used as the reference sequence B and vector A as the comparison sequence A. The distance is calculated using the formula:

[0104] D(i,j)=d(i,j)+min[D(i-1,j),D(i,j-1),D(i-1,j-1)];

[0105] d(i,j)=1-cos(ai,bj);

[0106] Wherein, d(i,j) is the cosine distance between ai and bj; D(i,j0) is the sum of the distances between the first i coordinates of the comparison sequence A and the first j coordinates of the reference sequence B; when the D(i,j) value is less than the preset threshold, it means that the smaller the distance between vector A and vector B, the more similar the user behavior sequence to be tested is to the abnormal feature sequence in the abnormal behavior pattern feature library; when the D(i,j0 value is greater than the set user behavior abnormal threshold, it means that the distance between vector A and vector B is larger, and the user behavior sequence to be tested is judged to be an abnormal sequence, and the user to be tested is marked as an abnormal user, completing the identification of the abnormal user.

[0107] Step 6: Extract abnormal behavior features of abnormal users, quantify the accuracy of extracted abnormal behaviors, and complete the analysis of abnormal behavior features of abnormal users. The specific quantification formula is as follows:

[0108]

[0109] Among them, a represents the amount of abnormal features detected; A represents the total amount of abnormal features in the data.

[0110] When the value of F is greater than or equal to the maximum value of the set accuracy threshold range δ1, the accuracy of extracting abnormal features of user behavior data is determined to be level one;

[0111] When the value of F is less than the maximum value δ1 of the accuracy threshold range and greater than or equal to the minimum value δ2 of the accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level 2;

[0112] When the value of F is less than the minimum value δ2 of the set accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level three.

[0113] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0114] Those skilled in the art will appreciate that the algorithmic steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only for some logical functions. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0117] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. The above is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A network behavior analysis method based on big data, characterized in that: The process is as follows: Step 1: Obtain user behavior data, identify users, and generate useful trajectory segments based on user access; user data includes user identity, access date, access path, access content, request results, and browsing time; Step 2: After dividing the useful trajectory segments into different regions, perform frequent sequence mining, output the user's frequent behavior sequence and its support, use the frequent behavior sequence as the user feature, and assign weights; Step 3: Distinguish the user's abnormal behavior from normal behavior and abstract the obtained abnormal behavior data into labels; Step 4: Build a user portrait model; Step 5: Establish an abnormal behavior pattern feature library, match the user behavior sequence to be tested with the abnormal behavior pattern feature library, and determine whether it is an abnormal user; Step 6: Extract abnormal behavior features of abnormal users, quantify the accuracy of extracting abnormal behaviors of users, and complete the analysis of abnormal behavior features of abnormal users; The step 1 is specifically as follows: Obtain user behavior data through log files generated by the web server; The process of user identification based on user behavior data is as follows: When the IP addresses are different, they represent multiple different users; When the IP address is the same and the user uses the same operating system and browser, it represents the same user; When the IP address is the same but the user uses different operating systems and browsers, it represents multiple different users; When a user requests to access a page that is not related to any page they have visited before, it means that a new user has appeared; When a user requests a page that has never been visited before and there is no reference trace in the log record, it means that a new user has appeared; Decompose the user's visit into trace segments, which can be expressed as follows: R=<U,{(P i ,T i ),…,(P k ,T k )}>; Where R represents the set sequence of pages requested by the user within a period of time; U represents the user ID; P represents the identifier of the page requested by the user; T represents time; Set the timeout threshold between adjacent requests of the user to θ, and determine whether T is satisfied i -T i-1 ≤Q…1≤i << Q, if satisfied, mark this trajectory as a useful trajectory segment; The step 2 is specifically as follows: Sort the useful trajectory segments in chronological order to obtain the user behavior data sequence Y and build a user behavior sequence database. Each user behavior node has a corresponding position in the sequence. The user behavior sequence represents a user's operation within a certain timestamp. User behavior data sequence Y = (y1, y2, y3, y4, ..., ym), where m is a positive integer. For each behavior xm in the sequence, create a corresponding point and add the corresponding behavior identifier, timestamp, and page ID. Use the timestamp to sort the created points to obtain the user behavior data sequence point set X = (x1, x2, x3, x4, ..., xm), where m is a positive integer. Take any point xm in the space as the center of the circle and set the radius to r to form a circular area. The set of all points in this circular area is marked as the neighborhood B. r (xm), B r (xm)={xn∈D│dist(xm,xn)≤r}; Among them, dist(xm,xn) represents the distance between xm and xn; The minimum number of samples in the neighborhood is marked as MinPts; Randomly select a point xm from the data set X and judge |B r (xm)| is greater than or equal to MinPts, when|B r (xm)|≥MinPts, then xm is determined to be a core object and added to the core set Y; Randomly select a core object xn from the core set Y, and add all points where its density is reachable to the new set C1 to form the first cluster; Continue to access the next point in the data set X and repeat the above steps until all points in the data set are processed to obtain the cluster Cm. The points not included in the cluster are marked as noise and deleted. The cluster Cm includes user basic attributes, position attributes, business attributes, time attributes and spatial attributes. Among them, basic attributes include user ID and IP address; position attributes include user's position level; business attributes are user's business functions; time attributes are the time segmentation of user access; spatial attributes are the regional segmentation of user access. The Apriori algorithm is used to mine frequent behavior sequences on the data after region segmentation, and the user behavior sequences and their support are output. The support threshold is set to ρ, and the behavior sequences below the support threshold ρ are eliminated. The behavior sequences above the support threshold ρ and their support are retained and marked as frequent behavior sequences. Frequent behavior sequences are used as user features. Each user feature represents a specific user behavior pattern, including multiple user behavior steps; a weight is assigned to each user feature.

2. A network behavior analysis method based on big data according to claim 1, characterized in that: The step three is as follows: Establish a normal behavior model to capture the user's normal behavior patterns; Set detection thresholds for abnormal behavior, including upper and lower limits for operation frequency and abnormal ranges for access duration, and classify behaviors that deviate from the detection thresholds as abnormal behavior; Label abnormal behaviors.

3. The network behavior analysis method based on big data according to claim 1, characterized in that: The step five is specifically as follows: Utilize the established user portrait to extract the user's abnormal behavior pattern features and build an abnormal behavior pattern feature library, where the behavior pattern features include behavior frequency, behavior sequence, and behavior time interval; The user behavior sequence to be tested is matched with the abnormal behavior pattern feature library. Whether the match is successful is used to determine whether the user is an abnormal user, and based on this, the abnormal features of the user behavior data are extracted. The specific process is as follows: Use TF-IDF to convert the user behavior sequence to be tested into a feature vector A = [a1, a2, a3, ..., ai], and convert the abnormal feature sequence in the abnormal behavior pattern feature library into a feature vector B = [b1, b2, b3, ..., bj]; The distance between vector A and vector B is calculated by dynamic time warping (DTW). Vector B is used as the reference sequence B and vector A as the comparison sequence A. The distance is calculated using the formula: D(i,j)=d(i,j)+min[D(i-1,j),D(i,j-1),D(i-1,j-1)]; d(i,j)=1-cos(ai,bj); Among them, d(i,j) is the cosine distance between ai and bj; D(i,j) is the sum of the distances between the first i coordinates of the comparison sequence A and the first j coordinates of the reference sequence B; when the D(i,j) value is less than the preset threshold, it means that the smaller the distance between vector A and vector B, the more similar the user behavior sequence to be tested is to the abnormal feature sequence in the abnormal behavior pattern feature library; when the D(i,j) value is greater than the set user behavior abnormal threshold, it means that the distance between vector A and vector B is larger, the user behavior sequence to be tested is judged to be an abnormal sequence, the user to be tested is marked as an abnormal user, and the identification of the abnormal user is completed.

4. The network behavior analysis method based on big data according to claim 1, characterized in that: The step six is ​​specifically as follows: The specific quantification formula is as follows: Where a represents the amount of abnormal features detected; A represents the total amount of abnormal features in the data; When the value of F is greater than or equal to the maximum value of the set accuracy threshold range δ1, the accuracy of extracting abnormal features of user behavior data is determined to be level one; When the value of F is less than the maximum value δ1 of the accuracy threshold range and greater than or equal to the minimum value δ2 of the accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level 2; When the value of F is less than the minimum value δ2 of the set accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level three.

5. The network behavior analysis method based on big data according to claim 1, characterized in that: The density reachability is defined as follows: if xn is in the neighborhood of xm and xm is a core object, then xn is directly reachable by the density of xm. If there are a1, a2, ..., a n , where a1=xm,a n =xn, and a i+1 By a i If the density is directly reachable, then xn can be reached through the density of xm.

6. The network behavior analysis method based on big data according to claim 1, characterized in that: The support is defined as follows: the frequency of a certain user behavior sequence appearing in the data set.

7. The network behavior analysis method based on big data according to claim 1 is characterized in that: In the labeling process for abnormal behavior, the number of labels is greater than or equal to 1.

Images