Network behavior analysis method based on big data

Abstract

The invention discloses a network behavior analysis method based on big data, and relates to the technical field of big data processing. The method comprises the following steps: acquiring user behavior data by using a server, identifying a user, and generating a useful track segment according to user access; performing frequent sequence mining to obtain a user frequent behavior sequence, taking the frequent behavior sequence as a user feature, distinguishing an abnormal behavior and a normal behavior of a user, abstracting obtained abnormal behavior data into a label, constructing a user portrait model, and establishing an abnormal behavior mode feature library; and matching the to-be-detected user behavior sequence with the abnormal behavior pattern feature library, judging whether the to-be-detected user is an abnormal user according to whether the matching is successful, and extracting user behavior data abnormal features according to the judgment result. According to the method, judgment and extraction of abnormal characteristics of the user behavior data are completed, the accuracy of user behavior data anomaly detection is improved, and the detection accuracy is graded.

Description

Technical Field

[0001] The present invention belongs to the technical field of big data processing, and in particular relates to a network behavior analysis method based on big data. Background Art

[0002] With the advent of the information age, the amount of data has exploded, and how to extract valuable information from huge data sets has become the key. Big data processing involves collecting, storing, processing and analyzing large-scale data sets, while big data analysis can use statistical and machine learning algorithms to extract patterns, trends and associations from data to help identify user behavior patterns.

[0003] The rapid development of machine learning and artificial intelligence technologies has provided strong support for network behavior analysis. Machine learning algorithms can learn and discover patterns and correlations from large amounts of data to predict the user's next behavior. The application of artificial intelligence technology makes network behavior analysis more intelligent and automated, and can more deeply understand user behavior and needs.

[0004] After the Internet entered the era of big data, user behavior has brought a series of changes and reshapes to the products and services of enterprises. The biggest change is that all user behaviors can be "traceable" and "analyzed" in front of enterprises. Enterprises save a large amount of original data and various business data, which are the real records of corporate business activities. How to use this data more effectively for analysis and evaluation has become a problem for enterprises based on the background of larger data volume. Summary of the invention

[0005] In order to overcome the shortcomings and deficiencies of the above-mentioned prior art, the purpose of the present invention is to provide a network behavior analysis method based on big data.

[0006] The present invention adopts the following technical solution:

[0007] A network behavior analysis method based on big data, the process is as follows:

[0008] Step 1: Obtain user behavior data, identify users, and generate useful track segments based on user access; user data includes user identity, access date, access path, access content, request results, and browsing time;

[0009] Step 2: After dividing the useful trajectory segments into different regions, perform frequent sequence mining, output the user's frequent behavior sequence and its support, use the frequent behavior sequence as the user feature, and assign weights;

[0010] Step 3: Distinguish the user's abnormal behavior from normal behavior, and abstract the obtained abnormal behavior data into labels;

[0011] Step 4: Build a user portrait model;

[0012] Step 5: Establish an abnormal behavior pattern feature library, match the behavior sequence of the user to be tested with the abnormal behavior pattern feature library, and determine whether it is an abnormal user;

[0013] Step 6: Extract abnormal behavior features of abnormal users, quantify the accuracy of extracting abnormal behaviors of users, and complete the analysis of abnormal behavior features of abnormal users.

[0014] Preferably, step one is as follows:

[0015] Obtain user behavior data through log files generated by the web server;

[0016] The process of user identification based on user behavior data is as follows:

[0017] When the IP addresses are different, they represent multiple different users;

[0018] When the IP address is the same and the user uses the same operating system and browser, it represents the same user;

[0019] When the IP address is the same but the user uses different operating systems and browsers, it represents multiple different users;

[0020] When a user requests to access a page that is not related to any page previously visited, it means that a new user has appeared;

[0021] When a user requests a page that has never been visited before and there is no reference trace in the log record, it means that a new user has appeared;

[0022] Decompose the user's visit into trace segments, expressed in the following form:

[0023] R= <U,{(P i ,T i ),…,(P k ,T k )}>;

[0024] Where R represents the set sequence of pages requested by the user within a period of time; U represents the user ID; P represents the identifier of the page requested by the user; T represents time;

[0025] Set the timeout threshold between adjacent user requests to θ, and determine whether T i -T i-1 ≤Q…1≤i<<Q. If it is satisfied, the trajectory is marked as a useful trajectory segment.

[0026] Preferably, step 2 is as follows:

[0027] Sort the useful trajectory segments in chronological order to obtain the user behavior data sequence Y, and build a user behavior sequence database. Each user behavior node has its corresponding position in the sequence, and the user behavior sequence represents the operation of a user within a certain timestamp;

[0028] User behavior data sequence Y = (y1, y2, y3, y4, ..., ym), where m is a positive integer. For each behavior xm in the sequence, create a corresponding point, add the corresponding behavior identifier, timestamp and page ID, use the timestamp to sort the created points, and obtain the user behavior data sequence point set X = x1, x2, x3, x4, ..., xm), where m is a positive integer;

[0029] Take any point xm in the space as the center of the circle and set the radius to r to form a circular area. The set of all points in this circular area is marked as neighborhood B. r (xm),

[0030] B r (xm)={xn∈D│dist(xm,xn)≤r};

[0031] Among them, dist(xm,xn) represents the distance between xm and xn;

[0032] The minimum number of samples in the neighborhood is marked as MinPts;

[0033] Randomly select a point xm from the data set X and judge |B r (xm)| is greater than or equal to MinPts, when|B r (xm)|≥MinPts, then xm is determined to be a core object and added to the core set Y;

[0034] Randomly select a core object xn from the core set Y, and add all its density-reachable points to the new set C1 to form the first cluster;

[0035] Continue to access the next point in the data set X and repeat the above steps until all points in the data set are processed to obtain the cluster Cm. The points not included in the cluster are marked as noise and deleted.

[0036] The cluster Cm includes user basic attributes, position attributes, business attributes, time attributes and space attributes; among which, the basic attributes include user ID and IP address; the position attributes include the position level of the user; the business attributes are the business functions of the user; the time attribute is the time segmentation of the user's visit; the space attribute is the regional segmentation of the user's visit;

[0037] The Apriori algorithm is used to mine frequent behavior sequences on the data after region segmentation, and the user behavior sequence and its support are output; the support threshold is set to ρ, the behavior sequences below the support threshold ρ are eliminated, the behavior sequences above the support threshold ρ and their support are retained, and they are marked as frequent behavior sequences;

[0038] Frequent behavior sequences are used as user features. Each user feature represents a specific user behavior pattern, including multiple user behavior steps; a weight is assigned to each user feature.

[0039] As a preference, step three is specifically as follows:

[0040] Establish a normal behavior model to capture the user's normal behavior pattern;

[0041] Set detection thresholds for abnormal behaviors, including upper and lower limits of operation frequency and abnormal range of access duration, and classify behaviors that deviate from the detection threshold as abnormal behaviors;

[0042] Label abnormal behaviors.

[0043] As a preference, step five is specifically as follows:

[0044] Use the established user portrait to extract the user's abnormal behavior pattern characteristics and build an abnormal behavior pattern feature library, where the behavior pattern characteristics include behavior frequency, behavior sequence and behavior time interval;

[0045] The user behavior sequence to be tested is matched with the abnormal behavior pattern feature library. Whether the user is an abnormal user is determined by whether the match is successful, and the abnormal features of the user behavior data are extracted accordingly. The specific process is as follows:

[0046] Use TF-IDF to transform the user behavior sequence to be tested into a feature vector A = [a1, a2, a3, ..., ai], and transform the abnormal feature sequence in the abnormal behavior pattern feature library into a feature vector B = [b1, b2, b3, ..., bj];

[0047] The distance between vector A and vector B is calculated by dynamic time warping (DTW). Vector B is used as the reference sequence B and vector A is used as the comparison sequence A. The distance is calculated by the formula as follows:

[0048] D(i,j)=d(i,j)+min[D(i-1,j),D(i,j-1),D(i-1,j-1)];

[0049] d(i,j)=1-cos(ai,bj);

[0050] Among them, d(i,j) is the cosine distance between ai and bj; D(i,j) is the sum of the distances between the first i coordinates of the comparison sequence A and the first j coordinates of the benchmark sequence B; when the D(i,j) value is less than the preset threshold, it means that the smaller the distance between vector A and vector B, the more similar the user behavior sequence to be tested is to the abnormal feature sequence in the abnormal behavior pattern feature library; when the D(i,j) value is greater than the set user behavior abnormal threshold, it means that the distance between vector A and vector B is larger, the user behavior sequence to be tested is judged to be an abnormal sequence, the user to be tested is marked as an abnormal user, and the identification of the abnormal user is completed.

[0051] As a preference, step six is ​​specifically as follows:

[0052] The specific quantification formula is as follows:

[0053]

[0054] Among them, a represents the amount of abnormal features detected; A represents the total amount of abnormal features in the data;

[0055] When the value of F is greater than or equal to the maximum value of the set accuracy threshold range δ1, the accuracy of extracting abnormal features of user behavior data is determined to be level one;

[0056] When the value of F is less than the maximum value δ1 of the accuracy threshold range and greater than or equal to the minimum value δ2 of the accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level 2;

[0057] When the value of F is less than the minimum value δ2 of the set accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level three.

[0058] Preferably, density reachability is defined as follows: if xn is in the neighborhood of xm and xm is a core object, then xn is density-reachable by xm, and if there exist a1, a2, ..., a n , where a1 = xm, a n = xn, and a i+1 By a i If the density is directly reachable, then xn can be reached by the density of xm.

[0059] Preferably, the support is defined as follows: the frequency of occurrence of a certain user behavior sequence in the data set.

[0060] Preferably, in labeling the abnormal behavior, the number of labels is greater than or equal to 1.

[0061] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0062] 1. The present invention obtains user behavior data through the server, and can record the user's operations on the platform in real time and comprehensively, thereby generating more accurate user trajectory segments. Furthermore, through frequent sequence mining, the user's frequent behavior sequences can be discovered. These sequences, as user features, can better represent the user's real behavior patterns. When distinguishing between abnormal and normal behaviors of users, the accuracy will be greatly improved, which helps to more accurately identify abnormal users;

[0063] 2. Based on the user portrait model and abnormal behavior pattern feature library, the present invention can more deeply understand the user's behavior preferences and needs. For abnormal behavior, the platform can respond in time, such as reminding users to pay attention to account security and taking necessary risk prevention and control measures, thereby protecting the rights and interests of users;

[0064] 3. The present invention establishes an abnormal behavior pattern feature library, so that the platform can quickly match the user behavior sequence to be tested with the abnormal behavior pattern in the library. Once the match is successful, the user can be judged as an abnormal user, and then corresponding risk prevention and control measures can be taken. This automated matching and judgment process greatly improves the efficiency and accuracy of risk prevention and control, and helps to reduce potential safety hazards and economic losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 A flow chart of a network behavior analysis method based on big data of the present invention is shown. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Example 1

[0069] See also Figure 1 As shown, a network behavior analysis method based on big data in this embodiment has the following process:

[0070] Step 1: Obtain user behavior data through the log files generated by the web server, identify users, and generate useful track segments based on user access; these log files record the activities of the web server, including user requests and server responses; user data includes but is not limited to user identity, access date, access path, access content, request results, and browsing time.

[0071] The process of user identification based on user behavior data is as follows:

[0072] When the IP addresses are different, they represent multiple different users;

[0073] When the IP address is the same and the user uses the same operating system and browser, it represents the same user;

[0074] When the IP address is the same but the user uses different operating systems and browsers, it represents multiple different users;

[0075] When a user requests to access a page that is not related to any page previously visited, it means that a new user has appeared;

[0076] When a user requests a page that has never been visited before and there is no reference trace in the log records, it means that a new user has appeared.

[0077] Decompose the user's visit into trace segments, expressed in the following form:

[0078] R= <U,{(P i ,T i ),…,(P k ,T k )}>;

[0079] Among them, R represents the set sequence of pages requested by the user within a period of time; U represents the user identifier; P represents the identifier of the page requested by the user; and T represents time.

[0080] Set the timeout threshold between adjacent user requests to θ, and determine whether T i -T i-1 ≤Q…1≤i<<Q. If it is satisfied, the trajectory is marked as a useful trajectory segment.

[0081] Step 2: After dividing the useful trajectory segments into different regions, perform frequent sequence mining, output the user's frequent behavior sequence and its support, use the frequent behavior sequence as the user feature, and assign weights.

[0082] The useful trajectory segments are sorted in chronological order to obtain the user behavior data sequence Y, and a user behavior sequence database is constructed. Each user behavior node has a corresponding position in the sequence, and the user behavior sequence represents the operation of a user within a certain timestamp.

[0083] User behavior data sequence Y = (y1, y2, y3, y4, ..., ym), where m is a positive integer. For each behavior xm in the sequence, create a corresponding point, and add the corresponding behavior identifier, timestamp and page ID to retain the context information of each point. Use the timestamp to sort the created points to obtain the user behavior data sequence point set X = (x1, x2, x3, x4, ..., xm), where m is a positive integer;

[0084] Take any point xm in the space as the center of the circle and set the radius to r to form a circular area. The set of all points in this circular area is marked as neighborhood B. r (xm),

[0085] B r (xm)={xn∈D│dist(xm,xn)≤r};

[0086] Among them, dist(xm,xn) represents the distance between xm and xn.

[0087] The minimum number of samples in the neighborhood is marked as MinPts;

[0088] Randomly select a point xm from the data set X and judge |B r (xm)| is greater than or equal to MinPts, when|B r (xm)|≥MinPts, then xm is determined to be a core object and added to the core set Y;

[0089] A core object xn is randomly selected from the core set Y, and all its density-reachable points are added to the new set C1 to form the first cluster; where density-reachable is defined as follows in this embodiment: if xn is in the neighborhood of xm and xm is a core object, then xn is directly reachable by xm through density, and if there are a1, a2, ..., a n , where a1 = xm, a n = xn, and a i+1 By a i Density is directly accessible, then xn is reachable by xm density;

[0090] Continue to access the next point in the data set X and repeat the above steps until all points in the data set are processed to obtain the cluster Cm. The points not included in the cluster are marked as noise and deleted.

[0091] The cluster Cm includes user basic attributes, position attributes, business attributes, time attributes and space attributes; among them, the basic attributes include user ID and IP address; the position attributes include the user's position level, such as administrator, ordinary user, etc.; the business attributes are the business functions of the user, such as operation user, audit user, etc.; the time attribute is the time segmentation of user access; the space attribute is the regional segmentation of user access.

[0092] The Apriori algorithm is used to mine frequent behavior sequences on the data after region segmentation, and the user behavior sequence and its support are output. The support threshold is set to ρ, and the behavior sequences below the support threshold ρ are eliminated. The behavior sequences above the support threshold ρ and their support are retained and marked as frequent behavior sequences. Among them, the support represents the frequency of a user behavior sequence in the data set.

[0093] Frequent behavior sequences are used as user features. Each user feature represents a specific user behavior pattern, including multiple user behavior steps. A weight is given to each user feature to reflect its importance in user behavior.

[0094] Step 3: Distinguish the user's abnormal behavior from normal behavior, and abstract the obtained abnormal behavior data into labels.

[0095] Establish a normal behavior model to capture the user's normal behavior pattern;

[0096] Set detection thresholds for abnormal behaviors, including but not limited to upper and lower limits of operation frequency and abnormal range of access duration, and classify behaviors that deviate from the detection threshold as abnormal behaviors;

[0097] Label abnormal behaviors.

[0098] Step 4: Build a user portrait model.

[0099] Step 5: Establish an abnormal behavior pattern feature library, match the user behavior sequence to be tested with the abnormal behavior pattern feature library, and determine whether it is an abnormal user.

[0100] The user portrait established in the above steps is used to extract the user's abnormal behavior pattern characteristics, and an abnormal behavior pattern feature library is constructed, where the behavior pattern characteristics include but are not limited to behavior frequency, behavior sequence and behavior time interval.

[0101] The user behavior sequence to be tested is matched with the abnormal behavior pattern feature library. Whether the user is an abnormal user is determined by whether the match is successful, and the abnormal features of the user behavior data are extracted accordingly. The specific process is as follows:

[0102] Use TF-IDF to transform the user behavior sequence to be tested into a feature vector A = [a1, a2, a3, ..., ai], and transform the abnormal feature sequence in the abnormal behavior pattern feature library into a feature vector B = [b1, b2, b3, ..., bj];

[0103] The distance between vector A and vector B is calculated by dynamic time warping (DTW). Vector B is used as the reference sequence B and vector A is used as the comparison sequence A. The distance is calculated by the formula as follows:

[0104] D(i,j)=d(i,j)+min[D(i-1,j),D(i,j-1),D(i-1,j-1)];

[0105] d(i,j)=1-cos(ai,bj);

[0106] Among them, d(i,j) is the cosine distance between ai and bj; D(i,j0) is the sum of the distances between the first i coordinates of the comparison sequence A and the first j coordinates of the reference sequence B; when the D(i,j) value is less than the preset threshold, it means that the smaller the distance between vector A and vector B, the more similar the user behavior sequence to be tested is to the abnormal feature sequence in the abnormal behavior pattern feature library; when the D(i,j0 value is greater than the set user behavior abnormal threshold, it means that the distance between vector A and vector B is larger, the user behavior sequence to be tested is judged to be an abnormal sequence, the user to be tested is marked as an abnormal user, and the abnormal user is identified.

[0107] Step 6: Extract abnormal behavior features of abnormal users, quantify the accuracy of extracting abnormal behaviors of users, and complete the analysis of abnormal behavior features of abnormal users. The specific quantification formula is as follows:

[0108]

[0109] Among them, a represents the amount of abnormal features detected; A represents the total amount of abnormal features in the data.

[0110] When the value of F is greater than or equal to the maximum value of the set accuracy threshold range δ1, the accuracy of extracting abnormal features of user behavior data is determined to be level one;

[0111] When the value of F is less than the maximum value δ1 of the accuracy threshold range and greater than or equal to the minimum value δ2 of the accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level 2;

[0112] When the value of F is less than the minimum value δ2 of the set accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level three.

[0113] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0114] Those of ordinary skill in the art will appreciate that the algorithmic steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0115] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only some logical function divisions. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code. The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A network behavior analysis method based on big data, characterized in that: The process is as follows: Step 1: Obtain user behavior data, identify users, and generate useful track segments based on user access; user data includes user identity, access date, access path, access content, request results, and browsing time; Step 2: After dividing the useful trajectory segments into different regions, perform frequent sequence mining, output the user's frequent behavior sequence and its support, use the frequent behavior sequence as the user feature, and assign weights; Step 3: Distinguish the user's abnormal behavior from normal behavior, and abstract the obtained abnormal behavior data into labels; Step 4: Build a user portrait model; Step 5: Establish an abnormal behavior pattern feature library, match the behavior sequence of the user to be tested with the abnormal behavior pattern feature library, and determine whether it is an abnormal user; Step 6: Extract abnormal behavior features of abnormal users, quantify the accuracy of extracting abnormal behaviors of users, and complete the analysis of abnormal behavior features of abnormal users.

2. According to the big data-based network behavior analysis method of claim 1, it is characterized in that: The step 1 is specifically as follows: Obtain user behavior data through log files generated by the web server; The process of user identification based on user behavior data is as follows: When the IP addresses are different, they represent multiple different users; When the IP address is the same and the user uses the same operating system and browser, it represents the same user; When the IP address is the same but the user uses different operating systems and browsers, it represents multiple different users; When a user requests to access a page that is not related to any page previously visited, it means that a new user has appeared; When a user requests a page that has never been visited before and there is no reference trace in the log record, it means that a new user has appeared; Decompose the user's visit into trace segments, expressed in the following form: R=<U,{(P i ,T i ),…,(P k ,T k )}>; Where R represents the set sequence of pages requested by the user within a period of time; U represents the user ID; P represents the identifier of the page requested by the user; T represents time; Set the timeout threshold between adjacent user requests to θ, and determine whether T i -T i-1 ≤Q…1≤i<<Q. If it is satisfied, the trajectory is marked as a useful trajectory segment.

3. According to the big data-based network behavior analysis method of claim 1, it is characterized in that: The step 2 is specifically as follows: Sort the useful trajectory segments in chronological order to obtain the user behavior data sequence Y, and build a user behavior sequence database. Each user behavior node has its corresponding position in the sequence, and the user behavior sequence represents the operation of a user within a certain timestamp; User behavior data sequence Y = (y1, y2, y3, y4, ..., ym), where m is a positive integer. For each behavior xm in the sequence, create a corresponding point, add the corresponding behavior identifier, timestamp and page ID, use the timestamp to sort the created points, and obtain the user behavior data sequence point set X = (x1, x2, x3, x4, ..., xm), where m is a positive integer; Take any point xm in the space as the center of the circle and set the radius to r to form a circular area. The set of all points in this circular area is marked as neighborhood B. r (xm), B r (xm)={xn∈D│dist(xm,xn)≤r}; Among them, dist(xm,xn) represents the distance between xm and xn; The minimum number of samples in the neighborhood is marked as MinPts; Randomly select a point xm from the data set X and judge |B r (xm)| is greater than or equal to MinPts, when|B r (xm)|≥MinPts, then xm is determined to be a core object and added to the core set Y; Randomly select a core object xn from the core set Y, and add all its density-reachable points to the new set C1 to form the first cluster; Continue to access the next point in the data set X and repeat the above steps until all points in the data set are processed to obtain the cluster Cm. The points not included in the cluster are marked as noise and deleted. The cluster Cm includes user basic attributes, position attributes, business attributes, time attributes and space attributes; among which, the basic attributes include user ID and IP address; the position attributes include the position level of the user; the business attributes are the business functions of the user; the time attribute is the time segmentation of the user's visit; the space attribute is the regional segmentation of the user's visit; The Apriori algorithm is used to mine frequent behavior sequences on the data after region segmentation, and the user behavior sequence and its support are output; the support threshold is set to ρ, the behavior sequences below the support threshold ρ are eliminated, the behavior sequences above the support threshold ρ and their support are retained, and they are marked as frequent behavior sequences; Frequent behavior sequences are used as user features. Each user feature represents a specific user behavior pattern, including multiple user behavior steps; a weight is assigned to each user feature.

4. According to the big data-based network behavior analysis method of claim 1, it is characterized in that: The step three is as follows: Establish a normal behavior model to capture the user's normal behavior pattern; Set detection thresholds for abnormal behaviors, including upper and lower limits of operation frequency and abnormal range of access duration, and classify behaviors that deviate from the detection threshold as abnormal behaviors; Label abnormal behaviors.

5. The network behavior analysis method based on big data according to claim 1 is characterized in that: The step five is as follows: Use the established user portrait to extract the user's abnormal behavior pattern characteristics and build an abnormal behavior pattern feature library, where the behavior pattern characteristics include behavior frequency, behavior sequence and behavior time interval; The user behavior sequence to be tested is matched with the abnormal behavior pattern feature library. Whether the user is an abnormal user is determined by whether the match is successful, and the abnormal features of the user behavior data are extracted accordingly. The specific process is as follows: Use TF-IDF to transform the user behavior sequence to be tested into a feature vector A = [a1, a2, a3, ..., ai], and transform the abnormal feature sequence in the abnormal behavior pattern feature library into a feature vector B = [b1, b2, b3, ..., bj]; The distance between vector A and vector B is calculated by dynamic time warping (DTW). Vector B is used as the reference sequence B and vector A is used as the comparison sequence A. The distance is calculated by the formula as follows: D(i,j)=d(i,j)+min[D(i-1,j),D(i,j-1),D(i-1,j-1)]; d(i,j)=1-cos(ai,bj); Among them, d(i,j) is the cosine distance between ai and bj; D(i,j) is the sum of the distances between the first i coordinates of the comparison sequence A and the first j coordinates of the benchmark sequence B; when the D(i,j) value is less than the preset threshold, it means that the smaller the distance between vector A and vector B, the more similar the user behavior sequence to be tested is to the abnormal feature sequence in the abnormal behavior pattern feature library; when the D(i,j) value is greater than the set user behavior abnormal threshold, it means that the distance between vector A and vector B is larger, the user behavior sequence to be tested is judged to be an abnormal sequence, the user to be tested is marked as an abnormal user, and the identification of the abnormal user is completed.

6. The network behavior analysis method based on big data according to claim 1 is characterized in that: The step six is ​​as follows: The specific quantification formula is as follows: Among them, a represents the amount of abnormal features detected; A represents the total amount of abnormal features in the data; When the value of F is greater than or equal to the maximum value of the set accuracy threshold range δ1, the accuracy of extracting abnormal features of user behavior data is determined to be level one; When the value of F is less than the maximum value δ1 of the accuracy threshold range and greater than or equal to the minimum value δ2 of the accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level 2; When the value of F is less than the minimum value δ2 of the set accuracy threshold range, the accuracy of extracting abnormal features of user behavior data is determined to be level three.

7. The network behavior analysis method based on big data according to claim 3 is characterized in that: The density reachability is defined as follows: if xn is in the neighborhood of xm and xm is a core object, then xn is density-reachable by xm, if there exist a1, a2, ..., a n , where a1 = xm, a n = xn, and a i+1 By a i If the density is directly reachable, then xn can be reached by the density of xm.

8. The network behavior analysis method based on big data according to claim 3 is characterized in that: The support is defined as follows: the frequency of a certain user behavior sequence appearing in the data set.

9. The network behavior analysis method based on big data according to claim 3 is characterized in that: In the labeling process for abnormal behavior, the number of labels is greater than or equal to 1.

Images