Intelligent e-commerce user abnormal behavior detection method and system

Abstract

The invention discloses an intelligent e-commerce user abnormal behavior detection method and system. The method comprises the steps of data acquisition, data preprocessing, data processing and e-commerce user abnormal behavior detection. The invention belongs to the field of anomaly detection, and particularly relates to an intelligent e-commerce user abnormal behavior detection method and system, and the method comprises the steps: filtering a part of abnormal behaviors in advance through designing low-fluctuation users, sudden abnormal users and diverse abnormal users; based on design distance measurement, the similarity of user behavior sequences is measured, and secondary optimization is carried out on the centroid, so that the adaptability of the centroid to user behavior distribution is improved; the accuracy of final user abnormal behavior detection is improved; by designing a dynamic distance threshold and multi-dimensional feature evaluation, comprehensively considering the user behavior data, and dynamically adjusting an anomaly detection threshold according to the distribution characteristics of the user behavior data, an abnormal user is accurately identified, the misjudgment risk is reduced, and the detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of anomaly detection technology, and specifically to an intelligent e-commerce user abnormal behavior detection method and system. Background Art

[0002] The e-commerce user abnormal behavior detection method is a technical means for identifying abnormal or suspicious user behavior on the e-commerce platform. By analyzing the user's behavior data, it identifies those users who deviate from the normal behavior pattern, helping the platform to deal with potential risks and problems, such as fraud, malicious operations or account abuse. However, the general e-commerce user abnormal behavior detection method has the problem of incomplete recognition of user abnormal behavior, inability to accurately measure the similarity of user behavior sequences, and adaptability to user behavior distribution, which leads to inaccurate detection of user abnormal behavior; the general e-commerce user abnormal behavior detection method has the problem of improper handling of the behavior distribution of different user groups, misjudgment due to abnormal fluctuations of a single feature, and poor accuracy in abnormal behavior identification. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an intelligent e-commerce user abnormal behavior detection method and system. In view of the problem that the general e-commerce user abnormal behavior detection method has incomplete recognition of user abnormal behavior, cannot accurately measure the similarity of user behavior sequences, and cannot adapt to user behavior distribution, which leads to inaccurate detection of user abnormal behavior, this solution filters some abnormal behaviors in advance by designing low-volatility users, sudden abnormal users and diverse abnormal users; measures the similarity of user behavior sequences based on the designed distance metric, and improves the adaptability of the centroid to the user behavior distribution by performing secondary optimization on the centroid; thereby improving the accuracy of abnormal behavior detection of final users; in view of the problem that the general e-commerce user abnormal behavior detection method has improper handling of the behavior distribution of different user groups, misjudgment due to abnormal fluctuations of a single feature, and thus leads to poor accuracy in abnormal behavior recognition, this solution designs a dynamic distance threshold and multi-dimensional feature evaluation, comprehensively considers user behavior data, dynamically adjusts the abnormal detection threshold according to the distribution characteristics of user behavior data, accurately identifies abnormal users, reduces the risk of misjudgment, and improves detection accuracy.

[0004] The technical solution adopted by the present invention is as follows: The present invention provides an intelligent e-commerce user abnormal behavior detection method, which includes the following steps:

[0005] Step S1: data collection;

[0006] Step S2: data preprocessing;

[0007] Step S3: data processing;

[0008] Step S4: Detection of abnormal behavior of e-commerce users.

[0009] Furthermore, in step S1, the data collection is to collect historical e-commerce user behavior data; the historical e-commerce user behavior data includes e-commerce platform activity data, transaction behavior data, platform interaction data, time and user behavior type; and the user behavior type is used as a data label.

[0010] Further, in step S2, the data preprocessing is to perform data cleaning, data conversion and preliminary anomaly detection on the collected data; the data cleaning is to process missing values ​​and duplicate values; the data conversion is to convert the collected data into vector form; the preliminary anomaly detection is to identify low-volatility users, sudden abnormal users and diverse abnormal users; the identification of low-volatility users is that users have abnormally stable behavior patterns, and the definition standard is that more than 90% of the behavior values ​​are the same or the daily behavior fluctuation is less than the dynamic threshold; the fluctuation condition detection is expressed as: ; Abnormal users with sudden changes are users who have extreme behaviors in a short period of time. The behavior change rate is defined as for ;like , it is marked as a sudden abnormality; diverse abnormal users are users whose behavior characteristics change frequently, and the behavior feature vector of each time period is extracted. , based on the cosine similarity, the similarity of the feature vectors of adjacent time periods is calculated. If the similarity of the feature vectors of adjacent time periods is lower than the diversity threshold, it is determined as a behavior mode switch. If the number of behavior mode switches within a month is higher than the switching threshold, it is determined as a diverse abnormal user; where L F is the low-volatility user detection result, 1 is an abnormal user, and 0 is a non-abnormal user; is the difference threshold; x i is the behavior sequence data of the i-th day; and are the maximum and minimum values ​​of the elements in the d-th dimension user behavior sequence data; n is the number of days in the time period; count(·) is the number of days that meet the conditions; constant is a fixed behavior vector; max(·) and min(·) are the maximum and minimum values ​​of the behavior value, respectively; β is the threshold coefficient; is the user standard deviation behavior; is the user's mean behavior; is the L1 norm; is the standard value; D is the total dimension, and d is the dimension index.

[0011] Furthermore, in step S3, the data processing specifically includes the following steps:

[0012] Step S31: Select the initial centroid; if there are low-volatility users, divide the remaining users into k-1 clusters; otherwise, divide into k clusters; use the result of the K-means algorithm as the initial centroid;

[0013] Step S32: define a distance metric; measure the similarity between behavior sequences based on the distance metric; express it as: ; Among them, PJL d is the similarity of the d-th dimension behavior sequence; PJL is the behavior sequence similarity; x1 and y1 are the feature values ​​of the d-th dimension behavior sequence of user x and user y respectively; is the weight of the d-th dimension behavior; X and Y are the behavior sequences of user x and user y respectively; i is the day index; and are the feature values ​​of user x and user y in the dth dimension on the i-th day;

[0014] Step S33: centroid optimization; calculate the initial cluster centroid based on the distance metric; expressed as: ; Optimize the initial cluster centroid, expressed as: ; Calculate optimization loss , expressed as: ;Use minimize method to get the final optimized centroid; Among them, is the initial cluster centroid; is the weight vector; is the optimized cluster centroid; s and t are linear adjustment parameters used to optimize the centroid vector; is the set of users included in the jth cluster; r is the deviation between the user behavior sequence and the centroid during the centroid optimization process; C is the cluster center position; is the regularization weight; is the square of the L2 norm; is the feature vector of the dth dimension in the time period; I is the unit vector;

[0015] Step S34: clustering iteration; calculate the distance from each user behavior sequence to the centroid based on PJL, and assign the user to the nearest cluster; update the cluster centroid and repeat the user assignment; until convergence or the maximum number of iterations is reached.

[0016] Furthermore, in step S4, the e-commerce user abnormal behavior detection is based on the clustering results to evaluate the similarity of user behavior sequences; an abnormal user is defined as a user whose distance from the centroid of the cluster to which he belongs exceeds a set threshold, expressed as: ;in, is abnormal user identification, 1 is an abnormal user, 0 is a non-abnormal user; is the average distance of the jth cluster; The standard deviation of the cluster distances; is the d-th dimension of C j .

[0017] The present invention provides an intelligent e-commerce user abnormal behavior detection system, which includes a data acquisition module, a data preprocessing module, a data processing module and an e-commerce user abnormal behavior detection module;

[0018] The data collection module collects historical e-commerce user behavior data and sends the data to the data preprocessing module;

[0019] The data preprocessing module performs data cleaning, data conversion and preliminary anomaly detection on the collected data; and sends the data to the data processing module;

[0020] The data processing module measures the similarity of user behavior sequences based on the designed distance metric and performs secondary optimization on the centroid; thereby realizing data processing; and sending the data to the e-commerce user abnormal behavior detection module;

[0021] The e-commerce user abnormal behavior detection module realizes e-commerce user abnormal behavior detection based on clustering results by designing dynamic distance thresholds and multi-dimensional feature evaluation.

[0022] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0023] (1) In view of the fact that the general e-commerce user abnormal behavior detection method has the problems of incomplete recognition of user abnormal behavior, inability to accurately measure the similarity of user behavior sequences, and inability to adapt to user behavior distribution, which leads to inaccurate detection of user abnormal behavior, this solution filters out some abnormal behaviors in advance by designing low-volatility users, sudden abnormal users, and diverse abnormal users; measures the similarity of user behavior sequences based on the designed distance metric, and improves the adaptability of the centroid to the user behavior distribution by performing secondary optimization on the centroid; thereby improving the accuracy of abnormal behavior detection for end users.

[0024] (2) The general e-commerce user abnormal behavior detection method has the problem of improper handling of the behavior distribution of different user groups, misjudgment due to abnormal fluctuations of a single feature, and thus poor accuracy in abnormal behavior identification. This solution designs a dynamic distance threshold and multi-dimensional feature evaluation, comprehensively considers user behavior data, and dynamically adjusts the anomaly detection threshold according to the distribution characteristics of user behavior data, accurately identifies abnormal users, reduces the risk of misjudgment, and improves detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flow chart of an intelligent e-commerce user abnormal behavior detection method provided by the present invention;

[0026] Figure 2A schematic diagram of an intelligent e-commerce user abnormal behavior detection system provided by the present invention;

[0027] Figure 3 It is a schematic diagram of the process of step S3.

[0028] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] In the description of the present invention, it is necessary to understand that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred system or element must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.

[0031] Example 1, see Figure 1 The present invention provides an intelligent e-commerce user abnormal behavior detection method, which includes the following steps:

[0032] Step S1: Data collection, collecting historical e-commerce user behavior data;

[0033] Step S2: Data preprocessing, data cleaning, data conversion and preliminary anomaly detection of the collected data;

[0034] Step S3: Data processing, measuring the similarity of user behavior sequences based on the designed distance metric, and performing secondary optimization on the centroid; thus achieving data processing;

[0035] Step S4: abnormal behavior detection of e-commerce users, based on clustering results, abnormal behavior detection of e-commerce users is implemented.

[0036] Example 2, see Figure 1This embodiment is based on the above embodiment. In step S1, the historical e-commerce user behavior data includes e-commerce platform activity data, transaction behavior data, platform interaction data, time and user behavior type; the user behavior type is used as a data label; the e-commerce platform activity data includes browsing behavior data, search behavior data, shopping cart behavior data, purchase behavior data and collection behavior data; the transaction behavior data includes order details data, payment data and refund and return data; the platform interaction data includes message interaction data and community participation data; the user behavior type includes normal behavior, mild abnormal behavior and serious abnormal behavior.

[0037] Example 3, see Figure 1 , this embodiment is based on the above embodiment. In step S2, data preprocessing is to perform data cleaning, data conversion and preliminary anomaly detection on the collected data; the data cleaning is to process missing values ​​and duplicate values; the data conversion is to convert the collected data into vector form; the preliminary anomaly detection is to identify low-volatility users, sudden abnormal users and diverse abnormal users; to identify low-volatility users, that is, users with abnormally stable behavior patterns, the definition standard is that more than 90% of the behavior values ​​are the same or the daily behavior fluctuation is less than the dynamic threshold; the fluctuation condition detection is expressed as: ; Abnormal users with sudden changes are users who have extreme behaviors in a short period of time. The behavior change rate is defined as for ;like , it is marked as a sudden abnormality; diverse abnormal users are users whose behavior characteristics change frequently, and the behavior feature vector of each time period is extracted. , based on the cosine similarity, the similarity of the feature vectors of adjacent time periods is calculated. If the similarity of the feature vectors of adjacent time periods is lower than the diversity threshold, it is determined as a behavior mode switch. If the number of behavior mode switches within a month is higher than the switching threshold, it is determined as a diverse abnormal user; where L F is the low-volatility user detection result, 1 is an abnormal user, and 0 is a non-abnormal user; is the difference threshold; x i is the behavior sequence data of the i-th day; and are the maximum and minimum values ​​of the elements in the d-th dimension user behavior sequence data; n is the number of days in the time period; count(·) is the number of days that meet the conditions; constant is a fixed behavior vector; max(·) and min(·) are the maximum and minimum values ​​of the behavior value, respectively; β is the threshold coefficient; is the user standard deviation behavior; is the user's mean behavior; is the L1 norm; is the standard value; D is the total dimension, and d is the dimension index.

[0038] Example 4, see Figure 1 and Figure 3 This embodiment is based on the above embodiment. In step S3, data processing specifically includes the following steps:

[0039] Step S31: Select the initial centroid; if there are low-volatility users, divide the remaining users into k-1 clusters; otherwise, divide into k clusters; use the result of the K-means algorithm as the initial centroid;

[0040] Step S32: define a distance metric; measure the similarity between behavior sequences based on the distance metric; express it as: ; Among them, PJL d is the similarity of the d-th dimension behavior sequence; PJL is the behavior sequence similarity; x1 and y1 are the feature values ​​of the d-th dimension behavior sequence of user x and user y respectively; is the weight of the d-th dimension behavior; X and Y are the behavior sequences of user x and user y respectively; i is the day index; and are the feature values ​​of user x and user y in the dth dimension on the i-th day;

[0041] Step S33: centroid optimization; calculate the initial cluster centroid based on the distance metric; expressed as: ; Optimize the initial cluster centroid, expressed as: ; Calculate optimization loss , expressed as: ;Use minimize method to get the final optimized centroid; Among them, is the initial cluster centroid; is the weight vector; is the optimized cluster centroid; s and t are linear adjustment parameters used to optimize the centroid vector; is the set of users included in the jth cluster; r is the deviation between the user behavior sequence and the centroid during the centroid optimization process; C is the cluster center position; is the regularization weight; is the square of the L2 norm; is the feature vector of the dth dimension in the time period; I is the unit vector;

[0042] Step S34: clustering iteration; calculate the distance from each user behavior sequence to the centroid based on PJL, and assign the user to the nearest cluster; update the cluster centroid and repeat the user assignment; until convergence or the maximum number of iterations is reached.

[0043] By performing the above operations, the general e-commerce user abnormal behavior detection method has the problem of incomplete recognition of user abnormal behavior, inability to accurately measure the similarity of user behavior sequences, and adaptability to user behavior distribution, which leads to inaccurate detection of user abnormal behavior. This solution filters some abnormal behaviors in advance by designing low-volatility users, sudden abnormal users, and diverse abnormal users; measures the similarity of user behavior sequences based on the designed distance metric, and improves the adaptability of the centroid to the user behavior distribution by performing secondary optimization on the centroid; thereby improving the accuracy of abnormal behavior detection for end users.

[0044] Example 5, see Figure 1 This embodiment is based on the above embodiment. In step S4, the abnormal behavior detection of e-commerce users is based on the clustering results, and the similarity of user behavior sequences is evaluated; an abnormal user is defined as a user whose distance from the centroid of the cluster to which he belongs exceeds a set threshold, which is expressed as: ;in, is abnormal user identification, 1 is an abnormal user, 0 is a non-abnormal user; is the average distance of the jth cluster; The standard deviation of the cluster distances; is the d-th dimension of C j .

[0045] By performing the above operations, the general e-commerce user abnormal behavior detection method has the problem of improper handling of the behavior distribution of different user groups, misjudgment due to abnormal fluctuations of a single feature, and then poor accuracy in abnormal behavior identification. This solution designs a dynamic distance threshold and multi-dimensional feature evaluation, comprehensively considers user behavior data, and dynamically adjusts the anomaly detection threshold according to the distribution characteristics of user behavior data, accurately identifies abnormal users, reduces the risk of misjudgment, and improves detection accuracy.

[0046] Example 6, see Figure 2 , this embodiment is based on the above embodiment, and the present invention provides an intelligent e-commerce user abnormal behavior detection system, including a data acquisition module, a data preprocessing module, a data processing module and an e-commerce user abnormal behavior detection module;

[0047] The data collection module collects historical e-commerce user behavior data and sends the data to the data preprocessing module;

[0048] The data preprocessing module performs data cleaning, data conversion and preliminary anomaly detection on the collected data; and sends the data to the data processing module;

[0049] The data processing module measures the similarity of user behavior sequences based on the designed distance metric and performs secondary optimization on the centroid; thereby realizing data processing; and sending the data to the e-commerce user abnormal behavior detection module;

[0050] The e-commerce user abnormal behavior detection module realizes e-commerce user abnormal behavior detection based on clustering results by designing dynamic distance thresholds and multi-dimensional feature evaluation.

[0051] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0052] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0053] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. An intelligent e-commerce user abnormal behavior detection method, characterized by: The method comprises the following steps: Step S1: Data collection, collecting historical e-commerce user behavior data; Step S2: data preprocessing; Step S3: Data processing, measuring the similarity of user behavior sequences based on the designed distance metric, and performing secondary optimization on the centroid; thus achieving data processing; Step S4: abnormal behavior detection of e-commerce users, based on clustering results; Step S3 includes centroid optimization; calculating the preliminary cluster centroid based on the distance metric; expressed as: ; Optimize the initial cluster centroid, expressed as: ; Calculate optimization loss , expressed as: ;Use minimize method to get the final optimized centroid; Among them, is the initial cluster centroid; is the weight vector; is the optimized cluster centroid; s and t are linear adjustment parameters used to optimize the centroid vector; is the set of users included in the jth cluster; r is the deviation between the user behavior sequence and the centroid during the centroid optimization process; C is the cluster center position; is the regularization weight; is the square of the L2 norm; is the feature vector of the dth dimension in the time period; I is the unit vector; is the square of the L2 norm; PJL is the behavior sequence similarity; X is the behavior sequence of user x; is the weight of the behavior in the dth dimension; β is the threshold coefficient; is the user standard deviation behavior; is the L1 norm.

2. According to claim 1, the intelligent e-commerce user abnormal behavior detection method is characterized by: In step S2, the data preprocessing is to clean the collected data, convert the data and perform preliminary anomaly detection; the data cleaning is to process missing values ​​and duplicate values; the data conversion is to convert the collected data into vector form; the preliminary anomaly detection is to identify low-volatility users, sudden abnormal users and diverse abnormal users; the identification of low-volatility users is that users have abnormally stable behavior patterns, and the definition standard is that more than 90% of the behavior values ​​are the same or the daily behavior fluctuation is less than the dynamic threshold; the fluctuation condition detection is expressed as: ; Abnormal users with sudden changes are users who have extreme behaviors in a short period of time. The behavior change rate is defined as for ;like , it is marked as a sudden abnormality; Diverse abnormal users are users whose behavior characteristics change frequently. The behavior feature vector of each time period is extracted. , based on the cosine similarity, the similarity of the feature vectors of adjacent time periods is calculated. If the similarity of the feature vectors of adjacent time periods is lower than the diversity threshold, it is determined as a behavior mode switch. If the number of behavior mode switches within a month is higher than the switching threshold, it is determined as a diverse abnormal user; where L F is the low-volatility user detection result, 1 is an abnormal user, and 0 is a non-abnormal user; is the difference threshold; x i is the behavior sequence data of the i-th day; and are the maximum and minimum values ​​of the elements in the d-th dimension of the user behavior sequence data; n is the number of days in the time period; count(·) is the number of days that meet the conditions; constant is a fixed behavior vector; max(·) and min(·) are the maximum and minimum values ​​of the behavior value, respectively; β is the threshold coefficient; is the user standard deviation behavior; is the user's mean behavior; is the L1 norm; is the standard value; D is the total dimension, and d is the dimension index.

3. According to claim 2, an intelligent e-commerce user abnormal behavior detection method is characterized by: In step S3, the data processing further includes the following steps: Step S31: Select the initial centroid; if there are low-volatility users, divide the remaining users into k-1 clusters; otherwise, divide into k clusters; use the result of the K-means algorithm as the initial centroid; Step S32: define a distance metric; measure the similarity between behavior sequences based on the distance metric; express it as: ; Among them, PJL d is the similarity of the d-th dimension behavior sequence; PJL is the behavior sequence similarity; x1 and y1 are the feature values ​​of the d-th dimension behavior sequence of user x and user y respectively; is the weight of the d-th dimension behavior; X and Y are the behavior sequences of user x and user y respectively; i is the day index; and are the feature values ​​of user x and user y in the dth dimension on the i-th day; Step S34: clustering iteration; calculate the distance from each user behavior sequence to the centroid based on PJL, and assign the user to the nearest cluster; update the cluster centroid and repeat the user assignment; until convergence or the maximum number of iterations is reached.

4. According to claim 3, the intelligent e-commerce user abnormal behavior detection method is characterized by: In step S4, the e-commerce user abnormal behavior detection is based on the clustering results to evaluate the similarity of user behavior sequences; an abnormal user is defined as a user whose distance from the centroid of the cluster to which he belongs exceeds a set threshold, expressed as: ;in, is abnormal user identification, 1 is an abnormal user, 0 is a non-abnormal user; is the average distance of the jth cluster; The standard deviation of the cluster distances; is the d-th dimension of C j .

5. According to claim 1, the intelligent e-commerce user abnormal behavior detection method is characterized by: In step S1, the historical e-commerce user behavior data includes e-commerce platform activity data, transaction behavior data, platform interaction data, time and user behavior type; the user behavior type is used as a data label.

6. An intelligent e-commerce user abnormal behavior detection system, used to implement an intelligent e-commerce user abnormal behavior detection method as described in any one of claims 1-5, characterized in that: It includes data collection module, data preprocessing module, data processing module and e-commerce user abnormal behavior detection module; The data collection module collects historical e-commerce user behavior data and sends the data to the data preprocessing module; The data preprocessing module performs data cleaning, data conversion and preliminary anomaly detection on the collected data; and sends the data to the data processing module; The data processing module measures the similarity of user behavior sequences based on the designed distance metric and performs secondary optimization on the centroid; thereby realizing data processing; and sending the data to the e-commerce user abnormal behavior detection module; The e-commerce user abnormal behavior detection module realizes e-commerce user abnormal behavior detection based on clustering results by designing dynamic distance thresholds and multi-dimensional feature evaluation.

Images