Big data security analysis method fusing anomaly detection and self-learning

Abstract

The invention belongs to the technical field of anomaly detection, and particularly relates to an anomaly detection and self-learning fused big data security analysis method, which comprises the following steps of: acquiring a user operation log, a transaction flow and equipment information in real time; s2, based on the data in the step S1, generating a time sequence operation feature and an equipment association feature; inputting the features in the step S2 into an integrated model, and outputting a dynamic risk score; a dynamic risk score corresponding to a current transaction behavior of a user is calculated and then compared with a decision threshold value, corresponding decision actions including transaction interception or security verification can be output, and in addition, the decision actions can be output through a time sequence operation feature, a device association feature, a time sequence risk score obtained through calculation and an association risk score obtained through calculation. According to the technical scheme, the system can be helped to identify the abnormal point and the associated account of the current transaction behavior of the user, so that after the abnormal point is identified, the corresponding decision-making action can be made in combination with the dynamic risk score, and the loss caused by embezzlement of the user account is avoided.

Description

Technical Field

[0001] This invention belongs to the field of anomaly detection technology, specifically a big data security analysis method that integrates anomaly detection and self-learning. Background Technology

[0002] In the online trading environment of e-commerce platforms, users can conveniently purchase a vast number of goods from different sellers, but this process also exacerbates the risk of account theft.

[0003] Specifically, this manifests as follows: Illegal groups steal user accounts through phishing attacks and credential stuffing, then use the linked bank cards or credit cards to conduct fraudulent transactions. For example, attackers may use stolen accounts to purchase electronic gift cards or virtual goods in bulk, then launder the money, resulting in financial losses for users. Existing defense technologies primarily rely on static rules (such as transaction amount thresholds) and single models (such as isolated forests for detecting abnormal logins), but they suffer from the following shortcomings: The rule engine cannot identify new account theft methods, such as using device farms to bypass IP restrictions, resulting in a high false negative rate; the single-dimensional model ignores the timing characteristics of account operations, such as high-frequency payment intervals, and mistakenly judges users' shopping behavior as fraudulent transactions. To address this, the present invention provides a big data security analysis method that integrates anomaly detection and self-learning. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0005] The technical solution adopted by this invention to solve its technical problem is: the big data security analysis method integrating anomaly detection and self-learning described in this invention includes the following steps: S1: Real-time acquisition of user operation logs, transaction records, and device information; S2: Based on the data from step S1, generate time-series operation features. Device-related features ; S3: Input the features from step S2 into the integrated model and output a dynamic risk score. : ; in, For feature weights, Time-series risk score For associated risk classification; S4: Based on risk score With decision threshold Based on the comparison results, execute transaction interception or security verification; S5: Update feature weights based on the interception results. , With decision threshold .

[0006] Preferably, in S2, the timing operation features The method for obtaining it is as follows: S21: Calculate the standard deviation of user operation intervals as a time-series operation characteristic. ; According to the formula: ; in, The time taken for a single operation, in units of , For the average interval, Number of operations within 1 minute; In S3, time-series risk classification The calculation method is as follows: S31: Based on Calculate time series risk score ; ; in, This is the hidden state of the LSTM. This serves as a benchmark for normal behavior. Number of consecutive operations ,and At that time, forced 0, and trigger device fingerprint collection.

[0007] Preferably, in S2, the device association feature The method for obtaining it is as follows: S22: Calculate the number of device-related accounts as a device association feature. ; According to the formula: ; in, For device fingerprint, For user-device relationship sets.

[0008] Preferably, in step S3, the associated risk is divided. The calculation method is as follows: S32: Based on device association features Calculate the associated risk score ; According to the formula: ; in, For the sigmoid function, For graph convolutional networks; When the new account rate ,and At that time, forced All associated accounts were frozen and cross-platform risk labels were generated.

[0009] Preferably, in step S4, the method for performing transaction interception or security verification is as follows: S41: Extract Risk Score and decision threshold , Comparison: when At that time, execute transaction interception; when At that time, a security verification is performed.

[0010] Preferably, step S3 further includes: S33: Calculate real-time login location deviation ; According to the formula: ; in, Commonly used coordinates, The coordinates of this login location; S34: When And after logging in When initiating a payment within minutes, a displacement penalty factor is added when calculating the risk score. According to the formula: ; in, As a penalty factor; S35: Based on displacement penalty factor Correct risk score ; ; in, This is the updated risk score.

[0011] Preferably, the parameter update in S5 includes: S51: Obtain the false alarm rate Adjust the weights of time-series features; According to the formula: ; in, For the learning rate, and , False alarm rate; S52: According to the updated Automatically calculate the updated ; in, .

[0012] Preferably, S5 further includes: S53: Obtain the displacement false alarm rate Update penalty factors ; According to the formula: ; in, This represents the false alarm rate for displacement.

[0013] Preferably, in S4, the decision threshold The calculation method is as follows: According to the formula: ; ; in, It is a sign function, and its range of values ​​is ; when , ,but 0.6; when , ,but 0.8; when , ,but 0.7.

[0014] The beneficial effects of the present invention are as follows: The big data security analysis method integrating anomaly detection and self-learning described in this invention calculates a dynamic risk score corresponding to a user's current transaction behavior and compares it with a decision threshold. This allows the system to output corresponding decision actions, including transaction interception or security verification. Furthermore, by using time-series operation features, device association features, and the calculated time-series risk score and association risk score, the system can identify anomalies in the user's current transaction behavior and associated accounts. After identifying anomalies, the system can combine the dynamic risk score to make corresponding decision actions, preventing user accounts from being stolen and resulting in losses. Attached Figure Description

[0015] The invention will now be further described with reference to the accompanying drawings.

[0016] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0017] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0018] like Figure 1 As shown in the embodiment of the present invention, the big data security analysis method integrating anomaly detection and self-learning includes the following steps: S1: Real-time acquisition of user operation logs, transaction records, and device information; S2: Based on the data from step S1, generate time-series operation features. Device-related features ; S3: Input the features from step S2 into the integrated model and output a dynamic risk score. : ; in, For feature weights, Time-series risk score For associated risk classification; S4: Based on risk score With decision threshold Based on the comparison results, execute transaction interception or security verification; S5: Update feature weights based on the interception results. , With decision threshold .

[0019] To address potential fraudulent transactions and order-brushing on e-commerce platforms, one embodiment of this invention first acquires user operation logs, including user ID, operation type (such as clicks, adding to cart, and payments), and page dwell time. Then, it collects user transaction details, including order ID, payment amount, payment method, receiving account, and timestamp. For example, if the same receiving account receives more than 50 small payments within 10 seconds, an anomaly is identified based on the transaction details, and targeted anomaly monitoring is implemented. Device information includes the user's device and IP address, as well as the device model. Based on this data, a device-related risk score is calculated. After obtaining the above information, the temporal operation characteristics of the corresponding users are calculated respectively. Device-related features ; Assume the timestamp sequence of user U1001's operations between 9:00 and 9:01 is as follows: ; Based on the calculations, the operation interval sequence is as follows: ; Then based on the calculation ; Then, based on the user's associated devices query, the device association characteristics of the corresponding user can be obtained. ; Based on the calculation, the user's dynamic risk score is obtained. The calculation results are shown below: ; in, , Based on the above... , The calculated feature weights are... According to the system preset, among which, , ; Based on the above calculation results, assuming the calculated dynamic risk score If the risk exceeds the decision threshold, the user's current transaction order will be blocked, and the currently used device and multiple associated accounts will be blocked. Based on the above, a dynamic risk score corresponding to the user's current transaction behavior will be calculated. Then, by comparing it with the decision threshold, the corresponding decision action can be output, including transaction interception or security verification. In addition, through time-series operation features... Device-related features and the calculated time series risk score Related risk classification This can help the system identify anomalies in a user's current trading behavior and related accounts, thereby enabling the system to combine dynamic risk scoring after identifying anomalies. They can make corresponding decisions and actions to prevent user accounts from being stolen and resulting in losses.

[0020] In one embodiment, in S2, the timing operation features The method for obtaining it is as follows: S21: Calculate the standard deviation of user operation intervals as a time-series operation characteristic. ; According to the formula: ; in, The time taken for a single operation, in units of , For the average interval, Number of operations within 1 minute; In S3, time-series risk classification The calculation method is as follows: S31: Based on Calculate time series risk score ; ; in, This is the hidden state of the LSTM. This serves as a benchmark for normal behavior. Number of consecutive operations ,and At that time, forced 0, and trigger device fingerprint collection.

[0021] As described above, the data from the above embodiments will be used for explanation and illustration: Among them, the average operation interval time generated by user U1001's transactions ; The time-series operation characteristics, that is, the standard deviation of the operation interval calculated based on the mean of the operation interval time, are calculated as follows: ; According to the calculation results, ,and In other words, based on the characteristics of time-series operations The calculation determines the timing operation characteristics corresponding to user U1001's transaction at this time. and number of operations It does not trigger constraint rules, that is, "forced". .0, and trigger the automatic operation of "device fingerprint collection"; The calculated timing operation characteristics will then be... In the formula for calculating the input temporal risk score, the calculation of the temporal risk score relies on the LSTM model, which incorporates temporal operational features. The model is input and the corresponding risk score is automatically output. .

[0022] In one embodiment, in S2, the device association feature The method for obtaining it is as follows: S22: Calculate the number of device-related accounts as a device association feature. ; According to the formula: ; in, For device fingerprint, For user-device relationship sets.

[0023] In one embodiment, in step S3, the associated risk is divided... The calculation method is as follows: S32: Based on device association features Calculate the associated risk score ; According to the formula: ; in, For the sigmoid function, For graph convolutional networks; When the new account rate ,and At that time, forced All associated accounts were frozen and cross-platform risk labels were generated.

[0024] As mentioned above, the associated risk score for user U1001 The calculation process is as follows: The following analysis also uses the data from the above examples to classify the associated risks corresponding to user U1001's transactions. The calculations are explained below: First, obtain the device association characteristics of user U1001. Upon investigation, it was found that the user's device is associated with 8 users. At this point, according to the constraint rules, Furthermore, through calculation and analysis, the new account rate of the device used by user U1001 was obtained. According to the constraint rules, the mandatory action will be triggered directly at this point. The operation, and freezing of all associated accounts, namely the 8 associated accounts found above; Then output The calculation results; Based on the calculated time-series risk score And due to the influence of constraint rules, the associated risk is divided. ; Based on the above: ; Among them, feature weights Then according to the system preset, and , ; Based on the calculation results, a dynamic risk score corresponding to user U1001's current trading behavior is obtained. Then the calculated dynamic risk score By comparing the result with a decision threshold, a preset action can be output, such as when the dynamic risk score is... If the decision threshold is exceeded, transaction interception or security verification will be implemented.

[0025] In one embodiment, the method for performing transaction interception or security verification in step S4 is as follows: S41: Extract risk score and decision threshold , Comparison: when At that time, execute transaction interception; when At that time, a security verification is performed.

[0026] In one embodiment, based on the user's time-series risk classification and related risk classification Calculated risk score Then, the user's risk score is compared with the decision threshold. Based on the comparison result, the corresponding action is output. In one embodiment of the present invention, the data from the above embodiments will be used for explanation: When the user's risk score is calculated Then, risk scoring will be performed. Compared with a decision threshold, in one embodiment, the decision threshold includes , ,and Based on the calculated risk score , and decision threshold , Comparison, because If the system detects an error, it will identify and output the action to intercept the transaction. In other words, based on the analysis of user U1001's transaction behavior and based on the risk score... The calculation and risk score Compared with the decision threshold, the transaction of user U1001 at this moment is intercepted. Simultaneously, based on the "new account rate"... ,and At that time, forced The constraint rule of "freezing all associated accounts" applies because of user U1001's associated accounts. And the rate of new accounts Therefore, the associated account was also frozen. As mentioned above, this can prevent user U1001 from engaging in malicious order-brushing behavior on the platform.

[0027] In one embodiment, step S3 further includes: S33: Calculate real-time login location deviation ; According to the formula: ; in, Commonly used coordinates, The coordinates of this login location; S34: When And after logging in When initiating a payment within minutes, a displacement penalty factor is added when calculating the risk score. According to the formula: ; in, As a penalty factor; S35: Based on displacement penalty factor Correct risk score ; ; in, This is the updated risk score.

[0028] Understandably, if the location of the device used for a transaction deviates significantly between two separate transactions, it can be interpreted as a high-risk transaction, possibly involving login from a different location. In such cases, penalties or monitoring of the transaction are necessary. In one embodiment, the data from the previous embodiment will be used again for explanation: Assuming user U1001's account has logged in from a different location and this is detected by the system, calculate the real-time login location deviation of the device used by user U1001: If user U1001 frequently uses coordinates Furthermore, in conjunction with GPS positioning, it was identified that the user's frequently used location was Beijing; According to the login coordinates displayed by the device at this moment, the transaction behavior... In conjunction with GPS positioning, it was determined that the coordinates of the transaction at that moment corresponded to Shanghai; Based on the calculation, the real-time login location deviation Then, according to the constraint rules, when And after logging in When initiating a payment within minutes, a displacement penalty factor is added when calculating the risk score. In this embodiment, due to the calculated real-time login location deviation Significantly greater than 100 In this case, it is necessary to check the user's login status again. If a payment was made within 10:00 AM, and user U1001's login time was 10:02 AM, then the interval would be 2 minutes, which is less than the preset time. If user U1001's login and transaction behaviors simultaneously meet and trigger the constraint conditions, then user U1001's risk score needs to be penalized, specifically including: Calculating the risk score for user U1001 Increase displacement penalty factor According to the formula: ; in, is the displacement sensitivity coefficient, which is the system preset value; Based on the above calculations, user U1001's risk score is... A displacement penalty factor needs to be added. Based on the above embodiment, the risk score for user U1001 can be obtained. According to the formula: ; This can be understood as the risk score of user U1001 after applying the displacement penalty factor. Furthermore, as mentioned above, when a user logs in from a different location and simultaneously triggers constraints, the greater the distance, the larger the user's displacement penalty factor, which in turn leads to a higher updated risk score. If it's merely a login from a different location without making a payment or not being in the preset... Payments made within minutes will not trigger the displacement penalty factor, thus preventing a further increase in the risk score; it is worth noting that when the updated risk score is calculated... Afterwards, if If the value is adjusted, it will be automatically adjusted to the maximum value of 1.0. Based on the above, if an illegal team steals user accounts to make purchases on e-commerce platforms, and the account is logged in from a different location and makes payments in a short period of time, the system will analyze and identify the account, increase the risk score of the account at this moment, and then freeze the account after comparing the decision threshold to prevent the illegal team from stealing the account and causing economic losses.

[0029] In one embodiment, the parameter update in S5 includes: S51: Obtain the false alarm rate Adjust the weights of time-series features; According to the formula: ; in, For the learning rate, and , False alarm rate; S52: According to the updated Automatically calculate the updated ; in, .

[0030] Based on the above, continuing with the data from the previous embodiments, in this embodiment, due to the time-series feature weights... Equipment feature weights Both are preset values, and if both are set to fixed values, it will lead to a distortion of the risk score. In one embodiment of the present invention, it is also necessary to combine the feedback false alarm rate. Adjusting weights for time series Update, and then based on constraints Weights of equipment features To update, based on the above, the data from the above embodiments will be explained again: Assuming the system intercepted a total of 15,230 transactions last week, including cross-regional transactions involving U1001, and 2,186 of these were confirmed as false positives, then the false positive rate would be... ; Based on the calculated false alarm rate Calculate the updated time series feature weights: ; Subsequently, according to the constraints Calculate the updated device association weights : ; Based on the above calculations, the updated temporal feature weights are obtained. and equipment association weight Through dynamically adjusted time-series feature weights and equipment association weight This can enhance the system's ability to identify abnormal behavior, thereby reducing user losses.

[0031] In one embodiment, S5 further includes: S53: Obtain the displacement false alarm rate Update penalty factors ; According to the formula: ; in, This represents the false alarm rate for displacement.

[0032] In one embodiment, continuing with the data from the above embodiments, assuming that the number of transactions triggering displacement penalties in the system is 1850, accounting for 12.15%, of which 428 are confirmed as false alarms, accounting for 23.1% of displacement transactions, then according to the calculation, the displacement false alarm rate is... ; Substitute the adjustment coefficient The calculation is as follows: ; Based on the above calculations, the penalty factor From 0.3 to This reduces the intensity of the displacement penalty. It is understandable that the higher the proportion of false interceptions among the number of transactions that trigger displacement penalties identified by the system, the lower the penalty factor calculated. The purpose is to reduce the intensity of the displacement penalty and avoid false interceptions.

[0033] In one embodiment, in S4, the decision threshold The calculation method is as follows: According to the formula: ; ; in, It is a sign function, and its range of values ​​is ; when , ,but 0.6; when , ,but 0.8; when , ,but 0.7.

[0034] Furthermore, in calculating the time series risk score... Related risk classification After that, it can also be based on time-series risk analysis. Related risk classification The size of the threshold is used to dynamically adjust the decision threshold. This can be understood as the time-order risk score... Greater than the associated risk score When the time sequence risk is higher, the decision threshold should be increased to raise the interception threshold. Less than the associated risk score When the risk level is higher, the decision threshold should be appropriately lowered to reduce the interception threshold. Based on the above, this is done by classifying risks according to time series. Related risk classification The size of the threshold is dynamically adjusted. Compared to a completely fixed threshold, a dynamically adjusted threshold is more adaptable and can avoid false interception.

[0035] As mentioned above, since the decision threshold includes , In one embodiment, by classifying according to time-series risk Related risk classification The size of the threshold dynamically adjusts the decision threshold. Specifically, the adjusted decision threshold, or the threshold adjusted according to the formula, is... ; Based on the time-series risk score calculated in the above embodiments Related risk classification Among them, time-series risk classification And associated risk classification Based on the comparison results, , Therefore, due to the dynamic risk score calculated in the above embodiments... Therefore, based on this judgment, it can be concluded that If this occurs, transaction interception will be executed. Furthermore, in conjunction with the above embodiments, due to associated risk... The operation that forces the transaction to version 1.0 is triggered, and the associated account is frozen. Therefore, the trigger action corresponding to the user's transaction behavior here is to intercept the transaction and freeze the associated account.

[0036] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A big data security analysis method integrating anomaly detection and self-learning, characterized by: Includes the following steps: S1: Real-time acquisition of user operation logs, transaction records, and device information; S2: Based on the data from step S1, generate time-series operation features. Device-related features ; S3: Input the features from step S2 into the integrated model and output a dynamic risk score. : ； in, For feature weights, Time-series risk score For associated risk classification; S4: Based on risk score With decision threshold Based on the comparison results, execute transaction interception or security verification; S5: Update feature weights based on the interception results. , With decision threshold .

2. The big data security analysis method integrating anomaly detection and self-learning as described in claim 1, characterized in that: In S2, the timing operation features The method for obtaining it is as follows: S21: Calculate the standard deviation of user operation intervals as a time-series operation characteristic. ; According to the formula: ； in, The time taken for a single operation, in units of , For the average interval, Number of operations within 1 minute; In S3, time-series risk classification The calculation method is as follows: S31: Based on Calculate time series risk score ; ； in, This is the hidden state of the LSTM. This serves as a benchmark for normal behavior. Number of consecutive operations ,and At that time, forced 0, and trigger device fingerprint collection.

3. The big data security analysis method integrating anomaly detection and self-learning as described in claim 2, characterized in that: In S2, the device association feature The method for obtaining it is as follows: S22: Calculate the number of device-related accounts as a device association feature. ; According to the formula: ； in, For device fingerprint, For user-device relationship sets.

4. The big data security analysis method integrating anomaly detection and self-learning as described in claim 3, characterized in that: In S3, the associated risk classification The calculation method is as follows: S32: Based on device association features Calculate the associated risk score ; According to the formula: ； in, For the sigmoid function, For graph convolutional networks; When the new account rate ,and At that time, forced All associated accounts were frozen and cross-platform risk labels were generated.

5. The big data security analysis method integrating anomaly detection and self-learning as described in claim 4, characterized in that: In S4, the method for performing transaction interception or security verification is as follows: S41: Extract Risk Score and decision threshold , Comparison: when At that time, execute transaction interception; when At that time, a security verification is performed.

6. The big data security analysis method integrating anomaly detection and self-learning as described in claim 5, characterized in that: S3 also includes: S33: Calculate real-time login location deviation ; According to the formula: ； in, Commonly used coordinates, The coordinates of this login location; S34: When And after logging in When initiating a payment within minutes, a displacement penalty factor is added when calculating the risk score. According to the formula: ； in, As a penalty factor; S35: Based on displacement penalty factor Correct risk score ; ； in, This is the updated risk score.

7. The big data security analysis method integrating anomaly detection and self-learning as described in claim 6, characterized in that: The parameter update in S5 includes: S51: Obtain the false alarm rate Adjust the weights of time-series features; According to the formula: ； in, For the learning rate, and , False alarm rate; S52: According to the updated Automatically calculate the updated ; in, .

8. The big data security analysis method integrating anomaly detection and self-learning as described in claim 7, characterized in that: The S5 also includes: S53: Obtain the displacement false alarm rate Update penalty factors ; According to the formula: ； in, This represents the false alarm rate for displacement.

9. The big data security analysis method integrating anomaly detection and self-learning as described in claim 8, characterized in that: In S4, the decision threshold The calculation method is as follows: According to the formula: ； ； in, It is a sign function, and its range of values ​​is ; when , ,but 0.6; when , ,but 0.8; when , ,but 0.7.

Images