Abnormal request determination method and device, and electronic device

Abstract

The present disclosure provides an abnormal request determination method and device and electronic equipment, relates to the technical field of data processing, and particularly relates to the fields of big data, network security and the like. The specific scheme is as follows: network requests corresponding to each feature vector in a request data table are aggregated, and a first request quantity of each feature vector under different request types is determined; according to a plurality of feature values in each feature vector and the first request quantity of each feature vector under different request types, a request total quantity of each feature subvector corresponding to each feature vector and a second request quantity of each feature subvector under different request types are determined; when a ratio of the second request quantity of any feature subvector to the request total quantity is greater than a first threshold value, it is determined that the network request corresponding to the any feature subvector is an abnormal request. The method realizes analysis of network requests from the perspective of various dimensional combinations, avoids missing the mining of abnormal requests under various dimensional combinations, and improves the accuracy of determining abnormal requests.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, particularly to the fields of big data and cybersecurity, and specifically to methods, apparatus and electronic devices for determining abnormal requests. Background Technology

[0002] In recent years, cybersecurity has been a hot topic, and information network security has become a widespread concern. Accurately determining whether a network request is abnormal is crucial for ensuring network security. Summary of the Invention

[0003] This disclosure provides a method and apparatus for determining abnormal requests.

[0004] According to one aspect of this disclosure, a method for determining abnormal requests is provided, comprising:

[0005] Obtain the request data table, which includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0006] Aggregate the network requests corresponding to each feature vector to determine the number of first requests for each feature vector under different request types;

[0007] Based on the number of first requests under different request types for each feature vector and the multiple feature values ​​included in each feature vector, determine the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests under different request types, wherein each feature subvector contains at least one feature value.

[0008] If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than the first threshold, the network request corresponding to any feature sub-vector is determined to be an abnormal request.

[0009] According to another aspect of this disclosure, an apparatus for determining an abnormal request is provided, comprising:

[0010] The acquisition module is used to acquire the request data table, which includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0011] The aggregation module is used to aggregate the network requests corresponding to each feature vector in order to determine the number of first requests for each feature vector under different request types.

[0012] The determination module is used to determine the total number of requests for each feature vector and the number of second requests for each feature subvector under different request types based on the number of first requests for each feature vector under different request types and the multiple feature values ​​included in each feature vector, wherein each feature subvector contains at least one feature value.

[0013] The aforementioned determining module is also used to determine that a network request corresponding to any feature sub-vector is an abnormal request if the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than a first threshold.

[0014] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in the above embodiments.

[0018] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in the above embodiments.

[0019] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the above embodiments.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 A flowchart illustrating a method for determining abnormal requests provided in an embodiment of this disclosure;

[0023] Figure 2 A flowchart illustrating another method for determining abnormal requests provided in an embodiment of this disclosure;

[0024] Figure 3 A flowchart illustrating another method for determining abnormal requests provided in an embodiment of this disclosure;

[0025] Figure 4 A flowchart illustrating another method for determining abnormal requests provided in an embodiment of this disclosure;

[0026] Figure 5 A flowchart illustrating another method for determining abnormal requests provided in an embodiment of this disclosure;

[0027] Figure 6 A flowchart illustrating another method for determining abnormal requests provided in an embodiment of this disclosure;

[0028] Figure 7 A flowchart illustrating an apparatus for determining an abnormal request, provided in an embodiment of this disclosure;

[0029] Figure 8 This is a block diagram of an electronic device used to determine abnormal requests in accordance with embodiments of the present disclosure. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] Big data, or massive data, refers to information that is so large that it cannot be captured, managed, processed, and organized into a more proactive business decision-making process using current mainstream software tools within a reasonable timeframe.

[0032] Network security refers to the ability of a network system to function normally and share resources without any threats or infringements. For a network to function normally, it is essential to ensure that its hardware and software are functioning properly, and then to guarantee the security of data exchange.

[0033] Typically, network requests can be identified as abnormal by using feature values ​​on a fixed dimension. However, due to the complexity of abnormal request scenarios, relying on feature values ​​on a fixed dimension to identify abnormal network requests may miss the discovery of typical abnormal requests under other dimension combinations, leading to inaccurate identification of abnormal requests.

[0034] In this disclosure, feature vectors are split across multiple dimensions to obtain feature sub-vectors under various dimension combinations. The network requests corresponding to each feature sub-vector are then aggregated, enabling analysis of network requests from the perspective of various dimension combinations. This avoids missing any abnormal requests under various dimension combinations and improves the accuracy of identifying abnormal requests.

[0035] The method, apparatus, electronic device, and storage medium for determining abnormal requests according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0036] Figure 1 This is a flowchart illustrating a method for determining abnormal requests provided in an embodiment of this disclosure.

[0037] like Figure 1 As shown, the method includes:

[0038] Step 101: Obtain the request data table, which includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0039] The dimensions can be the IP address (Internet Protocol Address) corresponding to the network request, the client identifier, the account identifier, etc., and this disclosure does not impose any restrictions on them.

[0040] In this disclosure, the request data table can be determined by analyzing network logs. When a user initiates a network request, the request type can be determined based on the business interfaces called by the network request. For example, when a user applies for after-sales service, the request type can be determined as after-sales service; when a user initiates a network request that only calls the login interface, the request type can be determined as login.

[0041] Optionally, the network logs corresponding to different services can be parsed separately, and the network requests corresponding to each service can be stored in different request data tables to determine the request data tables corresponding to each service. Then, based on the request data tables corresponding to each service, the abnormal traffic under each service can be determined.

[0042] Step 102: Aggregate the network requests corresponding to each feature vector to determine the number of first requests for each feature vector under different request types.

[0043] In this disclosure, the number of first requests for each feature vector under different request types can be determined first to reduce the amount of request data, thereby improving the efficiency of identifying abnormal requests.

[0044] In this disclosure, the network requests corresponding to each feature vector can be aggregated to determine the number of first requests for each feature vector under different request types. For example, if a user initiates 5 after-sales requests and 3 login requests using account 1 and client 1, then these 8 network requests are aggregated to determine that the number of first requests for feature vector [account identifier 1, client identifier 1] under the after-sales type is 5, and the number of first requests under the login type is 3.

[0045] Step 103: Based on the number of first requests under different request types for each feature vector and the multiple feature values ​​included in each feature vector, determine the total number of requests for each feature subvector corresponding to each feature subvector and the number of second requests under different request types, wherein each feature subvector contains at least one feature value.

[0046] In this disclosure, each feature vector can be split to determine the feature sub-vectors of each feature vector under various dimensional combinations, and the first request number of each feature vector under different request types can be determined as the request number of each feature sub-vector under the corresponding request type. Then, the request numbers of each feature sub-vector under different request types can be aggregated to determine the second request number of each feature sub-vector under different request types.

[0047] In addition, the total number of requests corresponding to a feature vector can be determined by adding up the number of first requests under the same request type for multiple feature vectors corresponding to a certain feature vector.

[0048] Step 104: If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than the first threshold, determine that the network request corresponding to any feature sub-vector is an abnormal request.

[0049] In this disclosure, when the ratio of the number of second requests corresponding to the feature sub-vector to the total number of requests is greater than the first threshold, it indicates that the user is frequently making the same network requests. Therefore, it can be determined that the network request corresponding to the feature sub-vector is an abnormal request.

[0050] In this disclosure, different request types can correspond to different thresholds. Therefore, based on the distribution of the number of requests under different request types, a first threshold corresponding to each request type can be dynamically determined. Then, if the ratio of the number of second requests to the total number of requests for any feature subvector under a certain request type is greater than the first threshold corresponding to that request type, the network request corresponding to that feature subvector is determined to be an abnormal request. This can improve the accuracy of identifying abnormal requests.

[0051] Optionally, the ratio of the number of second requests to the total number of requests corresponding to each feature subvector under different request types can be determined, and then the first threshold corresponding to each request type can be determined based on the distribution of multiple ratios under each request type.

[0052] Optionally, if the network request corresponding to the feature sub-vector is determined to be an abnormal request, the request data containing the feature sub-vector in the request data table will be marked as abnormal request data. This will facilitate subsequent statistics and data display of abnormal requests.

[0053] In this disclosure, after obtaining a request data table including multiple network requests, the request type corresponding to each network request, and feature vectors, the network requests corresponding to each feature vector can be aggregated to determine the number of first requests for each feature vector under different request types. Then, based on the number of first requests for each feature vector under different request types and the multiple feature values ​​included in each feature vector, the total number of requests for each feature sub-vector containing at least one feature value and the number of second requests under different request types corresponding to each feature vector can be determined. If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than a first threshold, the network request corresponding to any feature sub-vector is determined to be an abnormal request. Therefore, by splitting feature vectors across multiple dimensions to obtain feature sub-vectors under various dimension combinations, and aggregating the network requests corresponding to each feature sub-vector, analysis of network requests from the perspective of various dimension combinations is achieved, thereby avoiding the omission of abnormal requests under various dimension combinations and improving the accuracy of identifying abnormal requests.

[0054] Figure 2 This is a flowchart illustrating a method for determining abnormal requests provided in an embodiment of this disclosure.

[0055] like Figure 2 As shown, the method includes:

[0056] Step 201: Obtain the request data table, which includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0057] Step 202: Aggregate the network requests corresponding to each feature vector to determine the number of first requests for each feature vector under different request types.

[0058] The specific implementation process of steps 201-202 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0059] Step 203: Determine the feature sub-vectors corresponding to each feature vector based on the dimensions corresponding to the feature values ​​included in each feature vector.

[0060] In this disclosure, when multiple feature values ​​correspond to dimensions that are semantically related, the number of network requests corresponding to those feature values ​​alone cannot accurately determine whether the network requests are abnormal requests. Therefore, to improve the accuracy of identifying abnormal requests, semantic recognition can be performed on the dimensions corresponding to each feature value to determine the semantic type of each feature value. Then, each feature vector can be split according to the semantic types of the feature values ​​included in each feature vector to generate feature sub-vectors, where the semantic types of the feature values ​​contained in the feature sub-vectors are different.

[0061] Optionally, reference dimensions with no correlation can be configured via a configuration file. This allows the feature values ​​corresponding to the reference dimension in each feature vector to be combined to generate feature sub-vectors. This ensures that the feature values ​​in the feature sub-vectors are uncorrelated, thereby improving the accuracy of identifying abnormal requests.

[0062] Step 204: When any feature subvector corresponds to multiple feature vectors, aggregate the number of first requests under the same request type for each feature vector to determine the number of second requests under the same request type for any feature subvector.

[0063] In this disclosure, when any feature subvector corresponds to multiple feature vectors, the number of first requests under the same request type for each feature vector can be aggregated to determine the number of second requests under the same request type for that feature subvector. Then, based on the number of second requests for each feature subvector, it can be determined whether the network request corresponding to each feature subvector is an anomalous request, thereby avoiding the omission of anomalous requests under various dimensional combinations and improving the accuracy of anomalous request identification.

[0064] For example, suppose the feature sub-vector [IP] corresponds to two feature vectors, namely feature vector 1 [IP, account identifier 1] and feature vector 2 [IP, account identifier 2]. Feature vector 1 corresponds to 5 first requests under the after-sales type, and feature vector 2 corresponds to 4 first requests under the after-sales type. Then, by aggregating the first request counts of the above two feature vectors under the after-sales type, we determine that the second request count of feature sub-vector [IP] under the after-sales type is 9.

[0065] Step 205: Aggregate the number of second requests for any feature subvector under each request type to determine the total number of requests corresponding to any feature subvector.

[0066] In this disclosure, the number of second requests for any feature subvector under each request type can be aggregated to determine the total number of requests corresponding to that feature subvector.

[0067] For example, assuming that the number of second requests corresponding to the feature subvector [IP] is 9 under the after-sales type and 8 under the login type, then the number of second requests corresponding to the feature subvector [IP] under the after-sales type and the login type is aggregated to determine that the total number of requests corresponding to the feature subvector [IP] is 17.

[0068] Step 206: If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than the first threshold, determine that the network request corresponding to any feature sub-vector is an abnormal request.

[0069] The specific implementation process of step 206 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0070] In this disclosure, feature vectors are split across multiple dimensions to obtain feature sub-vectors under various dimension combinations. The network requests corresponding to each feature sub-vector are then aggregated, enabling analysis of network requests from the perspective of various dimension combinations. This avoids missing any abnormal requests under various dimension combinations and improves the accuracy of identifying abnormal requests.

[0071] Figure 3 This is a flowchart illustrating a method for determining abnormal requests provided in an embodiment of this disclosure.

[0072] like Figure 3 As shown, the method includes:

[0073] Step 301: Obtain the request data table, which includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0074] Step 302: Aggregate the network requests corresponding to each feature vector to determine the number of first requests for each feature vector under different request types.

[0075] Step 303: Based on the number of first requests under different request types for each feature vector and the multiple feature values ​​included in each feature vector, determine the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests under different request types, wherein each feature subvector contains at least one feature value.

[0076] Step 304: If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than the first threshold, determine that the network request corresponding to any feature sub-vector is an abnormal request.

[0077] The specific implementation process of steps 301-304 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0078] Step 305: Aggregate multiple network requests based on the request time corresponding to each network request to determine the number of fourth requests corresponding to each feature vector within each first preset time interval.

[0079] In this disclosure, the number of network requests initiated by users within each time interval typically follows a pattern. For example, the number of network requests initiated by users within each time interval is evenly distributed. If the number of network requests initiated by users increases within a certain time interval, then the network request may be an abnormal request. Therefore, multiple network requests can be aggregated based on the request time corresponding to each network request to determine the number of fourth requests corresponding to each feature vector within each first preset time interval.

[0080] For example, suppose feature vector 1 [IP, account identifier 1] has 3 corresponding network requests between 10:00 and 10:10, and 2 corresponding network requests between 10:10 and 10:20. Then, the number of fourth requests corresponding to feature vector 1 between 10:00 and 10:10 is 3, and the number of fourth requests corresponding to feature vector 1 between 10:10 and 10:20 is 2.

[0081] Step 306: When any feature subvector corresponds to multiple feature vectors within the same first preset time interval, the number of fourth requests corresponding to the multiple feature vectors is aggregated to determine the number of fifth requests corresponding to any feature subvector within the same first preset time interval.

[0082] In this disclosure, when a certain feature subvector corresponds to multiple feature vectors within the same first preset time interval, the number of fourth requests corresponding to the multiple feature vectors can be aggregated to determine the number of fifth requests corresponding to the feature subvector within the same first preset time interval. Then, based on the number of fifth requests corresponding to the feature subvector in each first preset time interval, it can be determined whether the network request corresponding to the feature subvector is an abnormal request.

[0083] For example, suppose the feature subvector [IP] corresponds to two feature vectors: feature vector 1 [IP, account identifier 1] and feature vector 2 [IP, account identifier 2]. Feature vector 1 has 5 fourth requests between 10:00 and 10:10, and feature vector 2 has 4 fourth requests between 10:00 and 10:10. Then, by aggregating the fourth request counts corresponding to feature vectors 1 and 2 respectively, we can determine that the fifth request count corresponding to feature subvector [IP] between 10:00 and 10:10 is 9.

[0084] Step 307: Determine whether the network request corresponding to each feature subvector is an abnormal request based on the difference between the number of fifth requests corresponding to each feature subvector in each first preset time interval.

[0085] In this disclosure, when the difference between the number of fifth requests corresponding to a certain feature vector in each of the first preset time intervals is greater than a preset threshold, it indicates that the network request corresponding to the feature vector is an abnormal request. When the difference between the number of fifth requests corresponding to a certain feature vector in each of the first preset time intervals is less than the preset threshold, the number of network requests corresponding to the feature vector is relatively stable in each of the first preset time intervals. Therefore, it can be determined that the network request corresponding to the feature vector is not an abnormal request, thereby improving the accuracy of identifying abnormal requests.

[0086] Optionally, the number of fifth requests corresponding to a certain feature sub-vector within each of the first preset time intervals can be sorted according to the order of the time intervals to determine the trend data corresponding to the feature sub-vector. This trend data can then be input into a preset trend anomaly detection network model to determine whether the network request corresponding to the feature sub-vector is an anomalous request. The trend anomaly detection network model can be used to determine whether network requests whose numbers do not exhibit a uniform distribution within each of the first preset time intervals are anomalous requests.

[0087] In this disclosure, feature vectors are split across multiple dimensions to obtain feature sub-vectors under various dimension combinations. The network requests corresponding to each feature sub-vector are then aggregated, enabling analysis of network requests from the perspective of various dimension combinations. This avoids missing any abnormal requests under various dimension combinations and improves the accuracy of identifying abnormal requests.

[0088] Figure 4 This is a flowchart illustrating a method for determining abnormal requests provided in an embodiment of this disclosure.

[0089] like Figure 4 As shown, the method includes:

[0090] Step 401: Obtain the request data table, which includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0091] Step 402: Aggregate the network requests corresponding to each feature vector to determine the number of first requests for each feature vector under different request types.

[0092] The specific implementation process of steps 401-402 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0093] Step 403: Determine the number of third requests for each feature value under different request types based on the number of first requests for each feature value in the feature vector under different request types.

[0094] In this disclosure, each feature vector in the request data table may contain many feature values. When the amount of data in the network request corresponding to a certain feature value is small, it can be determined that the network request corresponding to that feature value is not an abnormal request. To reduce the amount of computation, the number of third requests for each feature value under different request types can be determined based on the number of first requests for each feature vector under different request types. Then, the feature values ​​are masked based on the number of third requests.

[0095] Step 404: If the number of third requests corresponding to any feature value is less than the second threshold, replace any feature value in each feature vector with a preset character.

[0096] In this disclosure, when the number of third requests corresponding to a certain feature value is less than a second threshold, it indicates that the network request corresponding to that feature value is not an abnormal request. Therefore, the feature value in each feature vector can be replaced with a preset character. This reduces the number of feature values ​​and further improves the efficiency of identifying abnormal requests.

[0097] Step 405: Based on the number of first requests under different request types for each feature vector and the multiple feature values ​​included in each feature vector, determine the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests under different request types, wherein each feature subvector contains at least one feature value.

[0098] Step 406: If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than the first threshold, determine that the network request corresponding to any feature sub-vector is an abnormal request.

[0099] The specific implementation process of steps 405-406 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0100] In this disclosure, feature vectors are split across multiple dimensions to obtain feature sub-vectors under various dimension combinations. The network requests corresponding to each feature sub-vector are then aggregated, enabling analysis of network requests from the perspective of various dimension combinations. This avoids missing any abnormal requests under various dimension combinations and improves the accuracy of identifying abnormal requests.

[0101] Figure 5 This is a flowchart illustrating a method for determining abnormal requests provided in an embodiment of this disclosure.

[0102] like Figure 5 As shown, the method includes:

[0103] Step 501: Obtain the request data table, which includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0104] Step 502: Aggregate the network requests corresponding to each feature vector to determine the number of first requests for each feature vector under different request types.

[0105] Step 503: Based on the number of first requests under different request types for each feature vector and the multiple feature values ​​included in each feature vector, determine the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests under different request types, wherein each feature subvector contains at least one feature value.

[0106] Step 504: If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than the first threshold, determine that the network request corresponding to any feature sub-vector is an abnormal request.

[0107] Step 505: Aggregate multiple network requests based on the request time corresponding to each network request to determine the number of fourth requests corresponding to each feature vector within each first preset time interval.

[0108] The specific implementation process of steps 501-505 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0109] Step 506: Determine the number of sixth requests corresponding to each feature value within each second preset time interval based on the number of fourth requests corresponding to the feature vector where each feature value is located within each second preset time interval.

[0110] In this disclosure, each feature vector in the request data table may contain many feature values. When the amount of data in the network request corresponding to a feature value is small, the network request corresponding to that feature value is not an abnormal request. To reduce the amount of computation, the number of sixth requests corresponding to each feature value in each second preset time interval can be determined based on the number of fourth requests corresponding to the feature vector containing each feature value in each second preset time interval. Then, the feature values ​​are masked based on the number of sixth requests.

[0111] Step 507: If the number of sixth requests corresponding to any feature value is less than the third threshold, replace any feature value in each feature vector with a preset character.

[0112] In this disclosure, when the number of sixth requests corresponding to a certain feature value is less than a third threshold, it indicates that the network request corresponding to that feature value is not an abnormal request. Therefore, the feature value in each feature vector can be replaced with a preset character. This reduces the number of feature values ​​and further improves the efficiency of identifying abnormal requests.

[0113] Step 508: When any feature subvector corresponds to multiple feature vectors within the same first preset time interval, the number of fourth requests corresponding to the multiple feature vectors is aggregated to determine the number of fifth requests corresponding to any feature subvector within the same first preset time interval.

[0114] Step 509: Determine whether the network request corresponding to each feature sub-vector is an abnormal request based on the difference between the number of fifth requests corresponding to each feature sub-vector in each first preset time interval.

[0115] The specific implementation process of steps 508-509 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0116] In this disclosure, feature vectors are split across multiple dimensions to obtain feature sub-vectors under various dimension combinations. The network requests corresponding to each feature sub-vector are then aggregated, enabling analysis of network requests from the perspective of various dimension combinations. This avoids missing any abnormal requests under various dimension combinations and improves the accuracy of identifying abnormal requests.

[0117] Figure 6 This is a flowchart illustrating a method for determining abnormal requests provided in an embodiment of this disclosure.

[0118] like Figure 6 As shown, the method includes:

[0119] Step 601: Obtain the request data table, which includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0120] Step 602: Aggregate the network requests corresponding to each feature vector to determine the number of first requests for each feature vector under different request types.

[0121] Step 603: Based on the number of first requests under different request types for each feature vector and the multiple feature values ​​included in each feature vector, determine the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests under different request types, wherein each feature subvector contains at least one feature value.

[0122] Step 604: If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than the first threshold, determine that the network request corresponding to any feature sub-vector is an abnormal request.

[0123] The specific implementation process of steps 601-604 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0124] Step 605: Determine the number of different second feature values ​​corresponding to each first feature value based on the first feature value corresponding to the first preset dimension and the second feature value corresponding to the second preset dimension in each feature vector.

[0125] In this disclosure, network requests frequently initiated by a user using the same account on different IPs are considered abnormal requests. Therefore, to more accurately identify abnormal requests, the number of different second feature values ​​corresponding to each first feature value can be determined based on the first feature value corresponding to the first preset dimension and the second feature value corresponding to the second preset dimension in each feature vector. Then, the network request corresponding to each first feature value can be determined as an abnormal request based on the number of different second feature values ​​corresponding to each first feature value.

[0126] In addition, the first and second preset dimensions can be configured by the user according to their needs.

[0127] Step 606: If the number of different second feature values ​​corresponding to any first feature value is greater than the fourth threshold, determine that the network request corresponding to any first feature value is an abnormal request.

[0128] In this disclosure, when the number of different second feature values ​​corresponding to the first feature value is greater than the fourth threshold, it indicates that the user frequently initiates network requests. Therefore, the network request corresponding to the first feature value can be determined as an abnormal request, thereby improving the accuracy of determining abnormal requests.

[0129] For example, if the first characteristic value is account identifier 1, and account identifier 1 corresponds to 1000 IPs, then the network request corresponding to account identifier 1 is an abnormal request.

[0130] In this disclosure, feature vectors are split across multiple dimensions to obtain feature sub-vectors under various dimension combinations. The network requests corresponding to each feature sub-vector are then aggregated, enabling analysis of network requests from the perspective of various dimension combinations. This avoids missing any abnormal requests under various dimension combinations and improves the accuracy of identifying abnormal requests.

[0131] To implement the above embodiments, this disclosure also proposes an apparatus for determining abnormal requests. Figure 7 This is a schematic diagram of a device for determining abnormal requests provided in an embodiment of this disclosure.

[0132] like Figure 7 As shown, the device 700 for determining the abnormal request includes: an acquisition module 710, an aggregation module 720, and a determination module 730.

[0133] The acquisition module 710 is used to acquire a request data table, wherein the request data table includes multiple network requests, the request type and feature vector corresponding to each network request, and the feature vector includes the feature values ​​of the network request in multiple dimensions.

[0134] The aggregation module 720 is used to aggregate the network requests corresponding to each feature vector in order to determine the number of first requests for each feature vector under different request types.

[0135] The determining module 730 is used to determine the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests for each feature subvector under different request types based on the number of first requests for each feature vector under different request types and the multiple feature values ​​included in each feature vector, wherein each feature subvector contains at least one feature value.

[0136] The aforementioned determining module 730 is further configured to determine that a network request corresponding to any feature sub-vector is an abnormal request if the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than a first threshold.

[0137] In one possible implementation of this disclosure, the determining module 730 is used to:

[0138] Based on the dimension corresponding to the feature values ​​included in each feature vector, determine the feature sub-vector corresponding to each feature vector;

[0139] When any feature vector corresponds to multiple feature vectors, the number of first requests under the same request type of the multiple feature vectors is aggregated to determine the number of second requests under the same request type of any feature vector;

[0140] Aggregate the number of second requests for any feature subvector under each request type to determine the total number of requests corresponding to any feature subvector.

[0141] In one possible implementation of this disclosure, the determining module 730 is used to:

[0142] Semantic recognition is performed on the dimension corresponding to each feature value to determine the semantic type corresponding to each feature value;

[0143] Each feature vector is split into feature sub-vectors based on the semantic type of the feature values ​​included in each feature vector. The feature sub-vectors contain feature values ​​with different semantic types.

[0144] In one possible implementation of this disclosure, the determining module 730 is used to:

[0145] Obtain the configuration file, which includes reference dimensions;

[0146] The feature values ​​corresponding to the reference dimension included in each feature vector are combined to generate feature sub-vectors.

[0147] In one possible implementation of this disclosure, the determining module 730 is further configured to:

[0148] Based on the number of first requests for each feature value under different request types in the feature vector containing each feature value, determine the number of third requests for each feature value under different request types.

[0149] The above-mentioned device also includes:

[0150] The replacement module is used to replace any feature value in each feature vector with a preset character when the number of third requests corresponding to any feature value is less than a second threshold.

[0151] In one possible implementation of this disclosure, the aggregation module 720 is further configured to:

[0152] Based on the request time corresponding to each network request, multiple network requests are aggregated to determine the number of fourth requests corresponding to each feature vector within each first preset time interval;

[0153] In the case where any feature subvector corresponds to multiple feature vectors within the same first preset time interval, the number of fourth requests corresponding to the multiple feature vectors is aggregated to determine the number of fifth requests corresponding to any feature subvector within the same first preset time interval.

[0154] The aforementioned determining module 730 is used to determine whether the network request corresponding to each feature subvector is an abnormal request based on the difference between the number of fifth requests corresponding to each feature subvector in each first preset time interval.

[0155] In one possible implementation of this disclosure, the determining module 720 is further configured to:

[0156] Based on the number of fourth requests corresponding to the feature vector where each feature value is located within each second preset time interval, determine the number of sixth requests corresponding to each feature value within each second preset time interval.

[0157] The aforementioned replacement module is also used to replace any feature value in each feature vector with a preset character when the number of sixth requests corresponding to any feature value is less than the third threshold.

[0158] In one possible implementation of this disclosure, the determining module 720 is further configured to:

[0159] Based on the first feature value corresponding to the first preset dimension and the second feature value corresponding to the second preset dimension in each feature vector, determine the number of different second feature values ​​corresponding to each first feature value;

[0160] If the number of different second feature values ​​corresponding to any first feature value is greater than the fourth threshold, the network request corresponding to any first feature value is determined to be an abnormal request.

[0161] In one possible implementation of this disclosure, the determining module 720 is further configured to:

[0162] Determine the ratio of the number of second requests corresponding to each feature subvector to the total number of requests under different request types;

[0163] Based on the distribution of multiple ratios for each request type, determine the first threshold corresponding to each request type.

[0164] In one possible implementation of the embodiments disclosed herein, the following is also included:

[0165] The annotation module is used to annotate request data containing feature subvectors in the request data table as abnormal request data when it is determined that the network request corresponding to the feature subvector is an abnormal request.

[0166] It should be noted that the explanation of the aforementioned method for determining abnormal requests also applies to the apparatus of this embodiment, and therefore will not be repeated here.

[0167] In this disclosure, after obtaining a request data table including multiple network requests, the request type corresponding to each network request, and feature vectors, the network requests corresponding to each feature vector can be aggregated to determine the number of first requests for each feature vector under different request types. Then, based on the number of first requests for each feature vector under different request types and the multiple feature values ​​included in each feature vector, the total number of requests for each feature sub-vector containing at least one feature value and the number of second requests under different request types corresponding to each feature vector can be determined. If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than a first threshold, the network request corresponding to any feature sub-vector is determined to be an abnormal request. Therefore, by splitting feature vectors across multiple dimensions to obtain feature sub-vectors under various dimension combinations, and aggregating the network requests corresponding to each feature sub-vector, analysis of network requests from the perspective of various dimension combinations is achieved, thereby avoiding the omission of abnormal requests under various dimension combinations and improving the accuracy of identifying abnormal requests.

[0168] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0169] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0170] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 802 or a computer program loaded from storage unit 808 into RAM (Random Access Memory) 803. RAM 803 can also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. I / O (Input / Output) interface 805 is also connected to bus 804.

[0171] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0172] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the method for determining an abnormal request. For example, in some embodiments, the method for determining an abnormal request may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the method for determining an abnormal request described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform a method for determining an exception request by any other suitable means (e.g., by means of firmware).

[0173] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0175] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0177] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0178] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers integrated with blockchain technology.

[0179] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when an instruction processor in the computer program product is executed, performs the method for determining abnormal requests as described in the above embodiments of this disclosure.

[0180] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0181] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining an abnormal request, the method comprising: Obtain a request data table, wherein the request data table includes multiple network requests, a request type and a feature vector corresponding to each network request, and the feature vector includes feature values ​​of the network request in multiple dimensions; Aggregate the network requests corresponding to each feature vector to determine the first number of requests for each feature vector under different request types; Based on the number of first requests under different request types for each feature vector and the multiple feature values ​​included in each feature vector, the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests under different request types are determined, wherein each feature subvector contains at least one feature value. If the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than a first threshold, the network request corresponding to any feature sub-vector is determined to be an abnormal request. The method further includes, before determining the total number of requests for each feature subvector corresponding to each feature vector and the second number of requests for each feature vector under different request types based on the first number of requests for each feature vector under different request types and the multiple feature values ​​included in each feature vector: Based on the number of first requests for each feature value under different request types in the feature vector containing each feature value, determine the number of third requests for each feature value under different request types. If the number of third requests corresponding to any feature value is less than the second threshold, replace any feature value in each feature vector with a preset character.

2. The method as described in claim 1, wherein, The step of determining the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests for each feature vector under different request types, based on the first request quantity of each feature vector under different request types and the multiple feature values ​​included in each feature vector, includes: Based on the dimension corresponding to the feature value included in each feature vector, determine the feature sub-vector corresponding to each feature vector; In the case where any feature subvector corresponds to multiple feature vectors, the number of first requests under the same request type of the multiple feature vectors is aggregated to determine the number of second requests under the same request type of any feature subvector. The number of second requests for each feature subvector under each request type is aggregated to determine the total number of requests corresponding to each feature subvector.

3. The method as described in claim 2, wherein, The step of determining the feature sub-vector corresponding to each feature vector based on the dimension corresponding to the feature values ​​included in each feature vector includes: Semantic recognition is performed on the dimension corresponding to each feature value to determine the semantic type corresponding to each feature value; Each feature vector is split according to the semantic type corresponding to the feature value included in each feature vector to generate the feature sub-vector, wherein the feature values ​​included in the feature sub-vector correspond to different semantic types.

4. The method as described in claim 3, wherein, The step of splitting each feature vector according to the semantic type corresponding to the feature values ​​included in each feature vector to generate the feature sub-vector includes: Obtain the configuration file, wherein the configuration file includes reference dimensions; The feature values ​​corresponding to the reference dimension included in each feature vector are combined to generate the feature sub-vector.

5. The method as described in claim 1, wherein, The method further includes: Based on the request time corresponding to each network request, the multiple network requests are aggregated to determine the number of fourth requests corresponding to each feature vector within each first preset time interval; In the case where any feature subvector corresponds to multiple feature vectors within the same first preset time interval, the number of fourth requests corresponding to the multiple feature vectors is aggregated to determine the number of fifth requests corresponding to any feature subvector within the same first preset time interval. Based on the difference between the number of fifth requests corresponding to each feature subvector within each first preset time interval, it is determined whether the network request corresponding to each feature subvector is an abnormal request.

6. The method of claim 5, wherein, Before aggregating the fourth request counts corresponding to the multiple feature vectors, the method further includes: Based on the number of fourth requests corresponding to the feature vector where each feature value is located within each second preset time interval, determine the number of sixth requests corresponding to each feature value within each second preset time interval; If the number of sixth requests corresponding to any feature value is less than the third threshold, replace any feature value in each feature vector with a preset character.

7. The method of claim 1, wherein, Also includes: Based on the first feature value corresponding to the first preset dimension and the second feature value corresponding to the second preset dimension in each feature vector, determine the number of different second feature values ​​corresponding to each first feature value; If the number of different second feature values ​​corresponding to any first feature value is greater than the fourth threshold, the network request corresponding to any first feature value is determined to be an abnormal request.

8. The method of claim 1, wherein, Before determining that a network request corresponding to any feature sub-vector is an abnormal request when the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than a first threshold, the method further includes: Determine the ratio of the number of second requests corresponding to each feature subvector under different request types to the total number of requests; Based on the distribution of multiple ratios under each request type, a first threshold corresponding to each request type is determined.

9. The method according to any one of claims 1-8, wherein, Also includes: If it is determined that the network request corresponding to the feature sub-vector is an abnormal request, the request data containing the feature sub-vector in the request data table is marked as abnormal request data.

10. An apparatus for determining an abnormal request, the apparatus comprising: The acquisition module is used to acquire a request data table, wherein the request data table includes multiple network requests, a request type and a feature vector corresponding to each network request, and the feature vector includes feature values ​​of the network request in multiple dimensions. An aggregation module is used to aggregate the network requests corresponding to each feature vector to determine the first number of requests for each feature vector under different request types. The determining module is used to determine the total number of requests for each feature subvector corresponding to each feature vector and the number of second requests for each feature subvector under different request types based on the first number of requests for each feature vector under different request types and the multiple feature values ​​included in each feature vector, wherein each feature subvector contains at least one feature value; The determining module is further configured to determine that the network request corresponding to any feature sub-vector is an abnormal request if the ratio of the number of second requests corresponding to any feature sub-vector to the total number of requests is greater than a first threshold. The determining module is further configured to: Based on the number of first requests for each feature value under different request types in the feature vector containing each feature value, determine the number of third requests for each feature value under different request types. The device further includes: The replacement module is used to replace any feature value in each feature vector with a preset character when the number of third requests corresponding to any feature value is less than a second threshold.

11. The apparatus of claim 10, wherein, The determining module is used for: Based on the dimension corresponding to the feature value included in each feature vector, determine the feature sub-vector corresponding to each feature vector; In the case where any feature subvector corresponds to multiple feature vectors, the number of first requests under the same request type of the multiple feature vectors is aggregated to determine the number of second requests under the same request type of any feature subvector. The number of second requests for each feature subvector under each request type is aggregated to determine the total number of requests corresponding to each feature subvector.

12. The apparatus of claim 11, wherein, The determining module is used for: Semantic recognition is performed on the dimension corresponding to each feature value to determine the semantic type corresponding to each feature value; Each feature vector is split according to the semantic type corresponding to the feature value included in each feature vector to generate the feature sub-vector, wherein the feature values ​​included in the feature sub-vector correspond to different semantic types.

13. The apparatus of claim 12, wherein, The determining module is used for: Obtain the configuration file, wherein the configuration file includes reference dimensions; The feature values ​​corresponding to the reference dimension included in each feature vector are combined to generate the feature sub-vector.

14. The apparatus of claim 10, wherein, The aggregation module is also used for: Based on the request time corresponding to each network request, the multiple network requests are aggregated to determine the number of fourth requests corresponding to each feature vector within each first preset time interval; In the case where any feature subvector corresponds to multiple feature vectors within the same first preset time interval, the number of fourth requests corresponding to the multiple feature vectors is aggregated to determine the number of fifth requests corresponding to any feature subvector within the same first preset time interval. The determining module is used to determine whether the network request corresponding to each feature subvector is an abnormal request based on the difference between the number of fifth requests corresponding to each feature subvector in each first preset time interval.

15. The apparatus of claim 10, wherein, The determining module is further configured to: Based on the number of fourth requests corresponding to the feature vector where each feature value is located within each second preset time interval, determine the number of sixth requests corresponding to each feature value within each second preset time interval; The replacement module is further configured to replace any feature value in each feature vector with a preset character when the number of sixth requests corresponding to any feature value is less than a third threshold.

16. The apparatus of claim 10, wherein, The determining module is further configured to: Based on the first feature value corresponding to the first preset dimension and the second feature value corresponding to the second preset dimension in each feature vector, determine the number of different second feature values ​​corresponding to each first feature value; If the number of different second feature values ​​corresponding to any first feature value is greater than the fourth threshold, the network request corresponding to any first feature value is determined to be an abnormal request.

17. The apparatus of claim 10, wherein, The determining module is further configured to: Determine the ratio of the number of second requests corresponding to each feature subvector under different request types to the total number of requests; Based on the distribution of multiple ratios under each request type, a first threshold corresponding to each request type is determined.

18. The apparatus according to any one of claims 10-17, wherein, Also includes: The annotation module is used to annotate the request data containing the feature sub-vector as abnormal request data when it is determined that the network request corresponding to the feature sub-vector is an abnormal request.

19. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

21. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-9.

Images