Abnormal request detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the invention disclose an abnormal request detection method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining target payload information in a target access request; determining a target cache primary key according to the target effective load information; according to the target cache primary key, a target detection result corresponding to the target cache primary key is inquired from a cache, the corresponding relation between the cache primary key and the detection result is cached in the cache, and the detection result is obtained from a large language model in advance according to effective load information corresponding to the cache primary key; the detection result comprises a normal request or an abnormal request. According to the embodiment of the invention, the abnormal request detection efficiency can be improved, and the response speed of the detection engine is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of network security technology, and in particular, to an abnormal request detection method, device, electronic device and storage medium. Background Art

[0002] With the development of Internet technology, Web applications have become an important part of people's daily life and business operations, which makes Web applications a major target of attacks. Web applications face a variety of security threats, including SQL (Structured Query Language) injection, Cross Site Scripting (XSS), Cross-Site Request Forgery (CSRF), file inclusion, etc. WAF (Web Application Firewall) can provide special protection against these attacks. However, with the advancement of hacker technology, attacks have become more complex and hidden, and traditional security defense measures are difficult to deal with.

[0003] In the prior art, there are two common methods for WAF detection engines: one is to write a grammatical semantic parser based on artificial feature engineering, manually design and extract grammatical and semantic features, and then connect machine learning classifiers such as LR (Logistic Regression) in series to implement a detection engine for a certain type of language of WAF; the other is to analyze attack samples and manually design multiple regular rules to intercept attacks in different scenarios.

[0004] The method of manually designing regular rules has the following disadvantages: if the regular rules are too complex, the readability is poor and the maintenance is difficult; the regular rules are lagging and can only be designed based on existing sample patterns, and are not predictive.

[0005] Manually extracting semantic features in conjunction with machine learning classifiers has a certain degree of context awareness and is more accurate in extracting features for specific languages. However, this method also has the following disadvantages: the development cycle is too long, the threshold is high, and the design and implementation of a grammar parser is difficult and cumbersome; each engine can only implement attack detection for one language, which is too targeted and has poor generalization; it requires high security experience for personnel, and the features extracted by manual design greatly affect the engine's detection capabilities.

[0006] In contrast, the Large Language Model (LLM) does have the ability to deeply understand complex language and behavior patterns, and can identify different types of network attacks in a generalized manner, which is beyond the capabilities of traditional methods such as regular expressions. However, if the large model interface is directly introduced for synchronous traffic detection, there will be problems such as low detection efficiency and slow response speed of the detection engine. Summary of the invention

[0007] The embodiments of the present application provide a method, device, electronic device and storage medium for detecting abnormal requests, which are helpful to improve detection efficiency and increase the response speed of the detection engine.

[0008] In order to solve the above problems, in a first aspect, an embodiment of the present application provides a method for detecting abnormal requests, including:

[0009] Obtain target payload information in the target access request;

[0010] Determining a target cache primary key according to the target payload information;

[0011] According to the target cache primary key, a target detection result corresponding to the target cache primary key is queried from the cache, wherein the cache caches a correspondence between the cache primary key and the detection result, wherein the detection result is obtained from a large language model in advance based on the payload information corresponding to the cache primary key, and the detection result includes a normal request or an abnormal request.

[0012] In a second aspect, an embodiment of the present application provides an abnormal request detection device, including:

[0013] A target payload acquisition module, used to acquire target payload information in a target access request;

[0014] A target cache primary key determination module, used to determine a target cache primary key according to the target payload information;

[0015] A detection result query module is used to query the target detection result corresponding to the target cache primary key from the cache according to the target cache primary key. The cache caches the correspondence between the cache primary key and the detection result. The detection result is obtained from the large language model in advance based on the payload information corresponding to the cache primary key. The detection result includes a normal request or an abnormal request.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the abnormal request detection method described in the embodiment of the present application when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the abnormal request detection method disclosed in the embodiment of the present application are performed.

[0018] The abnormal request detection method, device, electronic device and storage medium provided in the embodiments of the present application obtain the payload information in the target access request, determine the target cache primary key according to the target payload information, and query the target detection result corresponding to the target cache primary key from the cache according to the target cache primary key. The cache caches the correspondence between the cache primary key and the detection result. The detection result is obtained from the large language model in advance according to the payload information corresponding to the cache primary key. In this way, there is no need to request the large language model in real time for real-time target access requests, but to query the pre-stored data from the cache, which can improve the detection efficiency of abnormal requests and the response speed of the detection engine. Moreover, compared with manually involving regular rules and manually extracting semantic features in conjunction with machine learning classifiers, it can generalize and detect network attacks of different languages ​​and different types. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 is a flow chart of an abnormal request detection method provided by an embodiment of the present application;

[0021] Figure 2 is a flow chart of determining a target cache primary key according to target payload information in an embodiment of the present application;

[0022] Figure 3 This is a flowchart of cache construction based on a large language model in an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of the overall process of generating cache data based on high-performance cache and asynchronous large language model in an embodiment of the present application;

[0024] Figure 5 It is a flowchart of WAF real-time detection in an embodiment of the present application;

[0025] Figure 6 It is a structural diagram of an abnormal request detection device provided in an embodiment of the present application;

[0026] Figure 7It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] Before introducing the technical solution of the embodiment of the present application in detail, the terms involved in the embodiment of the present application are explained as follows:

[0029] Asynchronous Caching: Asynchronous Caching is a caching mechanism that allows data read and write operations to be performed asynchronously in the background without blocking the main thread of execution. This means that the system can continue to process other tasks while the cache operation is completed in the background. Asynchronous Caching is often used to improve the performance and responsiveness of applications because it reduces the time spent waiting for cache read and write operations to complete. In high-load or big data processing scenarios, Asynchronous Caching can effectively improve the system's processing capabilities and user experience.

[0030] Large Language Model: Large language models, such as the GPT (Generative Pre-trained Transformer) series, are deep learning models trained with large-scale datasets, designed to understand and generate human language. These models typically have billions or even trillions of parameters and are capable of handling complex language tasks, including text generation, translation, summarization, question-answering, etc. Large language models have powerful language understanding and generation capabilities by learning language regularities and patterns from large amounts of text data, and are widely used in the field of Natural Language Processing (NLP).

[0031] The technical solution of the embodiments of the present application is described in detail below.

[0032] The embodiment of the present application is to solve the problem of the capability limitations of the existing conventional regular and machine learning classifier methods and the slow response speed of directly introducing a large language model for detection, and proposes a real-time detection technology that combines high-performance caching and asynchronous large language models.

[0033] In order to solve the limitations of the existing methods, the embodiment of the present application proposes a real-time detection technology that combines high-performance caching and asynchronous large language models. The core idea is to use massive traffic data to asynchronously query the large language model to build the cached key-value pair cache data. In real-time detection, the high-performance cache is queried first. If there is no cache hit, the large language model is asynchronously queried and the result is updated to the cache. Specifically, the network traffic is first decrypted, decoded, parsed, etc. to extract the payload information list, and then the payload information is compressed and generalized. On the one hand, the generalized payload information is hashed as the primary key (key value) of the high-performance cache. On the other hand, the decoded but not generalized payload information is embedded in the constructed LLM prompt information template and sent to the LLM API (Application Programming Interface) for result query, where the return result format is also a fixed json template. The returned detection result is the value of the high-performance cache, and then the primary key and value are saved as key-value pairs (k, v pairs) in the high-performance cache.

[0034] The WAF detection engine based on high-performance cache and LLM proposed in the embodiment of the present application can solve the problems of long cycle and difficulty in manually extracting semantic features, poor generalization, low readability of regular matching, and lag, and realizes a more intelligent and efficient WAF detection engine that combines the advantages of high-performance cache and LLM, which not only reduces the complicated manual extraction of semantic feature engineering of conventional machine learning algorithms, but also avoids the problem of excessive latency of synchronous requests to large language model APIs. This method gives full play to the advantages of the LLM model in that it can deeply understand and analyze the contextual features such as the semantic grammar of the attack, and can generalize the identification of attack requests of different language types. At the same time, it uses high-performance cache to improve the efficiency of the intelligent detection engine, so that the model can have both powerful semantic understanding and learning capabilities and high efficiency of detection response without doing a lot of manual feature engineering.

[0035] The specific scheme of the embodiment of this application is as follows:

[0036] Figure 1 1 is a flow chart of an abnormal request detection method provided by an embodiment of the present application, and the abnormal request detection method can be applied to a WAF detection engine to detect abnormal requests. Figure 1 As shown, the method includes: step 110 to step 130.

[0037] Step 110, obtaining target payload information in the target access request.

[0038] In an exemplary embodiment, payload information is valid information in an access request that characterizes the request.

[0039] In an exemplary embodiment, the target access request may be a Web application request, that is, an HTTP (Hypertext Transfer Protocol) request.

[0040] Decrypt, parse, and process the target access request, extract payload information from at least one of the request method, URL, request header, and request body of the target access request, and if the extracted payload information is encoded data, decode the payload information and use the decoded data as the target payload information, and if the extracted payload information is non-encoded data, use the acquired payload information as the target payload information. The target payload information may be in the form of a list, including multiple parameters and parameter values ​​corresponding to each parameter.

[0041] Step 120: Determine a target cache primary key according to the target payload information.

[0042] The target payload information is compressed and processed to reduce the data volume of the target payload information to obtain a target cache primary key. The target cache primary key can represent the target payload information, but the data volume of the target cache primary key is much smaller than the data volume of the target payload information.

[0043] Step 130, according to the target cache primary key, query the target detection result corresponding to the target cache primary key from the cache, the cache caches the correspondence between the cache primary key and the detection result, the detection result is pre-acquired from the large language model according to the payload information corresponding to the cache primary key, and the detection result includes a normal request or an abnormal request.

[0044] In an exemplary embodiment, the cache may be a common cache or a high-performance cache, preferably a high-performance cache, to further improve the response speed of the detection engine.

[0045] In an exemplary embodiment, a large number of access requests can be processed in advance, and the corresponding detection results can be queried from the large language model based on the payload information of each access request, and the cache primary key can be determined based on the payload information of the access request, and the cache primary key and the detection results can be saved to the cache in the form of a key-value pair. Exemplarily, when the cache is a high-performance cache, the cache primary key and the detection results can be asynchronously saved to the cache in the background without affecting the execution of the main thread, and the main thread can continue to query and process other access requests. Exemplarily, the detection results include normal requests or abnormal requests. When the detection result is determined to be an abnormal request, the detection result can also include the attack type and harm score, etc.

[0046] After determining the target cache primary key corresponding to the target payload information, you can query whether the target cache primary key exists in the cache. If the target cache primary key exists in the cache, obtain the detection result corresponding to the target cache primary key from the cache. The detection result is the target detection result corresponding to the target access request.

[0047] The abnormal request detection method provided in the embodiment of the present application obtains the payload information in the target access request, determines the target cache primary key according to the target payload information, and queries the target detection result corresponding to the target cache primary key from the cache according to the target cache primary key. The cache caches the correspondence between the cache primary key and the detection result. The detection result is pre-acquired from the large language model according to the payload information corresponding to the cache primary key. In this way, there is no need to request the large language model in real time for real-time target access requests, but to query the pre-stored data from the cache, which can improve the detection efficiency of abnormal requests and the response speed of the detection engine. Moreover, compared with manually involving regular rules and manually extracting semantic features in conjunction with machine learning classifiers, it can generalize and detect network attacks of different languages ​​and different types.

[0048] Figure 2 is a flow chart of determining a target cache primary key according to target payload information in an embodiment of the present application, such as Figure 2 As shown, the determining of the target cache primary key according to the target payload information, i.e., the above-mentioned step 120, includes:

[0049] Step 121, compress the target effective load information to obtain compressed load information.

[0050] Based on the characters included in the target payload information, the target payload information is compressed to reduce the amount of data and obtain compressed payload information.

[0051] In one embodiment of the present application, compressing the target payload information to obtain compressed payload information includes:

[0052] Compressing a plurality of consecutive identical characters in the target payload information into one character to obtain preliminary compressed information;

[0053] According to the character type of each character in the preliminary compressed information, the characters in the preliminary compressed information are replaced with generalized placeholders corresponding to the character type to obtain the compressed payload information.

[0054] The collected decoded target payload information is compressed and generalized. First, the payload information is preliminarily compressed, that is, n consecutive identical characters are replaced with one character, and n consecutive blank characters such as spaces and line breaks are also compressed and merged to obtain preliminary compressed information; after the preliminary compression, generalized replacement is performed, that is, according to the mapping relationship between the character type and the generalized placeholder, the characters of each character type in the preliminary compressed information are replaced with the generalized placeholder corresponding to the character type to obtain the compressed payload information. Among them, characters can be divided into visible characters and invisible characters, among which visible characters include: English characters, numbers, punctuation marks, special symbols and other language symbols, and invisible characters include control characters and special binary characters. For the character types listed in the preset mapping relationship, mapping replacement can be performed, and the remaining character types can retain the original characters. Table 2 gives an example of the mapping relationship between character types and return placeholders. As shown in Table 2, English characters can be mapped to the generalized placeholder "a", numeric characters can be mapped to the generalized placeholder "1", and other Unicode characters except English characters, numbers, and symbols can be mapped to the generalized placeholder "x".

[0055] Table 1 Example of mapping between character types and return placeholders

[0056]

[0057] By compressing and generalizing the target payload information, the amount of data in the compressed payload information can be further reduced and the speed of querying from the cache can be improved.

[0058] Step 122: Perform a hash operation on the compressed payload information to obtain the target cache primary key.

[0059] The compressed payload information after the compression generalization process is hashed (for example, using the SHA-256 function) to obtain the target cache primary key, which is used as the key value of the cache, for example, the target cache primary key is used as the key value of a high-performance cache. For example, the hash operation can be expressed as the following formula:

[0060] key=f SHa-256 (payload)

[0061] Among them, key represents the target cache primary key, payload represents the compressed payload information, and f SHA-256 Indicates that the SHA-256 function is used for hash operations.

[0062] By compressing and hashing the target payload information, the data volume of the target cache primary key obtained is much smaller than the data volume of the target payload information, so that the query efficiency can be improved when the target cache primary key is used to query in the cache.

[0063] Based on the above technical solution, the method may further include: if the target cache primary key does not exist in the cache, releasing the target access request, and asynchronously obtaining the target detection result corresponding to the target payload information from the large language model based on the target payload information; and saving the target cache primary key and the target detection result in the cache in correspondence.

[0064] After querying from the cache based on the target cache primary key, if the target cache primary key does not exist in the cache, the target access request can be released, and the target detection result corresponding to the target payload information is asynchronously obtained from the large language model based on the target payload information, and the target cache primary key and the target detection result are saved in the cache as a key-value pair.

[0065] When the target cache primary key does not exist in the cache, the target detection result is asynchronously obtained from the large language model, and the target cache primary key and the target detection result are saved in the cache accordingly, thereby replenishing the cache data and facilitating the detection result to be directly obtained from the cache the next time the same or similar access request is received.

[0066] On the basis of the above technical solution, the step of obtaining the target detection result corresponding to the target payload information from the large language model includes:

[0067] Adding the target payload information to a prompt information template to obtain target prompt information, wherein the prompt information template includes detection prompt information and return data format;

[0068] The target prompt information is input into the large language model, anomaly detection is performed on the target payload information according to the detection prompt information through the large language model, and the target detection result is returned in the return data format.

[0069] The target payload information that has not been compressed and generalized after decryption and decoding is embedded into the prompt information template in sequence to obtain the target prompt information, and a detection query is constructed based on the target prompt information. The target prompt information is input into the large language model based on the API of the large language model. The large language model performs anomaly detection on the target payload information according to the detection prompt information in the target prompt information, and the obtained target detection result is represented as a return data format in the target prompt information, and the target detection result is returned in the return data format.

[0070] For example, the prompt information template can be as follows:

[0071] “Based on the following request data, determine whether there is a security threat. If so, please identify the possible attack type and give a threat level and score.

[0072] payload (replaced here with the decoded but uncompressed generalized payload content, i.e., the effective payload information)"

[0073] Request the large language model API to return the detection results of the fixed template, and request the API to obtain the detection results of the large language model as the value of the high-performance cache. The following json return format can be designed to return whether it is an attack, the attack type, and the harm score:

[0074] {

[0075] "is_attack":x,

[0076] "attack_type":"xxx",

[0077] "threat_level":"xxx",

[0078] "score":xx

[0079] }

[0080] In the above data format, is_attack indicates whether it is an attack, attack_type indicates the attack type, threat_level indicates the threat score, and score indicates the confidence level.

[0081] By adding the target payload information to the prompt information template, the target prompt information is obtained, and the target prompt information is input into the large language model. The target payload information is detected for anomalies through the large language model. Since the large language model has a rich data set and logical reasoning ability, it can effectively identify and understand different attacks, and can improve the accuracy of detection.

[0082] Based on the above technical solution, the method of obtaining the target payload information in the target access request includes: obtaining the request method from the target access request, and determining the request type of the target access request based on the request method; and obtaining the target payload information from at least one of the URL, request header and request body in the target access request based on the request type.

[0083] The request method is obtained from the request line of the target access request. Based on the request method, the request type of the target access request can be determined. The request type can be, for example, GET, POST, or PUT. For different request types, the location of the payload information may be different. The target payload information can be extracted from at least one of the URL, request header, and request body including the payload information in the target access request according to the request type. After the target payload information is extracted from the target access request, if the target payload information is encoded data, each target payload information needs to be decoded.

[0084] The target payload information in the target access request can be extracted through the payload information extraction module. The payload information extraction model decrypts the target access request, extracts the payload information and decodes it. Although the large language model has a rich knowledge base and powerful reasoning ability, it may not be accurate enough in the decoding process, which is manifested as relatively weak decoding ability. In order to improve the accuracy and efficiency of decoding, a special decoder can be pre-designed and integrated to complement and enhance the decoding performance of the large language model. The access request is decoded by a decoder, and the decoded payload information is input into the large language model, which can optimize the subsequent output of the large language model. Extracting payload information is mainly parsing and extracting from at least one of the request method, URL, request header, request body, etc. The following describes this process in detail, including the specific location and parsing method of extracting data from HTTP requests.

[0085] (1) Request method and URL extraction

[0086] Request method: First, determine the request type of the target access request (HTTP request) from the request method, such as GET, POST, PUT, etc.

[0087] URL analysis: URLs usually contain query strings. For GET requests, the parameters and their values ​​are in the query part of the URL. You need to parse these parameters, separate the key-value pairs, and save the extracted parameters and corresponding parameter values ​​into the payload information list.

[0088] For example, the request line of an access request is as follows:

[0089] GET / search? q=keyword&category=128HTTP / 1.1,

[0090] In this example, the parameters that need to be extracted are q,keyword and category,128.

[0091] (2) Request header analysis

[0092] Extract the values ​​of user-related information in the request header, that is, extract the values ​​of all HTTP request headers, such as user agent (User-Agent), source page (Referer), data stored on the user's local terminal (Cookie), etc., extract the values ​​of these fields, ignore the content type (Content-Type), content length (Content-Length) and other common fields, and save all extracted parameters and parameter values ​​in the form of key-value pairs in the payload information list.

[0093] (3) Request body parsing

[0094] For POST and other request types, the payload information is usually located in the request body of the HTTP request.

[0095] The request body content needs to be parsed, which may involve different data formats, such as form data (application / x-www-form-urlencoded), multipart form data (multipart / form-data), or JSON (application / json). If the data is provided in a format such as JSON or XML, the corresponding parser needs to be used to extract information from the nested data structure.

[0096] For example, a POST type access request is as follows:

[0097] POST / submit HTTP / 1.1

[0098] Content-Type:application / x-www-form-urlencoded

[0099] username=john&password=12345

[0100] In this example, the data format is form data, and the data to be extracted are the username, john, password, and 12345 fields. After extracting this information, save it in the form of key-value pairs in the payload information list.

[0101] (4) Decoding

[0102] If the data in the target access request is encoded data, after the original payload information is extracted, each encoded payload information needs to be decoded, and all decoded payload information is used as the target payload information.

[0103] The following example illustrates that there is an original access request with the following content:

[0104] POST / api / login HTTP / 1.1

[0105] Host:example.com

[0106] Content-Type:application / json

[0107] Content-Length:56

[0108] {"username":"YWRtaW4=","password":"Y0dGemN6RXlNelE9"}

[0109] After extracting all the payload information (payload), if there is no coding feature, there is no need to decode, otherwise recursive decoding is performed to obtain the list of payload information after decoding as shown in Table 2.

[0110] Table 2 Payload information

[0111] Original payload information Decoded payload information host host example.com example.com username username YWRtaW4= admin password password Y0dGemN6RXlNelE9 pass1234

[0112] By obtaining the request method from the target access request and determining the request type of the target access request based on the request method, and then obtaining the target payload information from at least one of the URL, request header and request body in the target access request based on the request type, all the payload information can be extracted to improve the accuracy of subsequent detection.

[0113] Figure 3 1 is a flow chart of cache construction based on a large language model in an embodiment of the present application. Before real-time detection of access requests based on the cache is performed, cache construction based on the large language model is required, such as Figure 3 As shown, cache construction can be performed through the following steps:

[0114] Step 310: Obtain access request data set.

[0115] An access request dataset is collected and prepared, the access request dataset comprising a plurality of historical access requests.

[0116] Step 320: For each historical access request in the access request data set, obtain historical payload information in the historical access request.

[0117] The collected access request data set is sent to the payload information extraction module for decryption, payload information extraction and decoding. For each historical access request in the access request data set, payload information is extracted respectively, and the payload information extracted from the historical access request is used as the historical payload information.

[0118] The method of extracting the payload information from the historical access request may refer to the method of extracting the payload information from the target access request in the above embodiment, which will not be described in detail here.

[0119] Step 330: Determine a history cache primary key based on the history payload information.

[0120] The historical payload information is compressed, generalized, and hashed, and the resulting value is used as the primary key of the historical cache.

[0121] The specific process of determining the historical cache primary key according to the historical payload information may refer to the process of determining the target cache primary key according to the target payload information in the above embodiment, which will not be repeated here.

[0122] Step 340: According to the historical payload information, a historical detection result corresponding to the historical payload information is obtained from the large language model.

[0123] The historical payload information is added to the prompt information template, which includes detection prompt information and return data format. The obtained prompt information is input into the large language model, and the historical payload information is subjected to anomaly detection based on the detection prompt information through the large language model, and the obtained historical detection results are returned in the return data format.

[0124] The specific process of obtaining the historical detection results corresponding to the historical payload information through the large language model may refer to the process of obtaining the target detection results corresponding to the target payload information through the large language model in the above embodiment.

[0125] Step 350: Save the historical cache primary key and the historical detection result in the cache in correspondence.

[0126] The historical cache primary key and the historical detection result are saved in the cache in the form of a key-value pair. The cache is preferably a high-performance cache.

[0127] Steps 320 to 350 are continuously and repeatedly executed to obtain sufficient cache data.

[0128] By obtaining the corresponding historical detection results from the large language model based on the access request dataset in advance, and saving the historical cache primary key and the historical detection results in the cache accordingly, sufficient cache data is provided for real-time detection, which can improve the efficiency of real-time detection. Moreover, since the large language model is trained on large datasets in multiple languages, it can understand and process attack requests in different languages ​​and cultural backgrounds, thereby providing more extensive protection.

[0129] Figure 4Schematic diagram of the overall process of generating cache data based on high-performance cache and asynchronous large language model in the embodiment of the present application. Figure 4 As shown, for the original request (historical access request in the access request data set), the payload information is extracted and decoded to obtain a payload information list; the payload information in the payload information list is further compressed and generalized to obtain a generalized payload information list, and a hash value is calculated for the payload information in the generalized payload information list, and the hash value is used as the primary key (key) of the cache; a query is constructed for the payload information in the payload information list, the payload information is added to the large language model prompt information template, the API of the large language model is called, the detection result is obtained, and the detection result is used as the cache value; the primary key and the detection result are stored in the high-performance cache in the form of a key-value pair.

[0130] Figure 5 Schematic diagram of the process of real-time WAF detection in the embodiment of the present application. Figure 5 As shown, WAF receives a request (target access request), preprocesses the request, obtains a payload information list (target payload information), calculates a hash value of the payload information (target cache primary key), checks the high-performance cache, and determines whether the hash value exists in the high-performance cache; if the hash value exists in the high-performance cache, reads the cache result, and determines whether the request is an attack based on the cache result. If the request is an attack, the request is blocked; if the request is not an attack, the request is released; if the hash value does not exist in the high-performance cache, the request is released, and the API of the large language model is asynchronously queried, and the queried results are saved in the cache.

[0131] The abnormal request detection method provided in the embodiment of the present application has the following beneficial effects:

[0132] 1. Improved grammatical and semantic perception of real-time detection: In existing real-time detection technologies, regular methods are complex and rigid, and lack context perception. Large language models usually have context processing capabilities of 4,000+ tokens (words) and a wide corpus, including data from a variety of CVEs (Common Vulnerabilities & Exposures) and other attack types. Thanks to its rich data set and logical reasoning capabilities, large language models can effectively identify and understand attack variants and have strong semantic understanding capabilities, which regular methods do not have.

[0133] 2. Omit cumbersome feature engineering: The existing artificial semantic feature extraction and classifier technology all require complex and cumbersome feature engineering, manual implementation of syntax parsers, and design and extraction of feature vectors. Poor feature vector design will directly affect the accuracy of model detection, and a model can only classify and identify one text grammar, and is not generalizable. The large language model interface used in the embodiments of the present application can automatically learn features and patterns from generalized attack requests without manual selection and extraction of features. In addition, since large language models are trained on large data sets in multiple languages, they can understand and process attack requests in different languages ​​and cultural backgrounds, thereby providing a wider range of protection.

[0134] 3. High-performance cache improves the feasibility of embedding large language model detection capabilities: The large language model has a large number of parameters and a large amount of computation, which leads to increased latency in real-time processing, and the detection engine is a security detection product with high latency requirements. Asynchronous processing of request data through high-performance cache avoids blocking the main thread under high load conditions, thereby improving the response time and performance of the detection engine based on the large language model. Real-time detection technology that combines asynchronous large language models and high-performance cache can provide an efficient, accurate, and adaptable network security solution that helps to cope with increasingly complex and changing network threats.

[0135] Figure 6 is a schematic diagram of the structure of an abnormal request detection device provided in an embodiment of the present application, such as Figure 6 As shown, the device comprises:

[0136] A target payload acquisition module 610, configured to acquire target payload information in a target access request;

[0137] A target cache primary key determination module 620, configured to determine a target cache primary key according to the target payload information;

[0138] The detection result query module 630 is used to query the target detection result corresponding to the target cache primary key from the cache according to the target cache primary key. The cache caches the correspondence between the cache primary key and the detection result. The detection result is obtained from the large language model in advance based on the payload information corresponding to the cache primary key. The detection result includes a normal request or an abnormal request.

[0139] Optionally, the cache primary key determination module includes:

[0140] A compression unit, used for compressing the target effective load information to obtain compressed load information;

[0141] The hash operation unit is used to perform a hash operation on the compressed payload information to obtain the target cache primary key.

[0142] Optionally, the compression unit is specifically used for:

[0143] Compressing a plurality of consecutive identical characters in the target payload information into one character to obtain preliminary compressed information;

[0144] According to the character type of each character in the preliminary compressed information, the characters in the preliminary compressed information are replaced with generalized placeholders corresponding to the character type to obtain the compressed payload information.

[0145] Optionally, the device further comprises:

[0146] an asynchronous query module, configured to allow the target access request if the target cache primary key does not exist in the cache, and asynchronously obtain a target detection result corresponding to the target payload information from the large language model according to the target payload information;

[0147] The first cache module is used to store the target cache primary key and the target detection result in the cache in correspondence.

[0148] Optionally, the asynchronous query module includes:

[0149] a prompt information generating unit, configured to add the target payload information to a prompt information template to obtain target prompt information, wherein the prompt information template includes detection prompt information and a return data format;

[0150] The query unit is used to input the target prompt information into the large language model, perform anomaly detection on the target payload information according to the detection prompt information through the large language model, and return the target detection result in the return data format.

[0151] Optionally, the device further comprises:

[0152] A data set acquisition module, used to acquire an access request data set;

[0153] A historical payload acquisition module, configured to acquire historical payload information in each historical access request in the access request data set;

[0154] A historical cache primary key determination module, used to determine a historical cache primary key according to the historical payload information;

[0155] A historical detection result acquisition module, configured to acquire, from the large language model, historical detection results corresponding to the historical payload information according to the historical payload information;

[0156] The second cache module is used to store the historical cache primary key and the historical detection result in the cache in correspondence.

[0157] Optionally, the target payload acquisition module includes:

[0158] a request type determination unit, configured to obtain a request method from the target access request, and determine a request type of the target access request according to the request method;

[0159] The payload acquisition unit is used to acquire the target payload information from at least one of the URL, the request header and the request body in the target access request according to the request type.

[0160] The abnormal request detection device provided in the embodiment of the present application is used to implement the various steps of the abnormal request detection method described in the embodiment of the present application. The specific implementation methods of each module of the device refer to the corresponding steps and will not be repeated here.

[0161] The abnormal request detection device provided in the embodiment of the present application obtains the payload information in the target access request, determines the target cache primary key according to the target payload information, and queries the target detection result corresponding to the target cache primary key from the cache according to the target cache primary key. The cache caches the correspondence between the cache primary key and the detection result. The detection result is pre-acquired from the large language model according to the payload information corresponding to the cache primary key. In this way, there is no need to request the large language model in real time for real-time target access requests, but to query the pre-stored data from the cache, which can improve the detection efficiency of abnormal requests and the response speed of the detection engine. Moreover, compared with manually involving regular rules and manually extracting semantic features in conjunction with machine learning classifiers, it can generalize the detection of network attacks of different languages ​​and different types.

[0162] Figure 7 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 7 As shown, the electronic device 700 may include one or more processors 710 and one or more memories 720 connected to the processor 710. The electronic device 700 may also include an input interface 730 and an output interface 740 for communicating with another device or system. The program code executed by the processor 710 may be stored in the memory 720.

[0163] The processor 710 in the electronic device 700 calls the program code stored in the memory 720 to execute the abnormal request detection method in the above embodiment.

[0164] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the abnormal request detection method described in the embodiment of the present application are implemented.

[0165] The embodiment of the present application also provides a computer program product, which, when executed by a processor, implements the steps of the abnormal request detection method as described in the embodiment of the present application.

[0166] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] The above is a detailed introduction to an abnormal request detection method, device, electronic device and storage medium provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Claims

1. A method for detecting abnormal requests, characterized in that: include: Obtain target payload information in the target access request; Determining a target cache primary key according to the target payload information; According to the target cache primary key, a target detection result corresponding to the target cache primary key is queried from the cache, wherein the cache caches a correspondence between the cache primary key and the detection result, wherein the detection result is obtained from a large language model in advance based on the payload information corresponding to the cache primary key, and the detection result includes a normal request or an abnormal request.

2. The method according to claim 1, characterized in that Determining the target cache primary key according to the target payload information includes: compressing the target payload information to obtain compressed payload information; A hash operation is performed on the compressed payload information to obtain the target cache primary key.

3. The method according to claim 2, characterized in that The compressing the target effective load information to obtain compressed load information includes: Compressing a plurality of consecutive identical characters in the target payload information into one character to obtain preliminary compressed information; According to the character type of each character in the preliminary compressed information, the characters in the preliminary compressed information are replaced with generalized placeholders corresponding to the character type to obtain the compressed payload information.

4. The method according to any one of claims 1 to 3, characterized in that: Also includes: If the target cache primary key does not exist in the cache, the target access request is released, and according to the target payload information, a target detection result corresponding to the target payload information is asynchronously obtained from the large language model; The target cache primary key and the target detection result are correspondingly saved in the cache.

5. The method according to claim 4, characterized in that The acquiring, from the large language model, a target detection result corresponding to the target payload information comprises: Adding the target payload information to a prompt information template to obtain target prompt information, wherein the prompt information template includes detection prompt information and return data format; The target prompt information is input into the large language model, anomaly detection is performed on the target payload information according to the detection prompt information through the large language model, and the target detection result is returned in the return data format.

6. The method according to any one of claims 1 to 3, characterized in that: Before querying the target detection result corresponding to the target cache primary key from the cache according to the target cache primary key, the method further includes: Get access request data set; For each historical access request in the access request data set, obtaining historical payload information in the historical access request; Determining a historical cache primary key based on the historical payload information; According to the historical payload information, acquiring, from the large language model, historical detection results corresponding to the historical payload information; The historical cache primary key and the historical detection result are correspondingly saved in the cache.

7. The method according to any one of claims 1 to 3, characterized in that: The obtaining target payload information in the target access request includes: Acquire a request method from the target access request, and determine a request type of the target access request according to the request method; The target payload information is obtained from at least one of a URL, a request header, and a request body in the target access request according to the request type.

8. An abnormal request detection device, characterized in that: include: A target payload acquisition module, used to acquire target payload information in a target access request; A target cache primary key determination module, used to determine a target cache primary key according to the target payload information; A detection result query module is used to query the target detection result corresponding to the target cache primary key from the cache according to the target cache primary key. The cache caches the correspondence between the cache primary key and the detection result. The detection result is obtained from the large language model in advance based on the payload information corresponding to the cache primary key. The detection result includes a normal request or an abnormal request.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the abnormal request detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the abnormal request detection method described in any one of claims 1 to 7 are implemented.

Images