Abnormal request detection methods, devices, electronic equipment and storage media

Abstract

This application discloses an abnormal request detection method, apparatus, electronic device, and storage medium. The method includes: obtaining target payload information from a target access request; determining a target cache key based on the target payload information; and querying a target detection result corresponding to the target cache key from a cache, wherein the cache stores a correspondence between cache keys and detection results, and the detection results are pre-obtained from a large language model based on the payload information corresponding to the cache key. The detection results include normal requests or abnormal requests. This application embodiment can improve the detection efficiency of abnormal requests and increase the response speed of the detection engine.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to an abnormal request detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of internet technology, web applications have become an essential part of people's daily lives and business operations, making them a primary target for attacks. Web applications face a variety of security threats, including SQL (Structured Query Language) injection, cross-site scripting (XSS), cross-site request forgery (CSRF), and file inclusion. Web Application Firewalls (WAFs) can provide specialized protection against these attacks. However, with advancements in hacking techniques, attacks have become more complex and covert, rendering traditional security defenses ineffective.

[0003] In existing technologies, there are two common methods for WAF detection engines: one is based on manual feature engineering, which involves writing a syntax and semantic parser, manually designing and extracting syntax and semantic features, and then connecting them with machine learning classifiers such as LR (Logistic Regression) to realize a WAF detection engine for a certain type of language; the other is to analyze attack samples and manually design various regular rules to intercept attacks in different scenarios.

[0004] The method of manually designing regular rules has the following disadvantages: if the regular rules are too complex, the readability is poor and the maintenance is difficult; regular rules have a lag effect, can only be designed based on existing sample patterns, and do not have predictive ability.

[0005] Combining manually extracted semantic features with machine learning classifiers provides a degree of context awareness and more accurate feature extraction for specific languages. However, this approach also has the following drawbacks: long development cycles, high barriers to entry, and significant difficulty and complexity in designing and implementing syntax parsers; each engine can only specifically detect attacks in one language, resulting in overly specific targeting and poor generalization; and it requires a high level of security experience from personnel, as manually designing which features to extract significantly impacts the engine's detection capabilities.

[0006] In contrast, Large Language Models (LLMs) do indeed possess the ability to deeply understand complex languages ​​and behavioral patterns, and can generalize to identify different types of network attacks, exceeding the capabilities of traditional methods such as regular expressions. However, directly introducing the LLM interface for synchronous traffic detection will result in low detection efficiency and slow response times for the detection engine. Summary of the Invention

[0007] This application provides an abnormal request detection method, apparatus, electronic device, and storage medium, which helps to improve detection efficiency and increase the response speed of the detection engine.

[0008] To address the aforementioned problems, in a first aspect, embodiments of this application provide an abnormal request detection method, including:

[0009] Obtain the target payload information from the target access request;

[0010] Based on the target payload information, determine the target cache primary key;

[0011] Based on the target cache key, query the cache for the target detection result corresponding to the target cache key. The cache contains the correspondence between the cache key and the detection result. The detection result is obtained in advance from the large language model based on the payload information corresponding to the cache key. The detection result includes normal requests or abnormal requests.

[0012] Secondly, embodiments of this application provide an abnormal request detection device, including:

[0013] The target payload acquisition module is used to acquire target payload information from the target access request;

[0014] The target cache primary key determination module is used to determine the target cache primary key based on the target payload information;

[0015] The detection result query module is used to query the target detection result corresponding to the target cache key from the cache according to the target cache key. The cache stores the correspondence between the cache key and the detection result. The detection result is obtained in advance from the large language model according to the payload information corresponding to the cache key. The detection result includes normal requests or abnormal requests.

[0016] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the abnormal request detection method described in embodiments of this application.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, represents the steps of the abnormal request detection method disclosed in embodiments of this application.

[0018] The abnormal request detection method, apparatus, electronic device, and storage medium provided in this application obtain the payload information in the target access request, determine the target cache key based on the target payload information, and query the target detection result corresponding to the target cache key from the cache. The cache stores the correspondence between the cache key and the detection result. The detection result is obtained in advance from a large language model based on the payload information corresponding to the cache key. In this way, for real-time target access requests, it is not necessary to request the large language model in real time, but to query the pre-stored data from the cache, which can improve the detection efficiency of abnormal requests and improve the response speed of the detection engine. Moreover, compared with manually handling regular rules and manually extracting semantic features in conjunction with machine learning classifiers, it can generalize to detect different languages ​​and different types of network attacks. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of an abnormal request detection method provided in an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating the process of determining the target cache primary key based on the target payload information in an embodiment of this application.

[0022] Figure 3 This is a flowchart of cache construction based on a large language model in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the overall process of generating cached data based on high-performance caching and asynchronous large language models in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of the real-time detection process of WAF in the embodiments of this application;

[0025] Figure 6 This is a schematic diagram of the structure of an abnormal request detection device provided in an embodiment of this application;

[0026] Figure 7This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Before detailing the technical solutions of the embodiments of this application, the terms involved in the embodiments of this application are explained as follows:

[0029] High-performance caching (Asynchronous Caching): High-performance caching is a caching mechanism that allows data read and write operations to be performed asynchronously in the background without blocking the main thread. This means the system can continue processing other tasks while caching operations are completed in the background. High-performance caching is typically used to improve application performance and responsiveness because it reduces the time spent waiting for cache read and write operations to complete. In high-load or big data processing scenarios, high-performance caching can effectively improve system processing capabilities and user experience.

[0030] Large Language Models (MLMs), such as the GPT (Generative Pre-trained Transformer) series, are deep learning models trained on massive datasets, designed to understand and generate human language. These models typically have billions or even trillions of parameters and are capable of handling complex language tasks, including text generation, translation, summarization, and question answering. By learning language rules and patterns from vast amounts of text data, MLMs possess powerful language understanding and generation capabilities and are widely used in the field of Natural Language Processing (NLP).

[0031] The technical solutions of the embodiments of this application will be described in detail below.

[0032] The embodiments of this application aim to address the limitations of existing conventional regular expressions and machine learning classifiers, as well as the slow response speed of directly introducing large language models for detection. They propose a real-time detection technology that combines high-performance caching and asynchronous large language models.

[0033] To address the limitations of existing methods, this application proposes a real-time detection technique combining high-performance caching and asynchronous large language models. The core idea is to asynchronously query key-value pairs in the large language model cache using massive traffic data. During real-time detection, the high-performance cache is queried first; if no cache hit occurs, the large language model is queried asynchronously, and the result is updated in the cache. Specifically, network traffic is first decrypted, decoded, and parsed to extract a payload list. This payload information is then compressed and generalized. The generalized payload information is hashed and used as the primary key (key) of the high-performance cache. Simultaneously, the decoded but ungeneralized payload information is embedded into a pre-constructed LLM prompt template and sent to the LLM API (Application Programming Interface) for result querying. The returned result is also in a fixed JSON template format. This returned detection result is the value in the high-performance cache. Finally, the primary key and value are saved as key-value pairs (k, v) in the high-performance cache.

[0034] The WAF detection engine based on high-performance caching and LLM proposed in this application addresses the problems of long cycles, high difficulty, poor generalization, low readability of regular expression matching, and lag associated with manual semantic feature extraction. It achieves a more intelligent and efficient WAF detection engine that combines the advantages of high-performance caching and LLM, reducing the tedious manual extraction work required by conventional machine learning algorithms and avoiding the high latency issues of synchronously requesting large language model APIs. This method fully leverages the advantages of LLM models, which can deeply understand and analyze the semantic and syntactic contextual features of attacks and can generalize to identify attack requests of different language types. Simultaneously, it utilizes high-performance caching to improve the efficiency of the intelligent detection engine, enabling the model to possess both powerful semantic understanding and learning capabilities and high detection response efficiency without requiring extensive manual feature engineering.

[0035] The specific solution of this application embodiment is as follows:

[0036] Figure 1 This is a flowchart illustrating an abnormal request detection method provided in an embodiment of this application. This abnormal request detection method can be applied in a WAF detection engine to detect abnormal requests. Figure 1 As shown, the method includes steps 110 to 130.

[0037] Step 110: Obtain the target payload information from the target access request.

[0038] In one exemplary embodiment, the payload information is the valid information in the access request that characterizes the request features.

[0039] In one exemplary embodiment, the target access request may be a web application request, namely an HTTP (Hypertext Transfer Protocol) request.

[0040] The target access request is decrypted and parsed to extract payload information from at least one of the request method, URL, request headers, and request body. If the extracted payload information is encoded data, it is decoded, and the decoded data is used as the target payload information. If the extracted payload information is unencoded data, it is used as the target payload information. The target payload information can be in the form of a list, including multiple parameters and their corresponding values.

[0041] Step 120: Determine the target cache primary key based on the target payload information.

[0042] The target payload information is compressed and processed to reduce the amount of data, resulting in a target cache primary key. The target cache primary key can represent the target payload information, but the amount of data in the target cache primary key is much smaller than the amount of data in the target payload information.

[0043] Step 130: Based on the target cache key, query the cache for the target detection result corresponding to the target cache key. The cache contains the correspondence between the cache key and the detection result. The detection result is obtained in advance from the large language model based on the payload information corresponding to the cache key. The detection result includes normal requests or abnormal requests.

[0044] In one exemplary embodiment, the cache can be a regular cache or a high-performance cache, preferably a high-performance cache, to further improve the response speed of the detection engine.

[0045] In one exemplary embodiment, a large number of access requests can be pre-processed, and the corresponding detection results can be queried from a large language model based on the payload information of each access request. A cache primary key is determined based on the payload information of the access requests, and the cache primary key and the detection results are saved to the cache as key-value pairs. For example, when the cache is a high-performance cache, the cache primary key and detection results can be saved to the cache asynchronously in the background without affecting the execution of the main thread, which can continue to query and process other access requests. For example, the detection results include normal requests or abnormal requests. When the detection result determines that a request is abnormal, the detection result may also include attack type and harm score, etc.

[0046] After determining the target cache primary key corresponding to the target payload information, you can query whether the target cache primary key exists in the cache. If the target cache primary key exists in the cache, you can retrieve the detection result corresponding to the target cache primary key from the cache. This detection result is the target detection result corresponding to the target access request.

[0047] The abnormal request detection method provided in this application obtains the payload information in the target access request, determines the target cache primary key based on the target payload information, and queries the target detection result corresponding to the target cache primary key from the cache. The cache stores the correspondence between the cache primary key and the detection result. The detection result is obtained in advance from a large language model based on the payload information corresponding to the cache primary key. In this way, for real-time target access requests, it is not necessary to request the large language model in real time, but to query the pre-stored data from the cache, which can improve the detection efficiency of abnormal requests and improve the response speed of the detection engine. Moreover, compared with manually handling regular rules and manually extracting semantic features in conjunction with machine learning classifiers, it can generalize to detect network attacks of different languages ​​and types.

[0048] Figure 2 This is a flowchart illustrating the determination of the target cache primary key based on the target payload information in an embodiment of this application, as follows: Figure 2 As shown, the step of determining the target cache primary key based on the target payload information, i.e., step 120 above, includes:

[0049] Step 121: Compress the target payload information to obtain compressed payload information.

[0050] Based on the characters contained in the target payload information, the target payload information is compressed to reduce the amount of data, resulting in compressed payload information.

[0051] In one embodiment of this application, the step of compressing the target payload information to obtain compressed payload information includes:

[0052] Compress multiple consecutive identical characters in the target payload information into a single character to obtain preliminary compressed information;

[0053] Based on the character type of each character in the preliminary compression information, the characters in the preliminary compression information are replaced with generalized placeholders corresponding to the character type to obtain the compressed payload information.

[0054] The collected, decoded target payload information is compressed and generalized. First, the payload information undergoes preliminary compression, where n consecutive identical characters are replaced with a single instance of that character. Consecutive spaces, newlines, and other whitespace characters are also compressed and merged to obtain preliminary compressed information. After preliminary compression, generalized replacement is performed. Based on the mapping relationship between character types and generalized placeholders, characters of each character type in the preliminary compressed information are replaced with the corresponding generalized placeholder, resulting in compressed payload information. Characters can be divided into visible and invisible characters. Visible characters include English characters, numbers, punctuation marks, special symbols, and other language symbols. Invisible characters include control characters and special binary characters. Character types listed in the preset mapping relationship can be mapped and replaced; other character types retain their original characters. Table 2 provides an example of the mapping relationship between character types and return placeholders. As shown in Table 2, English characters can be mapped to the generalized placeholder "a", numeric characters can be mapped to the generalized placeholder "1", and other Unicode characters other than English characters, numbers, and symbols can be mapped to the generalized placeholder "x".

[0055] Table 1 shows an example of the mapping relationship between character types and returned placeholders.

[0056]

[0057] By compressing and generalizing the target payload information, the amount of data in the compressed payload information can be further reduced, thereby improving the speed of querying from the cache.

[0058] Step 122: Perform a hash operation on the compressed payload information to obtain the target cache primary key.

[0059] The compressed payload information, after compression and generalization, is hashed (e.g., using the SHA-256 function) to obtain the target cache key. This target cache key serves as the cache key, for example, as the key for a high-performance cache. The hash operation can be represented by the following formula:

[0060] key=f SHa-256 (payload)

[0061] Where key represents the target cache primary key, payload represents the compressed payload information, and f SHA-256 This indicates that the SHA-256 function is used for hashing.

[0062] By compressing and hashing the target payload information, the resulting target cache primary key is much smaller than the target payload information, thus improving query efficiency when using the target cache primary key to query the cache.

[0063] Based on the above technical solution, the method may further include: if the target cache key does not exist in the cache, then allow the target access request, and asynchronously obtain the target detection result corresponding to the target payload information from the large language model according to the target payload information; save the target cache key and the target detection result in the cache.

[0064] After querying the cache based on the target cache primary key, if the target cache primary key does not exist in the cache, the target access request can be allowed, and the target detection result corresponding to the target payload information can be obtained asynchronously from the large language model based on the target payload information. The target cache primary key and the target detection result are then saved as a key-value pair in the cache.

[0065] By asynchronously retrieving the target detection result from the large language model when the target cache key does not exist in the cache, and saving the target cache key and the target detection result in the cache, the cached data is supplemented, making it easier to directly retrieve the detection result from the cache when the same or similar access request is received next time.

[0066] Based on the above technical solution, the step of obtaining the target detection result corresponding to the target payload information from the large language model includes:

[0067] The target payload information is added to the prompt information template to obtain the target prompt information. The prompt information template includes detection prompt information and return data format.

[0068] The target prompt information is input into the large language model, and the large language model performs anomaly detection on the target payload information based on the detection prompt information, and returns the target detection result in the returned data format.

[0069] The target payload information, after decryption and decoding but without compression and generalization processing, is sequentially embedded into the prompt information template to obtain the target prompt information. A detection query is constructed based on the target prompt information. Based on the API of the large language model, the target prompt information is input into the large language model. The large language model performs anomaly detection on the target payload information according to the detection prompt information in the target prompt information, and represents the obtained target detection result as the return data format in the target prompt information, and returns the target detection result in this return data format.

[0070] For example, the prompt message template can use the following template:

[0071] "Based on the following requested data, determine whether a security threat exists. If so, identify the possible attack types and provide the threat level and score."

[0072] "payload (replace this with the decoded but uncompressed generalized payload content, i.e., the effective payload information)"

[0073] The requirement is for the large language model API to return detection results with a fixed template. The API request retrieves the detection results from the large language model, which are then used as the value for a high-performance cache. A possible JSON return format is as follows: indicating whether it's an attack, the attack type, and the severity score.

[0074] {

[0075] "is_attack":x,

[0076] "attack_type":"xxx",

[0077] "threat_level":"xxx",

[0078] "score":xx

[0079] }

[0080] In the above data format, is_attack indicates whether it is an attack, attack_type indicates the attack type, threat_level indicates the threat score, and score indicates the confidence level.

[0081] By adding the target payload information to the prompt information template, the target prompt information is obtained. The target prompt information is then input into a large language model, which performs anomaly detection on the target payload information. Since the large language model has a rich dataset and logical reasoning ability, it can effectively identify and understand different attacks, thereby improving the accuracy of detection.

[0082] Based on the above technical solution, obtaining the target payload information in the target access request includes: obtaining a request method from the target access request, and determining the request type of the target access request according to the request method; and obtaining the target payload information from at least one of the URL, request header, and request body in the target access request according to the request type.

[0083] The request method is obtained from the request line of the target access request. Based on the request method, the request type of the target access request can be determined. The request type can be, for example, GET, POST, or PUT. The location of the payload information may differ for different request types. The target payload information can be extracted from at least one of the URL, request headers, and request body that include the payload information in the target access request, depending on the request type. After extracting the target payload information from the target access request, if the target payload information is encoded data, it needs to be decoded.

[0084] The target payload information can be extracted from the target access request using a payload information extraction module. The payload information extraction model decrypts the target access request, extracts the payload information, and then decodes it. While large language models possess rich knowledge bases and powerful reasoning capabilities, their decoding accuracy may be insufficient, exhibiting relatively weak decoding capabilities. To improve decoding accuracy and efficiency, specialized decoders can be pre-designed and integrated to complement and enhance the decoding performance of the large language model. Decoding the access request using a decoder and inputting the decoded payload information into the large language model can optimize its subsequent output. Payload information extraction primarily involves parsing and extracting data from at least one of the following: request method, URL, request headers, and request body. This process is described in detail below, including the specific locations and parsing methods for extracting data from HTTP requests.

[0085] (1) Request method and URL extraction

[0086] Request method: First, determine the request type of the target access request (HTTP request) from the request method, such as GET, POST, PUT, etc.

[0087] URL analysis: URLs typically contain query strings. For GET requests, parameters and their values ​​are all in the query part of the URL. These parameters need to be parsed, key-value pairs separated, and the extracted parameters and their corresponding values ​​saved to a payload information list.

[0088] For example, the request line of an access request might look like this:

[0089] GET / search? q=keyword&category=128HTTP / 1.1,

[0090] In this example, the parameters to be extracted are q, keyword, category, and 128.

[0091] (2) Request header parsing

[0092] Extract user-related information from the request headers, i.e., extract all HTTP request header values, such as User-Agent, Referer, and data stored on the user's local terminal (Cookie). Extract the values ​​of these fields, ignoring common fields such as Content-Type and Content-Length. Save all extracted parameters and parameter values ​​as key-value pairs to the payload information list.

[0093] (3) Request body parsing

[0094] For POST and other request types, the payload information is usually located in the request body of the HTTP request.

[0095] The request body needs to be parsed, which may involve different data formats, such as form data (application / x-www-form-urlencoded), multipart form data (multipart / form-data), or JSON (application / json). If the data is provided in a format such as JSON or XML, the appropriate parser needs to be used to extract information from nested data structures.

[0096] For example, a POST type access request looks like this:

[0097] POST / submit HTTP / 1.1

[0098] Content-Type:application / x-www-form-urlencoded

[0099] username=john&password=12345

[0100] In this example, the data format is form data, and the data to be extracted are the fields username, john, password, and 12345. After extracting this information, it is saved as key-value pairs in the payload information list.

[0101] (4) Decoding

[0102] If the data in the target access request is encoded, after extracting the original payload information, it is necessary to decode each encoded payload information. All decoded payload information is used as the target payload information.

[0103] The following example illustrates this. Suppose we have an original access request with the following content:

[0104] POST / api / login HTTP / 1.1

[0105] Host:example.com

[0106] Content-Type: application / json

[0107] Content-Length: 56

[0108] {"username":"YWRtaW4=","password":"Y0dGemN6RXlNelE9"}

[0109] After extracting all payload information, if there are no encoded features, no decoding is required; otherwise, recursive decoding is performed, and the list of decoded payload information is shown in Table 2.

[0110] Table 2 Payload Information

[0111] Original payload information Decoded payload information host host example.com example.com username username YWRtaW4= admin password password Y0dGemN6RXlNelE9 pass1234

[0112] By obtaining the request method from the target access request and determining the request type of the target access request based on the request method, and then obtaining the target payload information from at least one of the URL, request headers, and request body in the target access request based on the request type, all payload information can be extracted, thus improving the accuracy of subsequent detection.

[0113] Figure 3 This is a flowchart illustrating cache construction based on a large language model in this embodiment of the application. Before real-time detection of access requests based on the cache, cache construction based on a large language model is required, such as... Figure 3 As shown, the cache can be built using the following steps:

[0114] Step 310: Obtain the access request dataset.

[0115] Collect and prepare an access request dataset that includes multiple historical access requests.

[0116] Step 320: For each historical access request in the access request dataset, obtain the historical payload information of the historical access request.

[0117] The collected access request dataset is sent to the payload information extraction module for decryption, payload information extraction, and decoding. For each historical access request in the dataset, payload information is extracted separately, and this extracted payload information is used as historical payload information.

[0118] The method for extracting payload information from historical access requests can refer to the method for extracting payload information from target access requests in the above embodiments, and will not be repeated here.

[0119] Step 330: Determine the historical cache primary key based on the historical payload information.

[0120] The historical payload information is compressed, generalized, and hashed to obtain a value that serves as the primary key for the historical cache.

[0121] The specific process of determining the historical cache primary key based on historical payload information can be found in the above embodiment of determining the target cache primary key based on target payload information, and will not be repeated here.

[0122] Step 340: Based on the historical payload information, obtain the historical detection results corresponding to the historical payload information from the large language model.

[0123] Historical payload information is added to the prompt information template, which includes detection prompt information and return data format. The obtained prompt information is input into the large language model. The large language model performs anomaly detection on the historical payload information based on the detection prompt information and returns the obtained historical detection results in the return data format.

[0124] The specific process of obtaining historical detection results corresponding to historical payload information through a large language model can be referred to in the above embodiment, which describes the process of obtaining target detection results corresponding to target payload information through a large language model.

[0125] Step 350: Save the historical cache primary key and the corresponding historical detection results to the cache.

[0126] The historical cache key and historical detection results are saved in the cache as key-value pairs. The cache is preferably a high-performance cache.

[0127] Repeat steps 320 to 350 continuously to obtain sufficient cached data.

[0128] By pre-retrieving corresponding historical detection results from the large language model based on the access request dataset, and saving the historical cache key and the historical detection results in the cache, sufficient cached data is provided for real-time detection, which can improve the efficiency of real-time detection. Moreover, since the large language model is trained on a large dataset of multiple languages, it can understand and process attack requests from different language and cultural backgrounds, thus providing broader protection.

[0129] Figure 4This is a schematic diagram illustrating the overall process of generating cached data based on high-performance caching and asynchronous large language models in this embodiment of the application. For example... Figure 4 As shown, for the original request (historical access requests in the access request dataset), payload information is extracted and decoded to obtain a payload information list. The payload information in the payload information list is further compressed and generalized to obtain a generalized payload information list. A hash value is calculated for the payload information in the generalized payload information list, and this hash value is used as the primary key of the cache. A query is constructed based on the payload information in the payload information list, the payload information is added to the large language model prompt information template, the large language model API is called to obtain the detection result, and the detection result is used as the cache value. The primary key and the detection result are stored in the high-performance cache in the form of key-value pairs.

[0130] Figure 5 This is a schematic diagram of the real-time detection process of WAF in an embodiment of this application. Figure 5 As shown, the WAF receives a request (target access request), preprocesses the request to obtain a list of payload information (target payload information), calculates the hash value of the payload information (target cache primary key), checks the high-performance cache to determine if the hash value exists in the high-performance cache; if the hash value exists in the high-performance cache, it reads the cache result and determines whether the request is an attack based on the cache result; if the request is an attack, it blocks the request; if the request is not an attack, it allows the request; if the hash value does not exist in the high-performance cache, it allows the request and asynchronously queries the API of the large language model, saving the query result to the cache.

[0131] The abnormal request detection method provided in this application has the following beneficial effects:

[0132] 1. Enhanced Syntax and Semantic Awareness in Real-Time Detection: Existing real-time detection technologies rely on complex and rigid regular expression methods, lacking context awareness. Large language models, on the other hand, typically possess context processing capabilities for 4000+ tokens and extensive corpora, including data on various CVEs (Common Vulnerabilities & Exposures) and other attack types. Benefiting from their rich datasets and logical reasoning capabilities, large language models can effectively identify and understand attack variants, demonstrating strong semantic understanding capabilities that regular expressions lack.

[0133] 2. Eliminating Tedious Feature Engineering: Existing techniques involving manual semantic feature extraction combined with classifiers require complex and tedious feature engineering, including manually implementing syntax parsers and designing feature vectors. Poorly designed feature vectors directly impact the accuracy of model detection, and a single model can only classify and recognize one type of text syntax, lacking generalization ability. In contrast, the large language model interface used in this application can automatically learn features and patterns from generalized attack requests without requiring manual feature selection and extraction. Furthermore, since large language models are trained on large datasets of multiple languages, they can understand and handle attack requests from different linguistic and cultural backgrounds, thus providing broader protection.

[0134] 3. High-performance caching enhances the feasibility of embedding large language model detection capabilities: The sheer number of parameters and computational demands of large language models leads to increased latency during real-time processing, while detection engines are security detection products with stringent latency requirements. High-performance caching asynchronously processes request data, avoiding blocking the main thread under high load, thereby improving the response time and performance of large language model-based detection engines. Combining asynchronous large language models and high-performance caching in real-time detection technology provides an efficient, accurate, and adaptable network security solution, helping to address increasingly complex and evolving network threats.

[0135] Figure 6 This is a schematic diagram of the structure of an abnormal request detection device provided in an embodiment of this application, as shown below. Figure 6 As shown, the device includes:

[0136] The target payload acquisition module 610 is used to acquire target payload information from the target access request;

[0137] The target cache primary key determination module 620 is used to determine the target cache primary key based on the target payload information;

[0138] The detection result query module 630 is used to query the target detection result corresponding to the target cache key from the cache according to the target cache key. The cache stores the correspondence between the cache key and the detection result. The detection result is obtained in advance from the large language model according to the payload information corresponding to the cache key. The detection result includes normal requests or abnormal requests.

[0139] Optionally, the cached primary key determination module includes:

[0140] A compression unit is used to compress the target payload information to obtain compressed payload information;

[0141] The hash operation unit is used to perform a hash operation on the compressed payload information to obtain the target cache primary key.

[0142] Optionally, the compression unit is specifically used for:

[0143] Compress multiple consecutive identical characters in the target payload information into a single character to obtain preliminary compressed information;

[0144] Based on the character type of each character in the preliminary compression information, the characters in the preliminary compression information are replaced with generalized placeholders corresponding to the character type to obtain the compressed payload information.

[0145] Optionally, the device further includes:

[0146] An asynchronous query module is used to allow the target access request if the target cache primary key does not exist in the cache, and asynchronously obtain the target detection result corresponding to the target payload information from the large language model according to the target payload information.

[0147] The first caching module is used to save the target cache primary key and the target detection result corresponding to each other in the cache.

[0148] Optionally, the asynchronous query module includes:

[0149] The prompt information generation unit is used to add the target payload information to the prompt information template to obtain target prompt information. The prompt information template includes detection prompt information and return data format.

[0150] The query unit is used to input the target prompt information into the large language model, perform anomaly detection on the target payload information based on the detection prompt information through the large language model, and return the target detection result in the returned data format.

[0151] Optionally, the device further includes:

[0152] The dataset acquisition module is used to acquire the access request dataset;

[0153] The historical payload acquisition module is used to acquire historical payload information from each historical access request in the access request dataset.

[0154] The historical cache primary key determination module is used to determine the historical cache primary key based on the historical payload information.

[0155] The historical detection result acquisition module is used to acquire historical detection results corresponding to the historical payload information from the large language model based on the historical payload information.

[0156] The second caching module is used to save the historical cache primary key and the corresponding historical detection results into the cache.

[0157] Optionally, the target payload acquisition module includes:

[0158] A request type determination unit is used to obtain a request method from the target access request and determine the request type of the target access request based on the request method.

[0159] The payload acquisition unit is configured to acquire the target payload information from at least one of the URL, request header, and request body in the target access request, based on the request type.

[0160] The abnormal request detection device provided in this application embodiment is used to implement the steps of the abnormal request detection method described in this application embodiment. The specific implementation of each module of the device is described in the corresponding steps, and will not be repeated here.

[0161] The abnormal request detection device provided in this application obtains the payload information in the target access request, determines the target cache key based on the target payload information, and queries the target detection result corresponding to the target cache key from the cache based on the target cache key. The cache stores the correspondence between the cache key and the detection result. The detection result is obtained in advance from the large language model based on the payload information corresponding to the cache key. In this way, for real-time target access requests, it is not necessary to request the large language model in real time, but to query the pre-stored data from the cache, which can improve the detection efficiency of abnormal requests and improve the response speed of the detection engine. Moreover, compared with manually handling regular rules and manually extracting semantic features in conjunction with machine learning classifiers, it can generalize to detect network attacks of different languages ​​and types.

[0162] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 7 As shown, the electronic device 700 may include one or more processors 710 and one or more memories 720 connected to the processors 710. The electronic device 700 may also include an input interface 730 and an output interface 740 for communicating with another device or system. Program code executed by the processor 710 may be stored in the memory 720.

[0163] The processor 710 in the electronic device 700 calls the program code stored in the memory 720 to execute the abnormal request detection method in the above embodiment.

[0164] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the abnormal request detection method as described in this application.

[0165] This application also provides a computer program product that, when executed by a processor, implements the steps of the abnormal request detection method as described in this application.

[0166] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus embodiments, since they are fundamentally similar to the method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0167] The above provides a detailed description of an abnormal request detection method, apparatus, electronic device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

Claims

1. An abnormal request detection method, characterized in that, include: Obtain the target payload information from the target access request; Based on the target payload information, determine the target cache primary key; Based on the target cache key, query the target detection result corresponding to the target cache key from the cache. The cache contains the correspondence between the cache key and the detection result. The detection result is obtained in advance from the large language model based on the payload information corresponding to the cache key. The detection result includes normal requests or abnormal requests. The step of determining the target cache primary key based on the target payload information includes: Compress multiple consecutive identical characters in the target payload information into a single character to obtain preliminary compressed information; Based on the character type of each character in the preliminary compression information, the characters in the preliminary compression information are replaced with generalized placeholders corresponding to the character type to obtain the compressed payload information; The compressed payload information is hashed to obtain the target cache primary key.

2. The method according to claim 1, characterized in that, Also includes: If the target cache key does not exist in the cache, the target access request is allowed, and the target detection result corresponding to the target payload information is asynchronously obtained from the large language model according to the target payload information; The target cache primary key is saved in the cache along with the target detection result.

3. The method according to claim 2, characterized in that, The step of obtaining the target detection result corresponding to the target payload information from the large language model includes: The target payload information is added to the prompt information template to obtain the target prompt information. The prompt information template includes detection prompt information and return data format. The target prompt information is input into the large language model, and the large language model performs anomaly detection on the target payload information based on the detection prompt information, and returns the target detection result in the returned data format.

4. The method according to claim 1, characterized in that, Before retrieving the target detection result corresponding to the target cache primary key from the cache based on the target cache primary key, the method further includes: Obtain the access request dataset; For each historical access request in the access request dataset, obtain the historical payload information from the historical access request; Based on the historical payload information, determine the historical cache primary key; Based on the historical payload information, obtain the historical detection results corresponding to the historical payload information from the large language model; The historical cache primary key and the corresponding historical detection results are saved to the cache.

5. The method according to claim 1, characterized in that, The step of obtaining the target payload information in the target access request includes: Obtain the request method from the target access request, and determine the request type of the target access request based on the request method; Based on the request type, the target payload information is obtained from at least one of the URL, request header, and request body in the target access request.

6. An abnormal request detection device, characterized in that, include: The target payload acquisition module is used to acquire target payload information from the target access request; The target cache primary key determination module is used to determine the target cache primary key based on the target payload information; The detection result query module is used to query the target detection result corresponding to the target cache key from the cache according to the target cache key. The cache stores the correspondence between the cache key and the detection result. The detection result is obtained in advance from the large language model according to the payload information corresponding to the cache key. The detection result includes normal requests or abnormal requests. The target cache primary key determination module includes: A compression unit is used to compress multiple consecutive identical characters in the target payload information into a single character to obtain preliminary compressed information; and to replace the characters in the preliminary compressed information with generalized placeholders corresponding to the character types according to the character types to obtain compressed payload information. The hash operation unit is used to perform a hash operation on the compressed payload information to obtain the target cache primary key.

7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the abnormal request detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the abnormal request detection method according to any one of claims 1 to 5.

Images