An eBPF-based redis large key real-time detection and protection method and system

Abstract

The application provides an eBPF-based Redis large Key real-time detection and protection method and system. The method obtains Redis process information, probe mounting parameters and large Key judgment threshold, loads an eBPF program to a kernel and writes the eBPF program into a BPF Map to obtain a runtime monitoring configuration; based on the runtime monitoring configuration, a probe is mounted on a Redis command receiving, command processing, response construction and command returning path; when a client request arrives, a request buffer and a client connection context are read, a RESP length prefix is parsed to obtain a command type, a target key and a request data size; according to the command type, a write detection process or a read detection process is entered, a write protection strategy is generated and executed when the request data size exceeds a write threshold, and a large Key read event is generated when an accumulated response data size exceeds a read threshold, so that real-time detection, write protection and read alarm of a Redis large Key are realized.

Description

Technical Field

[0001] This application relates to the field of database operation monitoring and security protection technology, specifically to a method and system for real-time detection and protection of large keys in Redis based on eBPF. Background Technology

[0002] Redis is a high-performance, in-memory key-value database widely used in caching, session storage, high-concurrency access, and distributed business systems. During Redis operation, if the value object corresponding to a single key occupies too much memory, or if the key of a collection type such as Hash, List, Set, or ZSet contains too many elements, it can easily form a large key. Large keys consume a lot of memory during writes and increase the pressure on Redis object allocation. During reads, they generate large amounts of response data, which may lead to command execution blocking, increased network transmission pressure, bloated client output buffers, and decreased Redis service stability.

[0003] Existing Redis large key detection solutions typically include offline data analysis, command-line scanning, and bypass traffic analysis. Offline data analysis solutions parse Redis persistent files to calculate the size of each key-value object; command-line scanning solutions use Redis's scanning commands to traverse the key space and count key sizes or element counts; bypass traffic analysis solutions capture communication data between the Redis client and server, reconstruct and parse protocol data in user space, thereby identifying operations with large request or response data sizes. However, existing solutions struggle to detect and protect against large keys in real time.

[0004] Therefore, this application provides a method and system for real-time detection and protection of large keys in Redis based on eBPF, in order to solve one of the above-mentioned technical problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for real-time detection and protection of large keys in Redis based on eBPF, aiming to improve the real-time performance, proactive protection capabilities, and operational stability of large key detection in Redis.

[0006] This invention discloses a real-time detection and protection method for large keys in Redis based on eBPF, including: Obtain Redis process information, probe mounting parameters, and large key judgment threshold; load the eBPF program into the kernel and write it into BPFMap to obtain runtime monitoring configuration. Based on runtime monitoring configuration, probes are mounted on the Redis command receiving, command processing, response construction, and command return paths; When a client request arrives, the probe reads the request buffer and the client connection context, parses the RESP length prefix to obtain the command context, which includes the command type, target key and request data size; The command context is assigned to either the write detection process or the read detection process based on the command type. The write detection process compares the size of the requested data with the write threshold. If the write threshold is exceeded, a write protection policy is generated and executed based on the client connection context. The read detection process is based on the cumulative response fragment length of the response construction path to obtain the cumulative response data scale. When the command returns, the cumulative response data scale is compared with the read threshold. If the read threshold is exceeded, a large key read event is generated.

[0007] Preferably, Redis process information, probe mounting parameters, and large key determination threshold are obtained, the eBPF program is loaded into the kernel and written to the BPFMap to obtain the runtime monitoring configuration, including: Obtain the Redis process identifier, Redis executable file path, Redis service port, and Redis node identifier as Redis process information; Obtain the name of the function to be mounted, the function offset, and the probe type as probe mounting parameters; Get the write threshold, read threshold, collection element count threshold, and rate limiting time window as the large key judgment threshold; Write Redis process information, probe mounting parameters, and large key determination threshold into a BPFMap so that the eBPF program can read the runtime monitoring configuration during Redis operation.

[0008] Preferably, based on runtime monitoring configuration, probes are mounted on the Redis command receiving, command processing, response construction, and command return paths, including: The Redis symbol table is queried based on the Redis executable file path and the name of the function to be mounted to obtain the function offsets of the command receiving function, command processing function, and response constructor. Mount an uprobe at the entry point of the command receiving function and the command processing function to obtain the client request buffer and command processing context; Mount an uprobe at the entry point of the response constructor to obtain the response fragment length; mount an uretprobe at the return point of the command processing function to obtain the command processing end event.

[0009] Preferably, the probe corresponding to the command receiving function is used to obtain the request buffer address when Redis receives client requests, the entry probe corresponding to the command processing function is used to obtain the command processing context of the current command, the probe corresponding to the response constructor is used to obtain the length of the current response fragment written by Redis to the client output buffer, and the return probe corresponding to the command processing function is used to trigger command context cleanup and read threshold comparison.

[0010] Preferably, when a client request arrives, the probe reads the request buffer and the client connection context, parses the RESP length prefix, and obtains the command context, including: The probe reads the RESP array identifier and parameter count based on the client request buffer address; Read the length prefix and parameter content of each batch of strings in the order of parameters; determine the command type based on the first parameter content. The position of the target key in the RESP parameter sequence is determined based on the command type, and the corresponding parameter content is read as the target key. Determine the data scale statistics object based on the command type, and convert the length prefix corresponding to the data scale statistics object into the request data scale. Write the command type, target key, requested data size, thread identifier, process identifier, source IP, source port, and Redis node identifier into the command context table in the BPFMap.

[0011] Preferably, the data scale statistics object is determined according to the command type, and the length prefix corresponding to the data scale statistics object is converted into the requested data scale, including: When the command type is a string write command, the Value parameter is used as the data scale statistics object, and the byte length corresponding to the length prefix of the Value parameter is used to determine the requested data scale; When the command type is a container write command, the fields, members or elements written this time are used as data scale statistics objects, and the byte lengths of the length prefixes corresponding to the fields, members or elements are summed to obtain the requested data scale; When the command type is a read command, the requested data size is written to the command context table, and the response data size statistics status of the read command is initialized to zero.

[0012] Preferably, assigning the command context to either the write detection process or the read detection process based on the command type includes: SET, MSET, HSET, LPUSH, RPUSH, SADD, and ZADD are identified as write commands, and the corresponding command context is written to the write detection queue. GET, HGET, HGETALL, LRANGE, SMEMBERS, and ZRANGE are identified as read commands, and the corresponding command context is written to the read detection queue. If the command type does not match the write command set or the read command set, delete the corresponding command context and allow the current command to run.

[0013] Preferably, the write detection process compares the requested data size with a write threshold. If the write threshold is exceeded, a write protection policy is generated and executed based on the client connection context, including: Read the requested data size, target key, command type, and client connection context from the command context; Read the write threshold and collection element count threshold from the threshold configuration table in the BPFMap; When the requested data size exceeds the write threshold, or the number of fields, members, or elements written this time exceeds the set element size threshold, a write protection policy is generated that includes the target key, command type, requested data size, source IP, source port, and Redis node identifier. According to the write protection policy, the corresponding client connection is closed, the corresponding client connection is restricted from continuing to execute write commands within the rate limiting time window, or the blocking event is reported to the user-mode control program and the user-mode control program performs connection control.

[0014] Preferably, the read detection process is based on the cumulative response fragment length of the response construction path to obtain the cumulative response data size, and compares the cumulative response data size with the read threshold when the command returns. If the read threshold is exceeded, a large key read event is generated, including: When a read command enters the command processing entry point, the thread identifier, client connection identifier, command type, and target key are written to the read command context table; Each time a probe in the response construction path is triggered, the length of the current response fragment is read and added to the cumulative response statistics item corresponding to the thread identifier and client connection identifier to obtain the cumulative response data scale. When a probe in the command return path is triggered, the cumulative response data size is read from the cumulative response statistics and compared with the reading threshold. When the cumulative response data size exceeds the read threshold, a large key read event is generated, which includes the target key, command type, cumulative response data size, source IP, source port, and Redis node identifier.

[0015] This invention discloses a Redis large key real-time detection and protection system based on eBPF, used to execute the above-mentioned Redis large key real-time detection and protection method based on eBPF, including a user-space control module, a probe mounting module, a protocol parsing module, a command diversion module, a write protection module, a read statistics module, a threshold update module, and a context cleanup module. The user-space control module is used to obtain Redis process information, probe mounting parameters and large key judgment threshold, load the eBPF program into the kernel and write it into BPFMap to obtain runtime monitoring configuration; The probe mounting module is used to mount probes in the Redis command receiving, command processing, response construction, and command return paths based on runtime monitoring configurations. The protocol parsing module is used to read the request buffer and client connection context, parse the RESP length prefix, and obtain the command context; The command routing module is used to assign command contexts to either the write detection process or the read detection process based on the command type. The write protection module is used to generate and execute write protection policies when the requested data size exceeds the write threshold; The read statistics module is used to accumulate the length of response fragments and generate a large key read event when the accumulated response data size exceeds the read threshold; The threshold update module is used to update the threshold configuration in the BPFMap during Redis service operation; The context cleanup module is used to delete context records and statistics corresponding to completed commands.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This application provides a real-time detection and protection method for large keys in Redis based on eBPF. It establishes a runtime monitoring configuration by acquiring Redis process information, probe mounting parameters, and a large key detection threshold. eBPF probes are mounted on the Redis command reception, processing, response construction, and return paths, enabling continuous collection and correlation of client requests, RESP length prefixes, command context, and response fragment lengths during Redis operation. By comparing the request data size of write commands with the write threshold and generating a write protection strategy, it can block large key writes before they cause significant memory pressure. By accumulating the response fragment length of read commands and comparing it with the read threshold upon command return, it can identify large-scale read responses and report them promptly, thereby improving the real-time performance, proactive protection capabilities, and operational stability of Redis large key detection. Attached Figure Description

[0017] Figure 1This is a flowchart illustrating the real-time detection and protection method for large Redis keys based on eBPF provided by the present invention. Figure 2 The logical architecture diagram of the real-time detection and protection method for large keys in Redis based on eBPF provided by this invention; Figure 3 The flowchart of RESP protocol parsing and blocking based on eBPF provided by this invention; Figure 4 This is a schematic diagram of the Redis large key real-time detection and protection system based on eBPF provided by the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This application provides a real-time detection and protection method for large keys in Redis based on eBPF. The method dynamically monitors the command receiving path, command processing path, response construction path, and command return path during the Redis runtime process through an eBPF program. Furthermore, it uses BPFMap to transmit threshold configurations, command contexts, cumulative response statistics, and protection policies between the eBPF program and the user-space control program, enabling the identification and protection of large keys in Redis to be completed during the Redis service runtime.

[0020] In Redis, a large key refers to a key object whose value object occupies a large amount of memory, or a key containing a large number of fields, members or elements in container types such as Hash, List, Set, and ZSet. This may cause Redis memory allocation pressure, command execution blocking, client output buffer expansion or increased network transmission pressure during the writing or reading process.

[0021] The embodiments of this application are described in detail below with reference to the accompanying drawings.

[0022] Figure 1 A flowchart illustrating a real-time detection and protection method for large keys in Redis based on eBPF is shown, as follows: Figure 1 As shown, it includes the following steps; S101. Obtain Redis process information, probe mounting parameters, and large key judgment threshold. Load the eBPF program into the kernel and write it into BPFMap to obtain the runtime monitoring configuration.

[0023] S102. Mount probes to the Redis command receiving, command processing, response construction, and command return paths based on runtime monitoring configuration.

[0024] S103. When a client request arrives, the probe reads the request buffer and the client connection context, parses the RESP length prefix, and obtains the command context. The command context includes the command type, target key, and request data size.

[0025] S104. Assign the command context to the write detection process or the read detection process according to the command type.

[0026] For example, based on the actual allocation in S104, execute S105 or S106.

[0027] S105. Write detection process: Compare the size of the requested data with the write threshold. If the write threshold is exceeded, a write protection policy is generated and executed based on the client connection context.

[0028] S106, Read Detection Process: Based on the response, construct the path and accumulate the response fragment length to obtain the cumulative response data scale. When the command returns, compare the cumulative response data scale with the read threshold. If the read threshold is exceeded, generate a large key read event.

[0029] The eBPF-based real-time detection and protection method for large keys in Redis provided in this application forms a runtime monitoring configuration by acquiring Redis process information, probe mounting parameters, and large key judgment thresholds. eBPF probes are mounted on the Redis command reception, command processing, response construction, and command return paths, enabling continuous collection and correlation of client requests, RESP length prefixes, command context, and response fragment lengths during Redis operation. By comparing the request data size of write commands with the write threshold and generating a write protection strategy, writes can be blocked before they cause significant memory pressure. By accumulating the response fragment length of read commands and comparing it with the read threshold upon command return, large-scale read responses can be identified and reported promptly, thereby improving the real-time performance, proactive protection capabilities, and operational stability of Redis large key detection.

[0030] Figure 2 The diagram illustrates the logical architecture of a real-time detection and protection method for large keys in Redis based on eBPF.

[0031] In some embodiments, such as Figure 2As shown, the logical architecture of this application includes a user-space control program, a BPFMap, an eBPF program, a Redis process, and a client. The user-space control program obtains Redis process information, probe mounting parameters, and large key judgment thresholds, loads the eBPF program into the kernel, and writes the threshold configuration, probe mounting parameters, and runtime control parameters into the BPFMap. The BPFMap shares threshold configurations, command contexts, cumulative response statistics, client connection contexts, protection policies, and exception events between the user-space control program and the eBPF program. The eBPF program mounts the command receiving path, command processing path, response construction path, and command return path to the Redis process according to runtime monitoring configurations, and collects the request buffer, client connection context, and response fragment length when the probe is triggered. The Redis process receives client requests, executes Redis commands, and constructs response data. The client sends Redis requests to the Redis process and receives response data.

[0032] In the above embodiments, Redis process information is data used to locate the monitored Redis process and its running instance. Redis process information may include a Redis process identifier, the path to the Redis executable file, the Redis service port, and a Redis node identifier. The Redis process identifier is used to determine the target process that the eBPF program needs to attach to; the Redis executable file path is used to locate the symbol table in the Redis executable file; the Redis service port is used to distinguish multiple Redis instances on the same host; and the Redis node identifier is used to mark the Redis node to which the event belongs when reporting large key events. The user-space control program can obtain Redis process information through process querying, configuration file reading, or startup parameter reading, and write the Redis process information into a BPFMap so that the eBPF program can associate client requests with Redis instances at runtime.

[0033] In some embodiments, probe mounting parameters are data used to determine the mounting location of an eBPF program. Probe mounting parameters may include the name of the function to be mounted, the function offset, and the probe type. The name of the function to be mounted is used to determine the Redis user-space function to be monitored; the function offset is used to locate the specific mounting address in the Redis executable file; and the probe type is used to distinguish between entry probes and return probes. The large key determination threshold is configuration data used to determine whether the current command involves large key operations. The large key determination threshold may include a write threshold, a read threshold, a set element count threshold, and a rate-limiting time window. The write threshold is used to determine whether the requested data size of a write command is too large; the read threshold is used to determine whether the cumulative response data size of a read command is too large; the set element count threshold is used to determine whether the number of fields, members, or elements written in this container-type command is too large; and the rate-limiting time window is used to limit the duration for which abnormal client connections are restricted from continuing to initiate write commands.

[0034] As a feasible implementation, when the user-space control program executes the runtime monitoring configuration establishment process, it obtains the Redis process identifier, Redis executable file path, Redis service port, and Redis node identifier as Redis process information; it obtains the name of the function to be mounted, function offset, and probe type as probe mounting parameters; it obtains the write threshold, read threshold, set element count threshold, and rate limiting time window as large key judgment thresholds; and it writes the Redis process information, probe mounting parameters, and large key judgment thresholds into a BPFMap so that the eBPF program can read the runtime monitoring configuration during Redis operation. Through this processing method, the eBPF program does not need to fix the Redis instance, function offset, and threshold parameters at compile time, but instead reads the current configuration from the BPFMap during Redis service operation, thereby adapting to different Redis instances and different threshold strategies.

[0035] In some embodiments, the BPFMap is used to store data that the eBPF program needs to read or update during runtime. The BPFMap may include a threshold configuration table, a command context table, a read command context table, a cumulative response statistics table, a client connection context table, a protection policy table, and an exception event buffer. The threshold configuration table stores write thresholds, read thresholds, set element count thresholds, and rate limiting time windows; the command context table stores command type, target key, requested data size, thread identifier, process identifier, and client connection identifier; the read command context table stores the thread identifier, client connection identifier, command type, and target key corresponding to read commands; the cumulative response statistics table stores the cumulative response data size formed during multiple response constructions of the same read command; the client connection context table stores the source IP, source port, Redis node identifier, and connection establishment time; the protection policy table stores client connections that need to be closed, rate-limited, or subject to connection control; and the exception event buffer reports large key write events, large key read events, and protection execution results to the user-space control program.

[0036] As a specific implementation, the user-space control program loads the eBPF program into the kernel via libbpf and mounts the Redis user-space execution path based on runtime monitoring configuration. This process includes: querying the Redis symbol table based on the Redis executable file path and the name of the function to be mounted to obtain the function offsets of the command receiving function, command processing function, and response constructor; mounting uprobe at the entry points of the command receiving and command processing functions to obtain the client request buffer and command processing context; mounting uprobe at the entry point of the response constructor to obtain the response fragment length; and mounting uretprobe at the return point of the command processing function to obtain the command processing end event. Here, uprobe is a probe used to trigger the eBPF program at the user-space function entry point, and uretprobe is a probe used to trigger the eBPF program at the user-space function return point. Through the cooperation of the entry probe and the return probe, the eBPF program can collect interrelated data at different stages of command receiving, command execution, response construction, and command end.

[0037] As a specific implementation, the command receiving path, command processing path, and response construction path in Redis can be located through Redis internal functions.

[0038] Specifically, the probe corresponding to the command receiving function is used to obtain the request buffer address when Redis receives client requests, the entry probe corresponding to the command processing function is used to obtain the command processing context of the current command, the probe corresponding to the response constructor is used to obtain the length of the current response fragment written by Redis to the client output buffer, and the return probe corresponding to the command processing function is used to trigger command context cleanup and read threshold comparison.

[0039] In this embodiment, the entry probe at the command receiving function is used to obtain the buffer containing the original request data sent by the client to Redis; the entry probe at the command processing function is used to associate the request data with the current command processing process in Redis; the entry probe at the response constructor is used to obtain the length of the data written to the client output buffer when Redis constructs the response data; and the return probe at the command processing function is used to confirm that the current command processing is complete and to trigger the comparison of the cumulative response data scale of read commands and context cleanup.

[0040] In some embodiments, the command receiving function is the readQueryFromClient function, the command processing function is the processCommand function, and the response constructor is the addReply function.

[0041] In some embodiments, the request buffer is a memory area used by Redis to store request data after receiving a client request, and the request buffer address is address data pointing to this memory area. The client connection context is data used to identify the request source and the target Redis instance, and may include the source IP, source port, target Redis node identifier, and connection establishment time. When the probe is triggered, the eBPF program reads the request buffer address and the client connection context through function parameters, register parameters, or the Redis command processing context, and writes the client connection context into a BPFMap so that the source of abnormal operations can be traced when generating write protection policies or large key read events.

[0042] In some embodiments, the RESP protocol is a protocol format for command and response interaction between Redis clients and Redis servers. Client requests are typically represented as RESP arrays, where each parameter is represented by a batch string containing a length prefix field indicating the parameter's byte length. When the eBPF program parses the RESP length prefix, it does not need to completely deserialize the Redis command object. Instead, it reads the number of parameters, the parameter order, and the length prefix and content of each parameter to obtain the command type, target key, and requested data size. This approach allows for determination before Redis actually allocates large values ​​or generates large-scale responses, reducing the impact on the Redis business execution path.

[0043] As a feasible implementation, the command context generation process includes: the probe reads the RESP array identifier and parameter count based on the client request buffer address; reads the length prefix and parameter content of each batch of strings in parameter order; determines the first parameter content as the command type; determines the position of the target key in the RESP parameter sequence based on the command type, and reads the corresponding parameter content as the target key; determines the data scale statistics object based on the command type, and converts the length prefix corresponding to the data scale statistics object into the requested data scale; and writes the command type, target key, requested data scale, thread identifier, process identifier, source IP, source port, and Redis node identifier into the command context table in the BPFMap. Specifically, the command type is used to distinguish whether the current command is a write command, a read command, or another command that does not require detection; the target key is used to locate the Redis key operated on by the current command; the requested data scale is used to determine whether a write command is likely to result in a large key write; and the thread identifier and client connection identifier are used to associate data collected by the same command at different probe triggering stages to the same context.

[0044] In some embodiments, the data size statistics object is a parameter object used to calculate the requested data size, determined based on the command type. For string write commands, the data size statistics object is the Value parameter; for container write commands, the data size statistics object is the field, member, or element being written; for read commands, the requested data size is usually not the primary basis for determining large key reads, but the requested data size, command type, and target key can be written to the command context table, and the response data size statistics status can be initialized to zero so that the response fragment length can be continuously accumulated in the response construction path.

[0045] As a specific implementation, when determining the requested data size based on the command type, the eBPF program uses the Value parameter as the data size statistics object when the command type is a string write command, and determines the requested data size by the byte length corresponding to the length prefix of the Value parameter; when the command type is a container write command, the fields, members, or elements written this time are used as the data size statistics object, and the byte lengths of the length prefixes corresponding to the fields, members, or elements are summed to obtain the requested data size; when the command type is a read command, the requested data size is written to the command context table, and the response data size statistics status of the read command is initialized to zero.

[0046] In the above embodiments, the request data size of the string write command can be expressed as: ; in, Indicates the size of the requested data. Indicates the length in bytes of the Value parameter.

[0047] The data size requested by the container's write command can be represented as: ; in, Indicates the size of the requested data. Indicates the byte length of the i-th field, member, or element. This indicates the number of fields, members, or elements written in this instance.

[0048] Figure 3 A flowchart illustrating the parsing and blocking process of the RESP protocol based on eBPF is shown.

[0049] In some embodiments, such as Figure 3 As shown, the RESP protocol parsing and write protection process of this application is used to perform lightweight parsing of the RESP protocol data in the request buffer after Redis receives a client request, and to determine whether to perform write protection based on the parsed command context. After the probe is triggered, the eBPF program reads the request buffer address, client connection context, and runtime monitoring configuration; based on the request buffer address, it reads the RESP array identifier, number of parameters, batch string length prefix, and parameter content; it determines the first parameter content as the command type, determines the position of the target key parameter based on the command type, and determines the request data scale based on the length prefix of the Value parameter, field parameter, member parameter, or element parameter; and it writes the command type, target key, request data scale, thread identifier, process identifier, source IP, source port, and Redis node identifier into the command context table. If the command type is a write command, the eBPF program reads the write threshold and the set element count threshold from the threshold configuration table, and compares the requested data size, the number of fields written, the number of members, or the number of elements with the corresponding threshold. When the requested data size exceeds the write threshold, or the number of fields, members, or elements written exceeds the set element count threshold, a write protection policy is generated based on the client connection context, and connection closure, rate limiting, or blocking event reporting is executed. When the corresponding threshold is not exceeded, the current command is allowed and a normal execution record is retained. After the command is processed, the eBPF program deletes the command context and temporary protection state corresponding to the current command based on the thread identifier and the client connection identifier to prevent subsequent requests from misusing the context data of the previous command.

[0050] In some embodiments, command type identification is used to assign a command context to the corresponding detection process. The command type is determined by the content of the first parameter in the RESP request. After obtaining the command type, the eBPF program determines whether the current command is a write command or a read command based on a preset command set. A write command is a command that may write a large value or many fields, members, or elements to Redis; a read command is a command that may read a large value or a large number of fields, members, or elements from Redis and generate large-scale response data.

[0051] As a specific embodiment, the command routing process includes: identifying SET, MSET, HSET, LPUSH, RPUSH, SADD, and ZADD as write commands and writing their corresponding command contexts to a write detection queue; identifying GET, HGET, HGETALL, LRANGE, SMEMBERS, and ZRANGE as read commands and writing their corresponding command contexts to a read detection queue; when a command type does not match either the write command set or the read command set, deleting the corresponding command context and allowing the current command to proceed. The write detection queue can be implemented using write detection status or flags in a BPFMap, and the read detection queue can also be implemented using read detection status or flags in a BPFMap. Through this routing method, this application can perform detection on commands that may pose a large key risk and reduce processing overhead for commands that do not involve large key risks.

[0052] In some embodiments, the write detection process is used to identify and handle abnormal write requests before storing large values ​​or a large number of container elements in Redis. The write detection process uses the requested data size, target key, command type, and client connection context from the command context, and reads the write threshold and set element count threshold from the BPFMap. The write threshold is used to determine whether the data to be written by the current command exceeds the allowed range, and the set element count threshold is used to determine whether the number of fields, members, or elements written by the container type command exceeds the allowed range. When either condition is triggered, the eBPF program generates a write protection policy and writes it to the protection policy table. The user-space control program or connection control path then executes connection closure, connection rate limiting, or connection control according to the protection policy.

[0053] As a feasible embodiment, the write protection process includes: reading the requested data size, target key, command type, and client connection context from the command context; reading the write threshold and set element count threshold from the threshold configuration table of BPFMap; when the requested data size exceeds the write threshold, or the number of fields, members, or elements written this time exceeds the set element count threshold, generating a write protection policy containing the target key, command type, requested data size, source IP, source port, and Redis node identifier; closing the corresponding client connection, restricting the corresponding client connection from continuing to execute write commands within the rate-limiting time window, or reporting the blocking event to the user-space control program and having the user-space control program perform connection control according to the write protection policy. Specifically, the target key in the write protection policy is used to mark the protected Redis key; the command type is used to describe the specific Redis command that triggered the protection; the requested data size is used to describe the numerical basis of the trigger threshold; the source IP and source port are used to locate the abnormal client connection; and the Redis node identifier is used to locate the Redis instance where the abnormal write occurred.

[0054] As a specific implementation, when the protection policy adopts the connection closure method, the user-space control program locates the corresponding client connection based on the source IP, source port, and Redis node identifier, and closes the client connection, preventing it from continuing to write the current large key to Redis or sending subsequent write commands. When the protection policy adopts the rate limiting method, the eBPF program or user-space control program writes the corresponding client connection to the rate limiting table and rejects the client connection from initiating new write commands within the rate limiting time window. When the protection policy adopts the return value overriding method, the target mount point is configured as a kernel probe that supports return value overriding. The return value overriding probe calls the bpf_override_return helper function to set the return result of the corresponding command processing path to an error code, causing the current command processing to end prematurely. The return value overriding method can be considered as a specific mode in write protection policies, while connection closure and rate limiting methods can be considered as connection-level protection modes that do not depend on modifications to the Redis source code.

[0055] For example, with a write threshold configured to 1MB, a client sends a SET command to Redis. The target key in the request content is `big_key`, and the length prefix of the Value parameter indicates that the Value byte length is 10,485,760 bytes. The eBPF program reads the request buffer from the command receiving path, parses the RESP length prefix to obtain the command type SET, the target key `big_key`, and the requested data size of 10,485,760 bytes. The eBPF program reads the write threshold of 1MB from the threshold configuration table, compares the requested data size with the write threshold, determines that the current write command exceeds the write threshold, and generates a write protection policy containing `big_key`, SET, 10,485,760 bytes, source IP, source port, and Redis node identifier. Based on this write protection policy, the user-space control program closes the corresponding client connection or performs rate limiting, and receives large key write events through the exception event buffer.

[0056] In some embodiments, the read detection process is used to identify large-scale response data generated by read commands. Unlike write commands, read commands typically request smaller amounts of data, but Redis may return a large amount of data from a large key after executing such a command. Since Redis may not generate a complete response all at once when constructing the response data, but instead writes response fragments to the client's output buffer multiple times through the response constructor, this application uses probes in the response construction path to read the length of the response fragments multiple times, and uses thread identifiers and client connection identifiers to associate multiple response fragments with the same read command.

[0057] As a feasible implementation, the read detection process includes: when a read command enters the command processing entry point, writing the thread identifier, client connection identifier, command type, and target key into the read command context table; each time a probe in the response construction path is triggered, reading the length of the current response fragment and accumulating it into the cumulative response statistics item corresponding to the thread identifier and client connection identifier to obtain the cumulative response data scale; when a probe in the command return path is triggered, reading the cumulative response data scale from the cumulative response statistics item and comparing it with the read threshold; when the cumulative response data scale exceeds the read threshold, generating a large key read event containing the target key, command type, cumulative response data scale, source IP, source port, and Redis node identifier. The thread identifier is used to distinguish the currently executing thread in Redis, and the client connection identifier is used to distinguish the client connection that initiated the command; both are used together to ensure that response fragments obtained from multiple response constructions can be accurately accumulated into the cumulative response statistics item corresponding to the same read command.

[0058] In the above embodiments, the cumulative response data scale can be expressed as: ; in, Indicates the cumulative response data size. This represents the length in bytes of the response fragment constructed in the j-th response. This indicates the number of times the same read command triggers the response construction path. This formula illustrates that the response data size of a read command is not determined by the length of a single response, but rather by the sum of the lengths of multiple response segments within the same command execution cycle.

[0059] For example, with a read threshold configured to 10MB, and a pre-existing hash key containing approximately 500,000 fields in Redis, the client sends an HGETALL command to Redis to read the entire contents of this hash key. The eBPF program records the thread identifier, client connection identifier, command type HGETALL, and target key at the command processing entry point. When Redis executes the HGETALL command, it calls the addReply function multiple times to construct RESP response fragments. The eBPF program reads the length of this response fragment each time the addReply function entry probe is triggered and adds this length to the cumulative response statistics. When the processCommand function returns, the eBPF program reads the cumulative response data size and compares it with the read threshold of 10MB. If the cumulative response data size exceeds 10MB, a large key read event is generated, containing the target key, HGETALL, cumulative response data size, source IP, source port, and Redis node identifier, and sent to the user-space control program through the exception event buffer. Through this processing, this application can identify large key read behavior during read-type command execution and provide a data foundation for subsequent alarms, auditing, and fault location.

[0060] In some embodiments, the user-space control program supports adjusting the large key detection threshold while the Redis service is running. This threshold adjustment does not require stopping the Redis service or reloading the eBPF application. After receiving the threshold adjustment instruction, the user-space control program writes the adjusted write threshold, read threshold, set element count threshold, or rate limiting time window to the threshold configuration table in the BPFMap. When processing subsequent client requests, the eBPF application reads the current threshold configuration from the threshold configuration table and executes the write detection process or read detection process according to the updated threshold configuration.

[0061] As a specific embodiment, the runtime threshold adjustment process includes: the user-space control program receives a threshold adjustment instruction during the Redis service operation; updates the write threshold, read threshold, set element count threshold, or rate limiting time window in the BPFMap according to the threshold adjustment instruction; the eBPF program reads the updated threshold configuration when processing subsequent client requests and executes the write detection process or read detection process according to the updated threshold configuration; wherein, when the write threshold is configured to 1MB, write commands requesting data exceeding 1MB trigger a write protection policy; when the read threshold is configured to 10MB, read commands with a cumulative response data exceeding 10MB trigger a large key read event. For example, if the initial write threshold is 1MB, and the user-space control program updates the write threshold to 2MB, the eBPF program reads the updated 2MB write threshold in subsequent write command processing. When a client writes a 1.5MB value, the requested data size does not exceed the updated write threshold, and the corresponding write command is allowed; when a client writes a 3MB value, the requested data size exceeds the updated write threshold, and the corresponding write command triggers a write protection policy.

[0062] In some embodiments, to ensure that command context, cumulative response statistics, and protection policies do not remain during continuous operation, this application incorporates context cleanup processing in the write detection process, read detection process, and command return path. Context cleanup processing deletes command contexts, read command contexts, and cumulative response statistics that have been used by the current command, and deletes corresponding protection policy records after the client connection is closed or the rate-limiting time window expires. Through this processing, the same thread or the same client connection will not misuse the context data of the previous command when processing subsequent commands.

[0063] As a feasible embodiment, the context cleanup process includes: after the write detection process ends, the read detection process ends, or a probe in the command return path is triggered, querying the command context table, read command context table, and cumulative response statistics based on the thread identifier and client connection identifier; deleting the command context, read command context, and cumulative response statistics corresponding to the current command; and deleting the protection policy record corresponding to the client connection in the protection policy table when the corresponding client connection is closed or the rate limiting time window expires. The thread identifier and client connection identifier serve as indexes for context cleanup, enabling the eBPF program to accurately locate the data record corresponding to the current command. For write commands, after the write detection process is completed and a release result or protection policy is generated, the record corresponding to the write command in the command context table can be deleted; for read commands, after the cumulative response data scale comparison and large key read event generation are completed in the command return path, the record corresponding to the read command in the read command context table and cumulative response statistics can be deleted.

[0064] In some embodiments, this application also provides a combination of hardware and program modules for performing the above-described methods. This combination of hardware and program modules can be deployed on a Linux host running Redis, or on an operations and maintenance node with Redis process access permissions and eBPF program loading permissions. It includes a user-space control module, a probe mounting module, a protocol parsing module, a command routing module, a write protection module, a read statistics module, a threshold update module, and a context cleanup module. Each module can be implemented by a program running in user space, an eBPF program loaded into the kernel, and shared data structures in the BPFMap.

[0065] As a specific embodiment, the user-space control module is used to obtain Redis process information, probe mounting parameters, and large key judgment threshold, load the eBPF program into the kernel and write it into the BPFMap to obtain the runtime monitoring configuration; the probe mounting module is used to mount probes in the Redis command reception, command processing, response construction, and command return paths based on the runtime monitoring configuration; the protocol parsing module is used to read the request buffer and client connection context, parse the RESP length prefix, and obtain the command context; the command routing module is used to allocate the command context to the write detection process or the read detection process according to the command type; the write protection module is used to generate and execute the write protection policy when the requested data size exceeds the write threshold; the read statistics module is used to accumulate the response fragment length and generate a large key read event when the accumulated response data size exceeds the read threshold; the threshold update module is used to update the threshold configuration in the BPFMap during the Redis service operation; and the context cleanup module is used to delete the context records and statistics items corresponding to completed commands. The user-space control module can run in the user-space process and is used to load the eBPF program, configure the threshold, and receive events. The probe mounting module, protocol parsing module, command routing module, write protection module, and read statistics module can be implemented through the eBPF program. The threshold update module and context cleanup module can be implemented together by the user-space control program and the eBPF program.

[0066] In some embodiments, the exception event buffer in this application can be implemented using RingBuffer or PerfEventBuffer. After generating a large key write event, a large key read event, or a protection execution result, the eBPF program writes the event data to the exception event buffer. The user-space control program reads the event data in the exception event buffer and outputs alarm records, audit records, or protection execution records according to the event type. The event data may include the event type, Redis node identifier, command type, target key, requested data size, cumulative response data size, source IP, source port, trigger threshold, and event timestamp. Through this event data, operations personnel can locate the client connection, Redis node, and target key that generated the large key operation, and determine whether the operation belongs to a write risk or a read risk based on the event type.

[0067] In some embodiments, this application does not rely on Redis keyspace scanning, nor does it require waiting for Redis to generate RDB persistent files before offline analysis. The eBPF program obtains the request buffer, command context, and response fragment length through the Redis runtime execution path, ensuring that large key detection occurs during Redis command execution. For write commands, this application generates a write protection policy when the size of the data to be written exceeds the write threshold, thereby reducing the impact of large value writes on Redis memory allocation. For read commands, this application accumulates multiple response fragment lengths during response construction and determines the accumulated response data size when the command returns, thereby identifying large-scale response data generated by a single read operation. Through the above processing, this application can achieve runtime detection, write protection, read alerts, dynamic threshold adjustment, and context closed-loop management of large keys in Redis without modifying the Redis source code or restarting the Redis service.

[0068] This application also provides system embodiments that follow the above embodiments, for implementing the method steps of the above embodiments. The interpretation of the same names is the same as that of the above embodiments, and they have the same technical effects as those of the above embodiments, so they will not be repeated here.

[0069] like Figure 4 As shown, this application provides a Redis large key real-time detection and protection system based on eBPF, including: like Figure 4 As shown, the structural diagram of the Redis large key real-time detection and protection system based on eBPF includes a user-space control module 401, a probe mounting module 402, a protocol parsing module 403, a command diversion module 404, a write protection module 405, a read statistics module 406, a threshold update module 407, and a context cleanup module 408. User-space control module 401 is used to obtain Redis process information, probe mounting parameters and large key judgment threshold, load the eBPF program into the kernel and write it into BPFMap to obtain runtime monitoring configuration; The probe mounting module 402 is used to mount probes in the Redis command receiving, command processing, response construction and command return paths based on runtime monitoring configuration; Protocol parsing module 403 is used to read the request buffer and client connection context, parse the RESP length prefix, and obtain the command context; Command routing module 404 is used to assign command contexts to the write detection process or the read detection process according to the command type; The write protection module 405 is used to generate and execute a write protection policy when the requested data size exceeds the write threshold; The reading statistics module 406 is used to accumulate the length of the response fragment and generate a large key read event when the accumulated response data size exceeds the reading threshold. The threshold update module 407 is used to update the threshold configuration in the BPFMap during the operation of the Redis service; The context cleanup module 408 is used to delete the context records and statistics corresponding to completed commands.

[0070] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0071] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for real-time detection and protection of large keys in Redis based on eBPF, characterized in that, include: Obtain Redis process information, probe mounting parameters, and large key judgment threshold; load the eBPF program into the kernel and write it into BPFMap to obtain runtime monitoring configuration. Based on runtime monitoring configuration, probes are mounted on the Redis command receiving, command processing, response construction, and command return paths; When a client request arrives, the probe reads the request buffer and the client connection context, parses the RESP length prefix, and obtains the command context, which includes the command type, target key, and request data size. The command context is assigned to either the write detection process or the read detection process based on the command type. The write detection process compares the size of the requested data with the write threshold. If the write threshold is exceeded, a write protection policy is generated and executed based on the client connection context. The read detection process is based on the cumulative response fragment length of the response construction path to obtain the cumulative response data scale. When the command returns, the cumulative response data scale is compared with the read threshold. If the read threshold is exceeded, a large key read event is generated.

2. The method according to claim 1, characterized in that, The process involves obtaining Redis process information, probe mounting parameters, and large key determination thresholds, loading the eBPF program into the kernel, and writing it into the BPFMap to obtain the runtime monitoring configuration, including: Obtain the Redis process identifier, Redis executable file path, Redis service port, and Redis node identifier as Redis process information; Obtain the name of the function to be mounted, the function offset, and the probe type as probe mounting parameters; Get the write threshold, read threshold, collection element count threshold, and rate limiting time window as the large key judgment threshold; The Redis process information, probe mounting parameters, and large key determination threshold are written into the BPFMap so that the eBPF program can read the runtime monitoring configuration during Redis operation.

3. The method according to claim 2, characterized in that, The runtime monitoring configuration includes mounting probes on the Redis command reception, command processing, response construction, and command return paths, including: The Redis symbol table is queried based on the Redis executable file path and the name of the function to be mounted to obtain the function offsets of the command receiving function, command processing function, and response constructor. Mount an uprobe at the entry point of the command receiving function and the command processing function to obtain the client request buffer and command processing context; Mount an uprobe at the entry point of the response constructor to obtain the response fragment length; mount an uretprobe at the return point of the command processing function to obtain the command processing end event.

4. The method according to claim 3, characterized in that, The probe corresponding to the command receiving function is used to obtain the request buffer address when Redis receives client requests. The entry probe corresponding to the command processing function is used to obtain the command processing context of the current command. The probe corresponding to the response constructor function is used to obtain the length of the current response fragment written by Redis to the client output buffer. The return probe corresponding to the command processing function is used to trigger command context cleanup and read threshold comparison.

5. The method according to claim 1, characterized in that, When the client request arrives, the probe reads the request buffer and the client connection context, parses the RESP length prefix, and obtains the command context, including: The probe reads the RESP array identifier and parameter count based on the client request buffer address; Read the length prefix and parameter content of each batch of strings in the order of parameters; determine the command type based on the first parameter content. The position of the target key in the RESP parameter sequence is determined based on the command type, and the corresponding parameter content is read as the target key. The data scale statistics object is determined according to the command type, and the length prefix corresponding to the data scale statistics object is converted into the request data scale. Write the command type, target key, requested data size, thread identifier, process identifier, source IP, source port, and Redis node identifier into the command context table in the BPFMap.

6. The method according to claim 5, characterized in that, The step of determining the data scale statistics object based on the command type and converting the length prefix corresponding to the data scale statistics object into the request data scale includes: When the command type is a string write command, the Value parameter is used as the data scale statistics object, and the byte length corresponding to the length prefix of the Value parameter is used to determine the requested data scale; When the command type is a container write command, the fields, members or elements written this time are used as data scale statistics objects, and the byte lengths of the length prefixes corresponding to the fields, members or elements are summed to obtain the requested data scale; When the command type is a read command, the requested data size is written to the command context table, and the response data size statistics of the read command are initialized to zero.

7. The method according to claim 1, characterized in that, The step of assigning the command context to the write detection process or the read detection process based on the command type includes: SET, MSET, HSET, LPUSH, RPUSH, SADD, and ZADD are identified as write commands, and the corresponding command context is written to the write detection queue. GET, HGET, HGETALL, LRANGE, SMEMBERS, and ZRANGE are identified as read commands, and the corresponding command context is written to the read detection queue. If the command type does not match the write command set or the read command set, delete the corresponding command context and allow the current command to run.

8. The method according to claim 1, characterized in that, The write detection process compares the requested data size with a write threshold. If the write threshold is exceeded, a write protection policy is generated and executed based on the client connection context, including: Read the requested data size, target key, command type, and client connection context from the command context; Read the write threshold and collection element count threshold from the threshold configuration table in the BPFMap; When the requested data size exceeds the write threshold, or the number of fields, members, or elements written this time exceeds the set element size threshold, a write protection policy is generated that includes the target key, command type, requested data size, source IP, source port, and Redis node identifier. According to the write protection policy, the corresponding client connection is closed, the corresponding client connection is restricted from continuing to execute write commands within the rate limiting time window, or the blocking event is reported to the user-mode control program and the user-mode control program performs connection control.

9. The method according to claim 1, characterized in that, The read detection process is based on the cumulative response fragment length of the response construction path to obtain the cumulative response data size. Upon command return, the cumulative response data size is compared with a read threshold. If the read threshold is exceeded, a large key read event is generated, including: When a read command enters the command processing entry point, the thread identifier, client connection identifier, command type, and target key are written to the read command context table; Each time a probe in the response construction path is triggered, the length of the current response fragment is read and added to the cumulative response statistics item corresponding to the thread identifier and client connection identifier to obtain the cumulative response data scale; When a probe in the command return path is triggered, the cumulative response data size is read from the cumulative response statistics and compared with the reading threshold. When the cumulative response data size exceeds the read threshold, a large key read event is generated, which includes the target key, command type, cumulative response data size, source IP, source port, and Redis node identifier.

10. A Redis large key real-time detection and protection system based on eBPF, characterized in that, The method for performing any one of claims 1 to 9 includes a user-space control module, a probe mounting module, a protocol parsing module, a command routing module, a write protection module, a read statistics module, a threshold update module, and a context cleanup module. The user-space control module is used to obtain Redis process information, probe mounting parameters and large key judgment threshold, load the eBPF program into the kernel and write it into BPFMap to obtain runtime monitoring configuration; The probe mounting module is used to mount probes in the Redis command receiving, command processing, response construction and command return paths based on runtime monitoring configuration; The protocol parsing module is used to read the request buffer and client connection context, parse the RESP length prefix, and obtain the command context; The command routing module is used to allocate command contexts to the write detection process or the read detection process according to the command type; The write protection module is used to generate and execute a write protection policy when the requested data size exceeds the write threshold. The read statistics module is used to accumulate the length of the response fragment and generate a large key read event when the accumulated response data size exceeds the read threshold. The threshold update module is used to update the threshold configuration in BPFMap during the operation of the Redis service; The context cleanup module is used to delete the context records and statistics corresponding to completed commands.

Images