Method, device, medium and product for dynamic data desensitization facing gigabit traffic

Abstract

A kind of dynamic data desensitization method, equipment, medium and product for gigabit flow.In the method, input message is acquired and parsed, and load characteristics are extracted;Matching is performed on the load characteristics, and a hit state bitmap is generated based on the matching result;The input message is aggregated with the hit state bitmap to generate metadata to be processed, and the metadata to be processed is written into a large page memory space;The metadata to be processed is read, and a corresponding set of network throughput performance parameters is collected;According to the set of network throughput performance parameters, a throughput peak prediction parameter is obtained, a scheduling instruction is generated, the number of physical cores is adjusted based on the scheduling instruction, and an active core mapping table is updated;The metadata to be processed is distributed to the target processing core in the target thread pool;The hit state bitmap contained in the metadata to be processed is read, the plaintext sensitive field is determined, the character replacement processing is performed, and the desensitization message is generated by combination. The technical solution provided by the present application prevents sensitive data leakage and provides bottom protection.

Description

Technical Field

[0001] This application relates to the field of network data security processing technology, specifically to a dynamic data desensitization method, device, medium, and product for 10 Gigabit traffic. Background Technology

[0002] Existing network data anonymization schemes employ a pure software architecture. After packets are received via the operating system kernel protocol stack, a software process running in user space performs regular expression matching on the packet payload on the central processing unit (CPU) to identify sensitive information fields contained in the packet. Character mask replacement is then performed on the matched fields, and the processed packet is finally forwarded to the downstream analysis platform. In this scheme, the number of processing cores and the rule set are configured statically, and regular expression matching calculations and packet transmission / reception processing are executed serially on the processor cores.

[0003] However, the above solution has the following technical problems: the software regular expression matching throughput of the single-core processor combined with the kernel protocol stack is insufficient, resulting in serious packet loss in the 10 Gigabit traffic environment. Furthermore, the static core binding strategy cannot dynamically adjust processing resources according to traffic peaks and valleys, causing latency jitter during the de-identification process during peak traffic. At the same time, there is a lack of fallback protection when the de-identification process fails, and the original sensitive data that has not been de-identified is at risk of being directly leaked to downstream platforms. Summary of the Invention

[0004] To address the aforementioned technical issues, this application provides a method, device, medium, and product for dynamic data desensitization for 10 Gigabit Ethernet traffic.

[0005] The first aspect of this application provides a dynamic data anonymization method for 10 Gigabit Ethernet traffic: The system acquires input packets received at the network port, performs protocol parsing on the input packets, and extracts the payload characteristics contained in the input packets. The preset FPGA hardware processing engine is invoked to perform parallel regular feature matching on the load features, and a hit status bitmap is generated based on the matching start offset and matching byte length in the matching result. The input message and the hit status bitmap are aggregated to generate metadata to be processed, and the metadata to be processed is written into a preset large page memory space through a preset direct memory access channel. The data to be processed is read from the large page memory space by the preset data plane development kit polling driver, and the network throughput performance parameter set corresponding to the current processing cycle is collected by the data plane development kit polling driver. The network throughput performance parameter set is input into a preset long short-term memory network traffic prediction model, and the peak throughput prediction parameter is output. A scheduling instruction is generated based on the difference between the peak throughput prediction parameter and the preset capacity assessment parameter. The number of physical cores in the preset active state in the preset worker thread pool is adjusted based on the scheduling instruction, and the preset active core mapping table is updated synchronously. Based on the preset receiver scaling algorithm and the active core mapping table, the metadata to be processed is distributed to the target processing cores in the worker thread pool as determined by the scheduling instruction. The target processing core reads the hit status bitmap contained in the metadata to be processed, determines the plaintext sensitive fields marked by the hit status bitmap, performs character replacement processing based on preset masking rules on the plaintext sensitive fields, and combines them to generate a de-identified message.

[0006] By adopting the above technical solutions, the hardware parallel regularization matching of FPGA (Field-Programmable Gate Array) is used to replace the pure software implementation, breaking through the 10 Gigabit throughput bottleneck and eliminating packet loss; combined with the DPDK (Data Plane Development Kit) and LSTM (Long Short-Term Memory) dynamic scaling processing core, peak and valley latency jitter is eliminated; the bitmap-driven desensitization process ensures that only masked packets are forwarded, preventing sensitive data leakage and providing fallback protection.

[0007] Optionally, the step of invoking a preset FPGA hardware processing engine to perform parallel regular feature matching on the load features, and generating a hit status bitmap based on the matching start offset and matching byte length in the matching result, includes: Read the regular expression rule set from the built-in rule storage area of ​​the FPGA hardware processing engine; The load features and the regular expression rule set are respectively loaded into the multi-parallel state machine of the FPGA hardware processing engine. The load features are then subjected to byte-by-byte synchronous scanning calculation according to each parallel state machine, and the corresponding rule hit record is output. The rule hit records are aggregated, and the matching start offset and the matching byte length are extracted based on the aggregated rule hit records. The bit positions in the preset all-zero initial bitmap with the matching start offset as the starting position and the matching byte length as the coverage range are set and flipped to generate the hit status bitmap.

[0008] By adopting the above technical solution, the FPGA multi-channel parallel state machine is used to perform byte-by-byte synchronous scanning of the load, which improves the regular expression matching throughput from software serial processing to hardware line-speed processing, completely eliminating the packet loss problem under 10 Gigabit traffic; at the same time, the precise bitmap positioning based on offset and length provides a deterministic delay hit location basis for subsequent de-identification.

[0009] Optionally, after reading the metadata to be processed stored in the large page memory space through the preset data plane development kit polling driver, and collecting the network throughput performance parameter set corresponding to the current processing cycle through the data plane development kit polling driver, the method further includes: Receive a rule update trigger signal carrying the target version number and the updated regular expression rule set; allocate a new storage area of ​​the corresponding size in the big page memory space based on the byte length of the updated regular expression rule set; and write the updated regular expression rule set into the new storage area to construct a new rule table. Based on the address mapping relationship between the set of rule entries that have changed in the updated regular expression rule set and the built-in rule storage area of ​​the FPGA hardware processing engine, a reconfiguration instruction is generated that covers the storage address range corresponding to the set of rule entries that have changed. The reconfiguration command is sent to the FPGA hardware processing engine, the reconfiguration completion status signal returned by the FPGA hardware processing engine is read, and the rule version identifier on the FPGA hardware processing engine side is updated to the target version number based on the reconfiguration completion status signal. The old rule table pointer currently held by the polling driver of the data plane development kit is read, and the old rule table pointer is switched to point to the new rule table through an atomic pointer swap operation. After all processing cycles holding the old rule table pointer have completed the reading and processing of the metadata to be processed, the original storage area corresponding to the old rule table pointer is released, and the rule version identifier of the polling driver side of the data plane development kit is updated to the target version number. The rule version identifier on the FPGA hardware processing engine side is compared with the rule version identifier on the polling driver side of the data plane development kit. When the two version identifiers are consistent, it is confirmed that the software and hardware rule versions are synchronized. When the two version identifiers are inconsistent, the reconfiguration instruction is re-executed.

[0010] By adopting the above technical solution, online hot updates of the rule set are realized. By using atomic pointer swapping and version synchronization verification, it is ensured that traffic is uninterrupted and there is zero packet loss during rule switching. This eliminates the desensitization window caused by the downtime required for static configuration updates, effectively preventing sensitive data from flowing out directly without processing, and providing continuous and reliable fallback protection for 10 Gigabit traffic environments.

[0011] Optionally, the step of inputting the set of network throughput performance parameters into a preset long short-term memory network traffic prediction model, outputting peak throughput prediction parameters, generating scheduling instructions based on the difference between the peak throughput prediction parameters and preset capacity assessment parameters, and adjusting the number of active physical cores in a preset worker thread pool based on the scheduling instructions and synchronously updating a preset active core mapping table includes: The set of network throughput performance parameters corresponding to a consecutive preset number of processing cycles is arranged in time sequence to form a sliding window sequence. The sliding window sequence is loaded into the long short-term memory network traffic prediction model to perform forward inference and output the peak throughput prediction parameters corresponding to the future preset step size. The capacity assessment parameter is determined by multiplying the current number of active physical cores in the worker thread pool with a preset single-core throughput benchmark value. The difference between the peak throughput prediction parameter and the capacity assessment parameter is calculated. When the difference is not lower than a preset expansion trigger threshold, an expansion instruction is generated. When the difference is lower than a preset shrinkage trigger threshold, a shrinkage instruction is generated. The expansion instruction or the shrinkage instruction is used as the scheduling instruction. Based on the scheduling instructions, determine the physical cores to be activated and the physical cores to be hibernated, perform an activation operation on the physical cores to be activated, and perform a hibernation operation on the physical cores to be hibernated, to obtain an adjusted set of active physical cores. Read the set of receiving queues recorded before adjustment in the active core mapping table, remap each receiving queue in the set of receiving queues to each active physical core in the set of active physical cores after adjustment, obtain the updated correspondence between active physical core numbers and receiving queues, and write the updated correspondence between active physical core numbers and receiving queues into the active core mapping table.

[0012] By adopting the above technical solution, the LSTM model is used to predict future throughput peaks, transforming passive response into active scheduling. Combined with the difference-driven expansion / shrinkage mechanism, the processing resources are accurately matched to traffic peak and valley changes, completely eliminating peak latency jitter, and realizing elastic scaling and stable low latency of desensitization processing in a 10 Gigabit environment.

[0013] Optionally, the step of remapping each receive queue in the receive queue set to each active physical core in the adjusted active physical core set, to obtain the updated correspondence between active physical core numbers and receive queues, includes: The total number of active physical cores within the adjusted set of active physical cores is counted. A preset modulo hash operation is performed based on the queue number of each receiving queue and the total number of active physical cores to obtain the target active physical core number corresponding to each receiving queue. Based on the target active physical core number, a binding relationship is established between each receiving queue in the receiving queue set and the active physical core corresponding to the number in the adjusted active physical core set, thereby generating the updated correspondence between the active physical core number and the receiving queue. Synchronize the updated correspondence between active physical core numbers and receiving queues with the active core mapping table.

[0014] By adopting the above technical solution, the receiving queue is uniformly remapped to the adjusted active core using modulo hash operation, ensuring that traffic is symmetrically distributed during the increase or decrease of cores, avoiding local overload and latency hotspots caused by uneven distribution, and ensuring the delay stability and smooth transition of the desensitization process during dynamic scaling.

[0015] Optionally, the step of reading the hit status bitmap contained in the metadata to be processed through the target processing core, determining the plaintext sensitive fields marked by the hit status bitmap, performing character replacement processing based on preset masking rules on the plaintext sensitive fields, and combining to generate a de-identified message includes: The target processing core separates the hit status bitmap and payload from the metadata to be processed, traverses the hit status bitmap, and converts the bit numbers of the hit status bitmap that are set into the corresponding byte offset positions in the payload to obtain the starting byte offset position and field byte length of the plaintext sensitive field. Based on the starting byte offset position and the field byte length, the complete byte content of the plaintext sensitive field is read from the payload, and the complete byte content is matched with a preset mask rule template to determine the character replacement method corresponding to the plaintext sensitive field; According to the character replacement method, a preset mask character filling operation is performed on the middle byte content after the preset prefix retention length and before the preset suffix retention length in the plaintext sensitive field to obtain the replaced desensitized field content; The replaced de-identified field content is written back to the original byte offset position corresponding to the plaintext sensitive field in the payload. The checksum is recalculated based on the written-back payload and the protocol header field of the input message. The updated checksum is written into the protocol header field. The protocol header field and the written-back payload are combined to generate the de-identified message.

[0016] By adopting the above technical solution, sensitive fields are accurately located using the hit bitmap and mask filling with preserving prefixes and suffixes is performed, eliminating the risk of leakage while ensuring data availability; the checksum is recalculated to ensure the protocol integrity of the de-identified message, preventing downstream platforms from discarding data due to check errors, thus achieving reliable end-to-end de-identification processing.

[0017] Optionally, the method further includes: The downstream queue depth and processing latency corresponding to the target processing core are continuously monitored. When the downstream queue depth is not lower than a preset queue depth trigger threshold or the processing latency is not lower than a preset latency trigger threshold, the degradation trigger condition is determined to be met. When the degradation trigger condition is met, the payload of the metadata to be processed is divided into a sequence of byte blocks according to a preset block length. A preset secure hash algorithm is performed on each byte block in the sequence of byte blocks to obtain the hash digest value corresponding to each byte block. The original byte content in the payload is replaced with each of the hash digest values. The checksum is recalculated based on the replaced payload and the protocol header field and written into the protocol header field. The protocol header field and the replaced payload are combined to generate a downgraded and desensitized message.

[0018] By adopting the above technical solution, hash replacement degradation is automatically triggered when the desensitization process is overloaded. Lightweight operations replace complex regular expression matching, avoiding further deterioration of processing latency. This provides reliable fallback protection for the 10 Gigabit traffic environment and ensures that the original sensitive data will not be directly leaked to the downstream platform under any stress conditions.

[0019] A second aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the foregoing.

[0020] A third aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any of the preceding descriptions.

[0021] A fourth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the method as described in any of the preceding claims.

[0022] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: By employing FPGA hardware parallel regular expression matching and DPDK large-page memory polling processing, 10 Gigabit line-speed data masking is achieved, eliminating packet loss issues. LSTM traffic prediction drives dynamic scaling of the processing core, enabling precise resource matching of peak and valley changes and eliminating latency jitter. Combining online hot rule updates and overload hash degradation as double fallbacks ensures zero leakage of sensitive data and uninterrupted service. Simultaneously, based on the hit bitmap, mask filling and checksum recalculation with prefixes and suffixes are performed, balancing data availability and protocol integrity, providing a high-performance, highly elastic, and highly reliable dynamic data masking solution. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the system architecture of an embodiment of a dynamic data desensitization method for 10 Gigabit traffic according to this application; Figure 2 This is a flowchart illustrating a dynamic data anonymization method for 10 Gigabit traffic disclosed in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0024] Figure labeling: 101, Service / Mirror Traffic Ingress; 102, 10 Gigabit Optical Port / SFP+; 103, FPGA Acceleration Card; 104, DPDK User-Space Processing Cluster; 105, Downstream Analysis Platform; 301, Processor; 302, Communication Bus; 303, User Interface; 304, Network Interface; 305, Memory. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0026] Figure 1 A schematic diagram of a system architecture for applying a dynamic data anonymization method for 10 Gigabit traffic according to this application is shown. Figure 1As shown, the system may include a service / mirror traffic inlet 101, a 10 Gigabit optical port / SFP+ 102, an FPGA acceleration card 103, a DPDK user-space processing cluster 104, and a downstream analysis platform 105. The service / mirror traffic inlet 101 can be a switch mirror port, a TAP splitter link, a core network DPI mirror link, or other network data inlet; the 10 Gigabit optical port / SFP+ 102 is used to receive network packets at 10Gbps or higher speeds; the FPGA acceleration card 103 is used to perform protocol parsing, parallel regular expression matching, hit status bitmap generation, and metadata encapsulation of packets and bitmaps; the DPDK user-space processing cluster 104 is used to perform high-speed user-space packet transmission and reception, multi-core scheduling, field mask replacement, checksum recalculation, and de-identified packet forwarding; the downstream analysis platform 105 can be an NDR platform, a DLP platform, a SIEM platform, a traffic auditing platform, or other security analysis systems. It should be noted that... Figure 1 The FPGA accelerator card 103 can also be replaced or combined with a DPU, SmartNIC, GPU accelerator card or a dedicated ASIC with hardware parallel processing capabilities.

[0027] This embodiment discloses a dynamic data anonymization method for 10 Gigabit Ethernet traffic. Figure 2 This is a flowchart illustrating a dynamic data anonymization method for 10 Gigabit traffic disclosed in an embodiment of this application, as shown below. Figure 2 As shown, the method includes the following steps: S201. Obtain the input message received by the network port, perform protocol parsing on the input message, and extract the load characteristics contained in the input message; Specifically, the network port can be a 10 Gigabit optical port, an SFP+ interface, a 25GE / 40GE / 100GE high-speed Ethernet interface, a mirror switch port, or a TAP (Test Access Point) splitter input port, etc., used to receive input packets copied from the business system or mirror link. The input packet may include an Ethernet header, VLAN tag, IP header, TCP / UDP header, application layer protocol header, and payload. After receiving the input packet, the system can use the hardware protocol parsing module in the FPGA acceleration card to perform layer-by-layer parsing, such as sequentially identifying the Layer 2 Ethernet type field, VLAN identifier, IPv4 / IPv6 header length, TCP / UDP port number, TCP header offset, and application layer protocol identifiers such as HTTP, DNS, SMTP, FTP, or database access protocols, and thereby determine the starting offset position and payload length in the input packet. The payload characteristics may include the original byte sequence in the payload, application layer field content, packet 5-tuple, protocol type, field boundaries, payload starting offset, payload byte length, and other characteristic information that can be used for subsequent sensitive information identification. For example, in an HTTP message, payload characteristics may include the byte content of URL parameters, Header fields, Cookie fields, and Body fields; in a form submission or JSON message, payload characteristics may include field names, field values, and their offset positions within the payload. Through the above protocol parsing process, the system can limit the input of subsequent FPGA parallel regular expression matching to the payload area that actually contains business data, avoiding invalid matching of irrelevant data such as message headers, thereby reducing hardware matching overhead and improving line-speed processing capability.

[0028] In optional implementations, protocol parsing can be implemented in several specific ways. The first method is a pure hardware parsing approach, where a multi-level pipelined parsing state machine is configured in the FPGA hardware processing engine. Each level of the state machine performs parsing of the Ethernet layer, network layer, transport layer, and application layer respectively, and outputs the effective payload start address, length, and protocol type as payload characteristics. This method has a fixed processing latency and is suitable for 10 Gigabit or higher speed scenarios. The second method is a hardware-software co-parsing approach, where the FPGA side performs rapid parsing of layer 2 to 4 protocol headers and locates the application layer payload. Then, the DPDK user-space processing module performs supplementary parsing for complex application layer structures such as HTTP, JSON, XML, and database protocols. This method balances hardware line-speed processing capabilities with software scalability. The third method is a software parsing approach, where input packets are written to large pages of memory via DMA (Direct Memory Access), and the DPDK polls and drives batch reading of packets and extracts payload characteristics using a preset protocol parsing function. This method is suitable for deployment scenarios involving rule debugging, frequent protocol type changes, or limited hardware resources. The network port type, supported protocol types, maximum packet length, whether to parse tunnel encapsulation, payload truncation length, and parsing timeout time mentioned above can all be configured according to actual application requirements, and this application does not impose any limitations on them. This step provides accurate, complete, and locatable payload input for the subsequent parallel regularization feature matching execution by the FPGA hardware processing engine, ensuring that the dynamic desensitization process can be stably executed in a 10 Gigabit Ethernet environment.

[0029] S202. Call the preset FPGA hardware processing engine to perform parallel regular feature matching on the load features, and generate a hit status bitmap based on the matching start offset and matching byte length in the matching result. Specifically, the FPGA hardware processing engine may include a rule storage area, a multi-channel parallel state machine, a hit record aggregation unit, and a bitmap generation unit. The system reads a pre-compiled regular expression rule set from the rule storage area. This rule set includes, for example, rules for identifying sensitive fields such as ID card numbers, mobile phone numbers, bank card numbers, and email addresses. The system simultaneously sends the effective payload byte stream corresponding to the payload characteristics into the multi-channel parallel state machine for byte-by-byte synchronous scanning. Each parallel state machine can correspond to different rules or rule groups. When a rule hit is detected, a rule hit record is output. This rule hit record includes at least a rule identifier, a matching start offset, and a matching byte length. The hit record aggregation unit merges the hit records from each rule path. For overlapping or adjacent hit intervals, they can be merged using a union method or retained according to rule priority. Subsequently, the bitmap generation unit creates an initial bitmap with all zeros corresponding to the position of the payload byte. Starting from the matching start offset in each hit record and covering the length of the matching byte, the corresponding bit positions are set to the set state. For example, the bits corresponding to the matching start offset from offset to offset+len-1 are all set to 1, thereby generating a hit state bitmap. Optionally, the parallel regular feature matching can be implemented using a multi-level filtering structure of DFA / NFA hardware state machine, PCRE-compatible hardware pipeline, or AC automaton plus precise regularity verification. The bitmap granularity is preferably one byte corresponding to one bit, but it can also be configured to multi-byte granularity according to performance requirements. This step releases the CPU from high-overhead regularity scanning through FPGA parallel matching and provides a deterministic sensitive field location basis for the subsequent DPDK processing core through the hit state bitmap.

[0030] Optionally, the step of invoking a preset FPGA hardware processing engine to perform parallel regular feature matching on the load features and generating a hit status bitmap based on the matching start offset and matching byte length in the matching result includes: reading a set of regular expression rules from the built-in rule storage area of ​​the FPGA hardware processing engine; loading the load features and the set of regular expression rules into the multi-channel parallel state machine of the FPGA hardware processing engine respectively; performing byte-by-byte synchronous scanning calculation on the load features according to each of the parallel state machines, and outputting the corresponding rule hit records; aggregating the rule hit records; extracting the matching start offset and the matching byte length based on the aggregated rule hit records; and performing a bit-set / flip operation on the bits in the preset all-zero initial bitmap with the matching start offset as the starting position and the matching byte length as the coverage range to generate the hit status bitmap.

[0031] Specifically, the FPGA hardware processing engine may include a built-in rule storage area, a multi-channel parallel state machine, a hit record buffer, a hit record aggregation unit, and a bitmap generation unit. The built-in rule storage area stores regular expression rule sets, which may include rules for identifying sensitive fields such as ID card numbers, mobile phone numbers, bank card numbers, email addresses, account identifiers, and token fields. To facilitate hardware execution, the rule sets can be pre-compiled by the control plane into state transition tables, character matching tables, mask comparison tables, or PCRE-compatible matching configurations, and written to the FPGA's on-chip BRAM, URAM, off-chip DDR, HBM, or register-mapped storage space. Upon receiving load characteristics, the FPGA hardware processing engine reads the currently effective version of the regular expression rule set from the built-in rule storage area and allocates it to the corresponding parallel state machine according to the rule number, rule type, or rule grouping.

[0032] Furthermore, the load characteristics may include information such as the payload byte stream, the payload start offset, the payload length, and the protocol type. The FPGA hardware processing engine inputs the payload byte stream from the load characteristics into a multi-parallel state machine and loads different rules or rule groups from the regular expression rule set into each parallel state machine. Each parallel state machine can be implemented based on a DFA, NFA, AC automaton, or PCRE hardware pipeline, and performs synchronous scan calculations on the input bytes within the same clock cycle or pipeline cycle. When each parallel state machine detects that the payload byte sequence satisfies the corresponding rule, it generates a rule hit record. The rule hit record includes at least a rule identifier, a match start offset, and a match byte length, and may further include fields such as a match end offset, protocol type, message identifier, and rule version number. For example, when the state machine recognizes that 11 consecutive digits starting at offset 100 in the payload match the mobile phone number rule, it can output the rule identifier "mobile phone number", the match start offset 100, and the match byte length 11.

[0033] Furthermore, the hit record aggregation unit receives the rule hit records output by each parallel state machine and aggregates multiple rule hit records corresponding to the same message. For cases where multiple rule hit records do not overlap, the hit record aggregation unit can directly retain each hit interval; for overlapping or adjacent hit intervals, methods such as union-set merging, rule priority retention, or long-match priority can be used. For example, when a bank card number rule and a regular number string rule hit the same interval, the hit record corresponding to the bank card number rule can be retained according to the preset rule priority; if the two hit intervals partially overlap, they can also be merged into a continuous hit range covering both intervals. The above merging strategy can be configured according to the actual de-identification rules, and this application does not limit it.

[0034] After the rule hit records are aggregated, the bitmap generation unit creates an all-zero initial bitmap corresponding to the position of the payload byte. Preferably, the hit status bitmap adopts a one-byte-to-one-bit mapping method, that is, the i-th byte in the payload corresponds to the i-th bit in the hit status bitmap; when the byte belongs to a sensitive field, the corresponding bit position is set to a set state, for example, set to 1, otherwise it remains 0. The bitmap generation unit extracts the matching start offset and the matching byte length based on the aggregated rule hit records, and performs a set-to-flip operation on the bits in the all-zero initial bitmap that start at the matching start offset and cover the matching byte length. Specifically, if the matching start offset is offset and the matching byte length is len, then all bits in the hit status bitmap from offset to offset+len-1 are set to 1, thereby obtaining a hit status bitmap that can identify the position of the sensitive field byte.

[0035] In other optional implementations, the granularity of the hit state bitmap can be configured to two bytes, four bytes, or a fixed field granularity, depending on performance and storage requirements. Parameters such as the number of parallel state machines, the number of rules per state machine, the maximum scan load length, rule priority, and bitmap length alignment can all be configured by the management plane. Through this implementation, the FPGA can perform regularity feature matching in hardware parallelism and convert the complex matching results into a hit state bitmap that is easily readable by the software side. This eliminates the need for the subsequent DPDK processing core to perform regularity scanning again; it only needs to locate the sensitive fields according to the bitmap and perform masking processing, thereby improving the de-identification throughput and processing determinism under 10 Gigabit traffic.

[0036] S203. Aggregate the input message and the hit status bitmap to generate metadata to be processed, and write the metadata to be processed into a preset large page memory space through a preset direct memory access channel. Specifically, after generating the hit status bitmap, the FPGA hardware processing engine can encapsulate the original input message, the hit status bitmap, and necessary descriptive information into metadata to be processed. This descriptive information includes, for example, message length, payload start offset, payload length, bitmap length, message sequence number, port number, protocol type, timestamp, and rule version number. The metadata to be processed can use a contiguous memory layout or a segmented descriptor layout. In a contiguous memory layout, message data, bitmap data, and descriptor headers are stored in a preset order, such as "descriptor header + input message + hit status bitmap." In a segmented descriptor layout, the descriptor header records the input message address, bitmap address, and corresponding length, facilitating subsequent address-based reading by the DPDK processing core. This aggregation method ensures that the input message and its corresponding hit status bitmap have consistent message identification and processing order, preventing message-bitmap mismatches during software reading.

[0037] Furthermore, the FPGA writes the metadata to be processed into a preset large page memory space through a preset direct memory access channel. The direct memory access channel can be a PCIe DMA channel, an AXI DMA channel, or a queued DMA channel supported by the network card / acceleration card; the large page memory space can be a HugePage memory area pre-allocated and registered by DPDK, whose physical address or IOVA address is issued to the FPGA through a descriptor queue. The FPGA determines the target write address based on the available buffer descriptors, writes the metadata to be processed into the large page memory in burst write or queue write mode, and updates the completion queue, doorbell register, or interrupt / polling flag after writing to notify the DPDK processing core to read. Parameters such as the page size, number of buffers, DMA queue depth, single write length, and alignment of the large page memory can be configured according to the traffic rate and device capabilities. Directly writing to the large page memory via DMA reduces the number of CPU copies, lowers the latency of message transmission between hardware and software, and provides a data foundation for subsequent fast de-identification processing based on the hit status bitmap.

[0038] S204. Read the metadata to be processed stored in the large page memory space through the preset data plane development kit polling driver, and collect the set of network throughput performance parameters corresponding to the current processing cycle through the data plane development kit polling driver. Specifically, the data plane development kit can be DPDK, and the polling driver can be DPDK's PMD (Poll Mode Driver) or a custom FPGA accelerator card polling driver implemented based on the DPDK user-space driver framework. The system pre-registers the large page memory space as a DPDK-accessible mempool or circular buffer, and establishes a receive queue, a completion queue, or a descriptor queue. After the FPGA writes the metadata to be processed into the large page memory via DMA, the polling driver periodically reads the completion queue or descriptor status using a non-interrupted polling method. For example, it obtains multiple available descriptors at once through a batch read interface and locates the corresponding metadata to be processed based on the physical address or IOVA address recorded in the descriptor. The metadata to be processed can be encapsulated as an rte_mbuf structure or saved as a custom structure of "descriptor header + message data + hit status bitmap." After reading it, the polling driver submits it to the subsequent de-identification processing thread or processing pipeline.

[0039] While reading the metadata to be processed, the polling driver also collects a set of network throughput performance parameters corresponding to the current processing cycle. The current processing cycle can be a preset time window, a preset number of polls, or a preset number of packet batches, such as 1 millisecond, 10 milliseconds, 1000 polls, or N packet batches. Its specific values ​​can be configured according to business throughput requirements. The set of network throughput performance parameters may include, but is not limited to, the number of received packets, the number of received bytes, the number of sent packets, the number of sent bytes, the number of lost packets, the DMA completion queue depth, the backlog length of the pending processing queue, the average batch read count, processing time, CPU core utilization, and packets per second (PPS) and bits per second (BPS) within the current cycle. These parameters can be obtained through the DPDK statistics interface, the PMD private statistics register, the queue descriptor counter, and the software cycle counter. By synchronously collecting throughput performance parameters during the polling process of reading metadata, subsequent steps can dynamically adjust the processing strategy, batch size, or de-identified execution path based on real-time load conditions, thereby improving processing stability and resource utilization in high-throughput scenarios.

[0040] Optionally, after reading the metadata to be processed stored in the large page memory space through the preset data plane development kit polling driver, and collecting the network throughput performance parameter set corresponding to the current processing cycle through the data plane development kit polling driver, the method further includes: receiving a rule update trigger signal carrying a target version number and an update regular expression rule set; allocating a new storage area of ​​corresponding size in the large page memory space based on the byte length of the update regular expression rule set; writing the update regular expression rule set into the new storage area to construct a new rule table; generating a reconfiguration instruction covering the storage address range corresponding to the changed rule entry set based on the address mapping relationship between the set of changed rule entries in the update regular expression rule set and the built-in rule storage area of ​​the FPGA hardware processing engine; issuing the reconfiguration instruction to the FPGA hardware processing engine, and reading the... The FPGA hardware processing engine returns a reconfiguration completion status signal. Based on this signal, the rule version identifier on the FPGA hardware processing engine side is updated to the target version number. The old rule table pointer currently held by the data plane development kit polling driver is read, and an atomic pointer swap operation is used to switch the old rule table pointer to point to the new rule table. After all processing cycles holding the old rule table pointer have completed reading and processing the metadata to be processed, the original storage area corresponding to the old rule table pointer is released, and the rule version identifier on the data plane development kit polling driver side is updated to the target version number. The rule version identifier on the FPGA hardware processing engine side is compared with the rule version identifier on the data plane development kit polling driver side. If the two version identifiers are the same, it is confirmed that the software and hardware rule versions are synchronized. If the two version identifiers are inconsistent, the reconfiguration instruction is re-executed.

[0041] Specifically, the system receives a rule update trigger signal, which can be issued by the management platform, a configuration file change event, or a remote interface, and carries the target version number and the updated regular expression rule set. The data plane development kit's polling driver or its rule management thread, based on the byte length of the updated regular expression rule set, allocates a new storage area of ​​the corresponding size within a preset large page memory space. For example, it allocates contiguous or nearly contiguous HugePage memory through DPDK's rte_malloc_socket, rte_memzone_reserve, or mempool buffer request interface. Subsequently, the updated regular expression rule set is written to this new storage area, and a new rule table is constructed according to a preset rule table format. The new rule table may include a rule header, rule count, rule version number, rule entry index, rule content area, and validation fields, so that it can be recognized by both the software and hardware sides.

[0042] After constructing the new rule table, the rule management thread generates a reconfiguration instruction based on the address mapping relationship between the set of changed rule entries in the update regular expression rule set and the built-in rule storage area of ​​the FPGA hardware processing engine. This address mapping relationship can be pre-recorded in a rule index table; for example, each rule corresponds to the starting address, storage length, and state machine number in the FPGA's built-in BRAM, URAM, DDR, or HBM. When only some rules are added, deleted, or modified, the reconfiguration instruction can cover only the storage address range corresponding to the changed rule entries; when the rule structure changes significantly, a full reconfiguration instruction covering the entire rule storage area can also be generated. The reconfiguration instruction can include fields such as target version number, write start address, write length, rule data address, checksum, state machine number, and reconfiguration type, where the reconfiguration type can be incremental update, full update, or rollback update.

[0043] Further, the rule management thread issues the reconfiguration command to the FPGA hardware processing engine. This issuance can be achieved through methods such as PCIe BAR register writing, DMA descriptor queues, AXI-Lite configuration bus, or device-specific control queues. Upon receiving the reconfiguration command, the FPGA hardware processing engine writes the modified rule entries into its built-in rule storage area according to the address range specified in the command, and performs verification upon completion, such as CRC check, length check, rule quantity check, or version number check. After successful verification, the FPGA hardware processing engine returns a reconfiguration completion status signal to the software side; this signal can be returned via a completion queue, status register, interrupt flag, or polling flag. The rule management thread reads this reconfiguration completion status signal. If the reconfiguration is successful, it updates the rule version identifier on the FPGA hardware processing engine side to the target version number; if the reconfiguration fails, the original rule version can be retained, and the reconfiguration command can be reissued or a rollback process can be triggered.

[0044] After the hardware-side rule update is completed, the rule management thread reads the old rule table pointer currently held by the data plane development kit polling the driver, and switches the old rule table pointer to point to the new rule table through an atomic pointer swap operation. This atomic pointer swap operation can be implemented using CAS operations, RCU (Read-Copy-Update) pointer replacement mechanisms, C11 atomic operations, or the atomic variable interface provided by DPDK, to ensure that pointer tearing or read-write inconsistencies do not occur when multiple processing cores concurrently read the rule table. After the pointer switch is completed, the newly entered processing cycle reads the new rule table, while processing cycles that have already started executing and still hold the old rule table pointer continue to use the old rule table to complete the reading and processing of the currently pending metadata, avoiding access exceptions caused by the rule table being released midway through processing.

[0045] Furthermore, to safely release the original storage area corresponding to the old rule table, the rule management thread can record the processing cycle status or reference count of each processing core. When it is detected that all processing cycles holding pointers to the old rule table have completed reading and processing the metadata to be processed, the original storage area corresponding to the old rule table pointer is released, for example, by calling `rte_free`, `rte_memzone_free`, or returning the corresponding buffer to the mempool. Afterwards, the rule version identifier of the data plane development kit polling driver side is updated to the target version number. The determination of processing cycle completion can include: checking whether the epoch value of each processing core has progressed to after the pointer switch, checking whether the old rule table reference count has returned to zero, or checking whether the current rule versions reported by the polling thread have all been changed to the new version.

[0046] Furthermore, the rule version identifiers on the FPGA hardware processing engine side and the polling driver side of the data plane development kit are compared. When the two version identifiers match, it is confirmed that the software and hardware rule versions are synchronized, and subsequent FPGA regular expression matching results and software-side rule interpretation, statistics, or auditing logic are all executed based on the same version rule. When the two version identifiers do not match, the reconfiguration command can be re-executed, or the FPGA-side error code, verification failure reason, and current rule address can be read first, and then the reconfiguration command can be regenerated according to the changed rule entry set and re-issued. The number of retries, retry interval, version number format, verification algorithm, and old rule table retention time can all be configured according to actual deployment requirements, and this application does not limit them.

[0047] By using the above rule update method, this application can complete the online replacement of the regular expression rule set without interrupting the DPDK polling processing and packet forwarding process, and use version identifier comparison to ensure the rule consistency between the FPGA hardware processing engine and the data plane development kit polling driver, thereby reducing the risk of packet loss and matching errors during rule updates.

[0048] S205. Input the set of network throughput performance parameters into a preset long short-term memory network traffic prediction model, output peak throughput prediction parameters, generate scheduling instructions based on the difference between the peak throughput prediction parameters and the preset capacity assessment parameters, adjust the number of physical cores in the preset worker thread pool that are in an active state based on the scheduling instructions, and synchronously update the preset active core mapping table. Specifically, the data plane development kit's polling driver can collect a set of network throughput performance parameters in each processing cycle and write them into the traffic feature window in chronological order of collection to form the input sequence of the Long Short-Term Memory (LSTM) network traffic prediction model. The set of network throughput performance parameters includes, for example, PPS, BPS, number of received bytes, number of received packets, queue backlog length, DMA completion queue depth, number of lost packets, average batch size, and processing latency. The LTM network traffic prediction model can be pre-trained offline and then deployed in the control thread executing resource scheduling, or its parameters can be periodically corrected through online incremental updates. This model takes the throughput feature sequence from multiple consecutive processing cycles as input and outputs peak throughput prediction parameters for the next preset time window or multiple future time windows, such as predicting peak PPS, peak BPS, or peak queue backlog. The electronic device compares the peak throughput prediction parameters with preset capacity assessment parameters. These capacity assessment parameters characterize the maximum PPS, maximum BPS, or maximum number of packet batches that the currently active physical cores can stably process per unit time. These parameters can be obtained through load testing calibration, historical statistics, or real-time moving averages. When the comparison result indicates that the predicted peak throughput exceeds the current capacity, an expansion command is generated, switching one or more idle physical cores to an active state. When the predicted peak throughput is lower than the current capacity and meets preset reduction conditions, a reduction command is generated, switching some low-load physical cores to an idle or dormant state. When the comparison result is within a preset stable range, the number of currently active physical cores remains unchanged. During expansion or reduction, the electronic device can adjust resources by setting thread affinity, starting or pausing the data plane development kit polling thread, and adjusting the binding relationship between the receive queue and physical cores. Simultaneously, it updates the active core mapping table, which records the active physical core number, thread identifier, receive queue number, running status, and rule version. This allows for pre-configuration of processing core resources based on predicted traffic, reducing the risk of congestion and packet loss caused by sudden traffic spikes and minimizing core resource consumption in low-load scenarios.

[0049] Optionally, the step of inputting the network throughput performance parameter set into a preset long short-term memory network traffic prediction model, outputting peak throughput prediction parameters, generating scheduling instructions based on the difference between the peak throughput prediction parameters and preset capacity assessment parameters, and adjusting the number of active physical cores in a preset worker thread pool based on the scheduling instructions and synchronously updating a preset active core mapping table includes: arranging the network throughput performance parameter set corresponding to a consecutive preset number of processing cycles in a time-series manner to form a sliding window sequence; loading the sliding window sequence into the long short-term memory network traffic prediction model to perform forward inference, outputting the peak throughput prediction parameters corresponding to a future preset step size; determining the capacity assessment parameters by multiplying the current number of active physical cores in the worker thread pool by a preset single-core throughput benchmark value, and calculating the peak throughput prediction parameters and the capacity assessment parameters. The difference in estimated parameters is used to generate an expansion command when the difference is not lower than a preset expansion trigger threshold, and a reduction command when the difference is lower than a preset reduction trigger threshold. The expansion command or the reduction command is used as the scheduling command. Based on the scheduling command, physical cores to be activated and physical cores to be dormant are determined. An activation operation is performed on the physical cores to be activated, and a dormant operation is performed on the physical cores to be dormant, resulting in an adjusted set of active physical cores. The set of receiving queues recorded before adjustment in the active core mapping table is read, and each receiving queue in the receiving queue set is remapped to each active physical core in the adjusted set of active physical cores, resulting in an updated correspondence between active physical core numbers and receiving queues. This updated correspondence between active physical core numbers and receiving queues is written into the active core mapping table.

[0050] In a preferred implementation, the set of network throughput performance parameters corresponding to a predetermined number of consecutive processing cycles can be arranged in time sequence to form a sliding window sequence. This sliding window sequence is then loaded into a predetermined Long Short-Term Memory (LSTM) network traffic prediction model for forward inference. Specifically, the set of network throughput performance parameters for each processing cycle can serve as a feature vector for a time step. This feature vector may include, for example, PPS, BPS, number of received packets, number of received bytes, queue backlog length, DMA completion queue depth, number of lost packets, average batch read count, and processing latency. The feature vectors of N consecutive processing cycles are arranged from earliest to latest according to their acquisition time to form an input sequence of length N, where N can be 16, 32, 64, or other values ​​configured according to the business scenario. After receiving this sliding window sequence, the LSM network traffic prediction model models the historical throughput change trend through an input gate, a forget gate, and an output gate, outputting the peak throughput prediction parameters corresponding to a predetermined step size in the future, such as the predicted peak PPS, predicted peak BPS, or predicted peak queue backlog within the next 1, 5, or 10 processing cycles. The preset quantity, preset step size, and model parameters can be configured according to the actual traffic fluctuations and the computing power of the deployed equipment; this application does not impose any limitations on these.

[0051] After obtaining the peak throughput prediction parameters, the current system capacity assessment parameters are determined by multiplying the current number of active physical cores in the worker thread pool by a preset single-core throughput baseline value. The single-core throughput baseline value can be obtained through offline stress testing or dynamically adjusted based on the historical stable average throughput during online operation; for example, when the current number of active physical cores is M and the single-core throughput baseline value is C, the capacity assessment parameter can be determined as M×C. Then, the difference between the peak throughput prediction parameters and the capacity assessment parameters is calculated. If the difference is not lower than a preset expansion trigger threshold, it indicates that future traffic may exceed the current processing capacity, and an expansion command is generated; if the difference is lower than a preset reduction trigger threshold, it indicates that the current number of active cores is redundant, and a reduction command is generated; if the difference is between the expansion trigger threshold and the reduction trigger threshold, the current core configuration can remain unchanged. The expansion trigger threshold and the reduction trigger threshold can be set as a fixed throughput difference or as a percentage of the capacity assessment parameters to avoid frequent expansion and reduction due to slight traffic fluctuations.

[0052] Further, the scheduling module determines the physical cores to be activated and the physical cores to be hibernated based on the scheduling instructions. For expansion instructions, physical cores that are idle, have the lowest load, or are closest to the NUMA (Non-Uniform Memory Access) node of the network card can be selected from the worker thread pool as the physical cores to be activated, and activation operations can be performed on them. The activation operations may include creating or waking up the DPDK polling thread, setting the thread CPU affinity, initializing the receive queue context, binding the rule table pointer, and starting the packet polling loop. For shrinking instructions, active physical cores with the lowest current load, the least queue backlog, or a low priority can be selected as the physical cores to be hibernated, and hibernation operations can be performed on them. The hibernation operations may include stopping the retrieval of new packets from the corresponding receive queue, waiting for the acquired pending metadata to be processed to be completed, submitting the thread status, unbinding the queue, and putting the thread into a paused or low-power state. After the above processing, the adjusted set of active physical cores is obtained.

[0053] After the set of active physical cores changes, the scheduling module reads the set of receive queues recorded in the active core mapping table before the adjustment, and remaps each receive queue in that set to each active physical core in the adjusted set of active physical cores to obtain the updated correspondence between active physical core numbers and receive queues. Specifically, the receive queues can be evenly distributed among the active physical cores using a round-robin balancing method, or weighted according to the real-time traffic weight of each receive queue, or priority can be given to mapping receive queues on the same NUMA node to the active physical cores of that node to reduce cross-node memory access overhead. After mapping, the updated active physical core number, thread identifier, receive queue number, running status, NUMA node information, and current rule version are written to the active core mapping table. Thus, the DPDK round-robin driver can subsequently receive packets and read pending metadata based on the updated active core mapping table, thereby enabling pre-scheduling of CPU physical core resources based on predicted throughput, reducing congestion and packet loss risks under sudden traffic surges, and minimizing resource waste in low-load scenarios.

[0054] Optionally, the step of remapping each receiving queue in the receiving queue set to each active physical core in the adjusted active physical core set to obtain the updated correspondence between active physical core numbers and receiving queues includes: counting the total number of active physical cores in the adjusted active physical core set; performing a preset modulo hash operation based on the queue number of each receiving queue and the total number of active physical cores to obtain the target active physical core number corresponding to each receiving queue; establishing a binding relationship between each receiving queue in the receiving queue set and the active physical core corresponding to the number in the adjusted active physical core set according to each target active physical core number, generating the updated correspondence between active physical core numbers and receiving queues; and synchronizing the updated correspondence between active physical core numbers and receiving queues with the active core mapping table.

[0055] As an optional implementation of the receive queue remapping process, after activating or hibernating physical cores according to scheduling instructions, the electronic device executing this method can count the total number of active physical cores in the adjusted active physical core set. Specifically, the data plane development kit polling driver can read the running status of each physical core in the active core mapping table, determine the active physical cores as the adjusted active physical core set, and count the total number of active physical cores in the set, denoted as K. To facilitate subsequent mapping, the data plane development kit polling driver can also generate a list of active physical core numbers, such as ActiveCoreList[0] to ActiveCoreList[K-1]. Each element in this list is used to record the number of a physical core that is actually in an active state. If K is 0, it means that there are currently no active physical cores available to carry the receive queue. In this case, the receive queue mapping relationship before adjustment can be maintained, or an exception alarm can be triggered to prevent the receive queue from being mapped to an unavailable core.

[0056] Furthermore, the data plane development kit's polling driver performs a preset modulo hash operation based on the queue number of each receiving queue and the total number of active physical cores to obtain the target active physical core number corresponding to each receiving queue. Specifically, for a receiving queue with queue number QID, Index = QID mod K can be calculated, and the physical core number recorded in ActiveCoreList[Index] can be determined as the target active physical core number corresponding to that receiving queue. Optionally, a lightweight hash calculation can be performed on the queue number QID first, followed by a modulo operation, for example, Index = Hash(QID) mod K, where Hash(QID) can use multiplicative hashing, CRC hashing, or other hash algorithms that can output integer results. In this way, even if the number of receiving queues is greater than the number of active physical cores, multiple receiving queues can be evenly distributed to different active physical cores; when the number of receiving queues is less than or equal to the number of active physical cores, it can also be guaranteed that each receiving queue has a definite target processing core.

[0057] After obtaining the target active physical core number corresponding to each receiving queue, the data plane development kit polling driver establishes a binding relationship between each receiving queue in the receiving queue set and the active physical core corresponding to the number in the adjusted active physical core set, thereby generating an updated correspondence between active physical core numbers and receiving queues. For example, when the receiving queue set includes queues 0 to 7, and the adjusted active physical core set includes 4 active physical cores, queues 0 and 4 can be bound to the first active physical core, queues 1 and 5 to the second active physical core, queues 2 and 6 to the third active physical core, and queues 3 and 7 to the fourth active physical core, according to the result of QID mod 4. The binding relationship can be stored in the form of a binary tuple of "active physical core number - receiving queue number", or it can be stored in the form of a list of receiving queues corresponding to each active physical core. This application does not limit this.

[0058] Furthermore, the data plane development kit's polling driver synchronously writes the updated mapping between active physical core numbers and receive queues into the active core mapping table. Specifically, during the write process, a complete new mapping can be generated in a temporary mapping table first, and then synchronized to the active core mapping table using methods such as atomic pointer switching, version number updates, read-write locks, or mutexes. This avoids inconsistencies when processing cores within the worker thread pool read the mapping. After the update, active physical cores within the worker thread pool can read their bound receive queues according to the active core mapping table and obtain the metadata to be processed from the corresponding receive queues. Thus, by using modulo hash operations, deterministic, low-overhead, and relatively balanced allocation of receive queues among the adjusted active physical cores is achieved. This reduces local core overload during scaling up or down, ensuring the stability of the 10 Gigabit traffic anonymization processing.

[0059] S206. Based on the preset receiver scaling algorithm and the active core mapping table, the metadata to be processed is distributed to the target processing cores in the worker thread pool as determined by the scheduling instruction. Specifically, the receiver scaling algorithm is used to determine the processing destination of the metadata to be processed according to the "receive queue number - active physical core number" correspondence stored in the active core mapping table after the number of active physical cores is adjusted. When the data plane development kit polls and drives to read the metadata to be processed from the big page memory space, it simultaneously determines the receive queue number carrying the metadata to be processed; then it reads the currently effective active core mapping table, queries the active physical core number bound to the receive queue number, and determines the active physical core obtained from the query as the target processing core. The metadata to be processed can be distributed through a lock-free circular queue, cache queue, or batch task queue corresponding to the target processing core. The target processing core polls and reads the metadata to be processed from its corresponding queue to perform subsequent hit status bitmap reading and character replacement processing.

[0060] In one optional implementation, when the scheduling instruction is a scaling up or scaling down instruction, the data plane development kit's polling driver, after the updated active core mapping table takes effect, distributes the newly read metadata to be processed to the target processing core according to the new receive queue binding relationship. For metadata already distributed to the original processing core before the mapping table update, it can continue to be processed by the original processing core, thus avoiding metadata migration and duplicate processing during the processing. Optionally, the active core mapping table can take effect using a version number or atomic pointer switching method, ensuring that each batch of metadata to be processed is distributed based on the same version of the mapping relationship. Through this method, continuous and deterministic distribution of metadata to be processed can be achieved during the scaling up or scaling down of the worker thread pool, avoiding metadata loss or mismatch, and ensuring that the subsequent de-identification processing load matches the current number of active physical cores.

[0061] S207. The target processing core reads the hit status bitmap contained in the metadata to be processed, determines the plaintext sensitive fields marked by the hit status bitmap, performs character replacement processing based on preset masking rules on the plaintext sensitive fields, and combines them to generate a de-identified message.

[0062] Specifically, after the target processing core obtains the metadata to be processed from the corresponding processing queue, it can separate the protocol header field, payload, and hit status bitmap of the input message based on the description information in the metadata. The hit status bitmap indicates the position of the hit sensitive field rule in the payload. The target processing core traverses the hit status bitmap, converting the bit numbers with set values ​​into byte offset positions in the payload according to a preset bitmap granularity. It then merges consecutive set bits into a sensitive field interval, using the first set bit in this interval to determine the starting byte offset position of the plaintext sensitive field, and the number of consecutive set bits to determine the field byte length. Subsequently, the target processing core reads the corresponding plaintext sensitive field from the payload based on the starting byte offset position and the field byte length, and matches it with a preset mask rule template to determine the prefix retention length, suffix retention length, mask characters, and character replacement methods. For example, a mobile phone number can retain the first 3 and last 4 digits, an ID card number can retain the first 6 and last 4 digits, and bank card numbers, email addresses, or account identifiers can also be replaced according to the corresponding template. The target processing core replaces the middle byte content of plaintext sensitive fields with mask characters according to a defined character replacement method, and writes the replaced de-identified field content back to the original byte offset position in the payload to obtain the written-back payload. Then, based on the written-back payload and protocol header fields, a checksum is recalculated and written, including, for example, IP header checksum, TCP checksum, or UDP checksum. Finally, the updated protocol header fields and the written-back payload are combined to generate a de-identified message, which is then output to the downstream analysis platform or forwarding interface. Thus, the software can directly complete sensitive field location and de-identification based on the hit status bitmap without performing complex regular expression matching again, thereby improving the de-identification processing efficiency in high-throughput traffic scenarios and reducing the risk of plaintext sensitive data leakage.

[0063] Optionally, the step of reading the hit status bitmap contained in the metadata to be processed through the target processing core, determining the plaintext sensitive fields marked by the hit status bitmap, performing character replacement processing based on a preset masking rule on the plaintext sensitive fields, and combining to generate a de-identified message includes: separating the hit status bitmap and the payload from the metadata to be processed through the target processing core; traversing the hit status bitmap; converting the bit numbers of the hit status bitmap that are in the set state into the corresponding byte offset positions in the payload to obtain the starting byte offset position and field byte length of the plaintext sensitive field; and reading the plaintext sensitive field from the payload according to the starting byte offset position and the field byte length. The complete byte content is matched with a preset masking rule template to determine the character replacement method corresponding to the plaintext sensitive field. A preset masking character filling operation is performed on the middle bytes after a preset prefix retention length and before a preset suffix retention length in the plaintext sensitive field according to the character replacement method, resulting in the replaced desensitized field content. The replaced desensitized field content is written back to the original byte offset position corresponding to the plaintext sensitive field in the payload. The checksum is recalculated based on the written-back payload and the protocol header field of the input message. The updated checksum is written into the protocol header field. The protocol header field and the written-back payload are combined to generate the desensitized message.

[0064] As an optional implementation, after obtaining the metadata to be processed, the target processing core can first parse and separate the hit status bitmap and the payload from the metadata. The hit status bitmap is used to mark the positions of the sensitive field identification rules in the payload, and its bitmap granularity can be 1 byte, 2 bytes, 4 bytes, or other preset lengths. The target processing core traverses the hit status bitmap, converting the bit numbers with set values ​​into corresponding byte offset positions within the payload according to the bitmap granularity. For multiple consecutive set bits, they can be merged into the same plaintext sensitive field range, and the starting byte offset position of the plaintext sensitive field is determined by the first set bit. The field byte length is determined by the number of consecutive set bits and the bitmap granularity.

[0065] After determining the starting byte offset and field byte length, the target processing core reads the corresponding length of complete byte content from the payload based on the starting byte offset, which serves as the plaintext sensitive field to be de-identified. Subsequently, this complete byte content is matched against a preset masking rule template to determine the character replacement method corresponding to the plaintext sensitive field. The masking rule template may include field type, prefix retention length, suffix retention length, mask characters, and applicable conditions. For example, a mobile phone number template may retain the first 3 and last 4 digits, an ID card number template may retain the first 6 and last 4 digits, and bank card number, email address, or account identifier templates may also be configured with different retention lengths and mask characters according to business needs; this application does not impose any limitations on these.

[0066] The target processing core performs mask character padding on the middle bytes of the plaintext sensitive field that are located after a preset prefix retention length and before a preset suffix retention length, according to the character replacement method obtained by matching, to obtain the replaced de-identified field content. In specific implementation, fixed character padding, different mask characters selected for padding according to field type, or padding with the same length as the original field can be used; preferably, the replaced de-identified field content maintains the same byte length as the original plaintext sensitive field to avoid changing the effective payload length and message structure.

[0067] After the field replacement is completed, the target processing core writes back the replaced de-identified field content to the original byte offset position corresponding to the plaintext sensitive field in the payload, obtaining the written-back payload. To ensure that the message still meets the protocol verification requirements during subsequent transmission or parsing, the target processing core recalculates the checksum based on the written-back payload and the protocol header field of the input message, and writes the updated checksum into the corresponding protocol header field; the checksum may include, for example, an IP header checksum, a TCP checksum, or a UDP checksum. Finally, the target processing core combines the updated protocol header field and the written-back payload to generate a de-identified message, which can be output to a downstream analysis platform, storage module, or forwarding interface.

[0068] Optionally, the method further includes: continuously monitoring the downstream queue depth and processing latency corresponding to the target processing core; determining that a degradation triggering condition is met when the downstream queue depth is not lower than a preset queue depth triggering threshold or the processing latency is not lower than a preset latency triggering threshold; dividing the payload contained in the metadata to be processed when the degradation triggering condition is met into a byte block sequence according to a preset block length; performing a preset secure hash algorithm operation on each byte block in the byte block sequence to obtain a hash digest value corresponding to each byte block; replacing the original byte content corresponding to the payload with each hash digest value; recalculating the checksum based on the replaced payload and the protocol header field and writing it into the protocol header field; and combining the protocol header field and the replaced payload to generate a degradation and desensitization message.

[0069] Specifically, the depth and processing latency of the downstream queue corresponding to the target processing core are continuously monitored to determine whether there is a risk of congestion in the current de-identification processing link. The downstream queue can be a ring queue, message queue, or buffer queue through which the target processing core submits de-identified messages to subsequent forwarding modules, log output modules, storage modules, or analysis modules. The downstream queue depth can represent the number of messages or bytes in the queue that have not yet been consumed. The processing latency can represent the time elapsed from when the metadata to be processed enters the processing queue corresponding to the target processing core to when the de-identification is completed and output, or it can represent the average time taken by the target processing core to process a single message or batch of messages within a preset statistical period. The downstream queue depth and processing latency can be obtained by reading the queue counter, timestamp, or performance statistics register according to a preset sampling period.

[0070] During monitoring, when the downstream queue depth is not lower than a preset queue depth trigger threshold, or the processing latency is not lower than a preset latency trigger threshold, the electronic device determines that the degradation trigger condition has been met. The queue depth trigger threshold and latency trigger threshold can be pre-configured based on business real-time requirements, downstream consumption capacity, average message length, and physical core processing capacity. For example, they can be configured as 70%, 80%, or 90% of the queue capacity, or as a fixed number of messages or bytes. The latency trigger threshold can be configured as a microsecond, millisecond, or other time value suitable for the business scenario; this application does not limit this. To avoid frequent entry or exit from the degradation state due to instantaneous jitter, a continuous trigger count, a moving average window, or a recovery threshold can also be set. For example, the degradation and desensitization mode can only be entered when any of the above trigger conditions are met for multiple consecutive sampling periods.

[0071] When the degradation trigger condition is met, the target processing core can stop performing rule matching and character-level mask replacement field by field. Instead, it performs byte-block-based hash replacement processing on the payload contained in the current metadata to be processed. Specifically, the target processing core divides the payload into a sequence of byte blocks according to a preset block length. Each byte block can include a fixed number of consecutive bytes, such as 16 bytes, 32 bytes, 64 bytes, or 128 bytes. Any remaining bytes at the end of the payload that are less than one block length can be used directly as the last byte block or padded according to a preset padding method. The preset block length can be configured based on the average message length, CPU cache line size, hash algorithm computation overhead, and de-identification strength requirements; no specific limitations are specified here.

[0072] Furthermore, the target processing core performs a preset secure hash algorithm operation on each byte block in the byte block sequence to obtain a hash digest value corresponding to each byte block. The secure hash algorithm can be SHA-256, SHA-1, SM3, BLAKE2, or other algorithms capable of mapping the original byte content to an irreversible digest value. For each byte block, the target processing core uses the original byte content of that byte block as input to the hash algorithm to obtain a hash digest value of fixed or configurable length. If the hash digest value length is greater than the corresponding byte block length, the first or last few bytes of the hash digest value can be truncated as replacement content. If the hash digest value length is less than the corresponding byte block length, the hash digest value can be repeatedly padded, the salt value calculation result can be appended, or preset padding bytes can be added to form replacement content with the same length as the original byte block. By keeping the length of the replacement content consistent with the length of the original byte block, changes to the effective payload length can be avoided as much as possible, thereby reducing the complexity of subsequent protocol field adjustments.

[0073] After obtaining the hash digest value or the hash replacement content after length adaptation for each byte block, the target processing core replaces the original byte content of the corresponding byte block in the payload with each hash digest value to obtain the replaced payload. Further, the target processing core recalculates the checksum based on the replaced payload and the protocol header field of the input packet, and writes the updated checksum into the protocol header field. The checksum may include IP header checksum, TCP checksum, UDP checksum, or other integrity check fields required by other protocols; for network cards using hardware checksum offloading, a checksum offloading flag can also be set in the packet descriptor, allowing the network card to perform checksum calculation during transmission. Finally, the target processing core combines the updated protocol header field with the replaced payload to generate a downgraded and de-identified packet, and outputs this downgraded and de-identified packet to a downstream queue, forwarding interface, or storage module.

[0074] This embodiment also discloses an electronic device, as shown in the reference. Figure 3The electronic device may include: at least one processor 301, at least one communication bus 302, user interface 303, network interface 304, and at least one memory 305.

[0075] The communication bus 302 is used to enable communication between these components.

[0076] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0077] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0078] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0079] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0080] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the disclosure in this specification. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A dynamic data anonymization method for 10 Gigabit Ethernet traffic, characterized in that, The method includes: The system acquires input packets received at the network port, performs protocol parsing on the input packets, and extracts the payload characteristics contained in the input packets. The preset FPGA hardware processing engine is invoked to perform parallel regular feature matching on the load features, and a hit status bitmap is generated based on the matching start offset and matching byte length in the matching result. The input message and the hit status bitmap are aggregated to generate metadata to be processed, and the metadata to be processed is written into a preset large page memory space through a preset direct memory access channel. The data to be processed is read from the large page memory space by the preset data plane development kit polling driver, and the network throughput performance parameter set corresponding to the current processing cycle is collected by the data plane development kit polling driver. The network throughput performance parameter set is input into a preset long short-term memory network traffic prediction model, and the peak throughput prediction parameter is output. A scheduling instruction is generated based on the difference between the peak throughput prediction parameter and the preset capacity assessment parameter. The number of physical cores in the preset active state in the preset worker thread pool is adjusted based on the scheduling instruction, and the preset active core mapping table is updated synchronously. Based on the preset receiver scaling algorithm and the active core mapping table, the metadata to be processed is distributed to the target processing cores in the worker thread pool as determined by the scheduling instruction. The target processing core reads the hit status bitmap contained in the metadata to be processed, determines the plaintext sensitive fields marked by the hit status bitmap, performs character replacement processing based on preset masking rules on the plaintext sensitive fields, and combines them to generate a de-identified message.

2. The method according to claim 1, characterized in that, The process involves invoking a preset FPGA hardware processing engine to perform parallel regular feature matching on the load characteristics, and generating a hit status bitmap based on the matching start offset and matching byte length in the matching result, including: Read the regular expression rule set from the built-in rule storage area of ​​the FPGA hardware processing engine; The load features and the regular expression rule set are respectively loaded into the multi-parallel state machine of the FPGA hardware processing engine. The load features are then subjected to byte-by-byte synchronous scanning calculation according to each parallel state machine, and the corresponding rule hit record is output. The rule hit records are aggregated, and the matching start offset and the matching byte length are extracted based on the aggregated rule hit records. The bit positions in the preset all-zero initial bitmap with the matching start offset as the starting position and the matching byte length as the coverage range are set and flipped to generate the hit status bitmap.

3. The method according to claim 1, characterized in that, After reading the metadata to be processed stored in the large page memory space through the preset data plane development kit polling driver, and collecting the network throughput performance parameter set corresponding to the current processing cycle through the data plane development kit polling driver, the method further includes: Receive a rule update trigger signal carrying the target version number and the updated regular expression rule set; allocate a new storage area of ​​the corresponding size in the big page memory space based on the byte length of the updated regular expression rule set; and write the updated regular expression rule set into the new storage area to construct a new rule table. Based on the address mapping relationship between the set of rule entries that have changed in the updated regular expression rule set and the built-in rule storage area of ​​the FPGA hardware processing engine, a reconfiguration instruction is generated that covers the storage address range corresponding to the set of rule entries that have changed. The reconfiguration command is sent to the FPGA hardware processing engine, the reconfiguration completion status signal returned by the FPGA hardware processing engine is read, and the rule version identifier on the FPGA hardware processing engine side is updated to the target version number based on the reconfiguration completion status signal. The old rule table pointer currently held by the polling driver of the data plane development kit is read, and the old rule table pointer is switched to point to the new rule table through an atomic pointer swap operation. After all processing cycles holding the old rule table pointer have completed the reading and processing of the metadata to be processed, the original storage area corresponding to the old rule table pointer is released, and the rule version identifier of the polling driver side of the data plane development kit is updated to the target version number. The rule version identifier on the FPGA hardware processing engine side is compared with the rule version identifier on the polling driver side of the data plane development kit. When the two version identifiers are consistent, it is confirmed that the software and hardware rule versions are synchronized. When the two version identifiers are inconsistent, the reconfiguration instruction is re-executed.

4. The method according to claim 1, characterized in that, The process of inputting the set of network throughput performance parameters into a preset long short-term memory network traffic prediction model, outputting peak throughput prediction parameters, generating scheduling instructions based on the difference between the peak throughput prediction parameters and preset capacity assessment parameters, and adjusting the number of active physical cores in a preset worker thread pool based on the scheduling instructions and synchronously updating a preset active core mapping table includes: The set of network throughput performance parameters corresponding to a consecutive preset number of processing cycles is arranged in time sequence to form a sliding window sequence. The sliding window sequence is loaded into the long short-term memory network traffic prediction model to perform forward inference and output the peak throughput prediction parameters corresponding to the future preset step size. The capacity assessment parameter is determined by multiplying the current number of active physical cores in the worker thread pool with a preset single-core throughput benchmark value. The difference between the peak throughput prediction parameter and the capacity assessment parameter is calculated. When the difference is not lower than a preset expansion trigger threshold, an expansion instruction is generated. When the difference is lower than a preset shrinkage trigger threshold, a shrinkage instruction is generated. The expansion instruction or the shrinkage instruction is used as the scheduling instruction. Based on the scheduling instructions, determine the physical cores to be activated and the physical cores to be hibernated, perform an activation operation on the physical cores to be activated, and perform a hibernation operation on the physical cores to be hibernated, to obtain an adjusted set of active physical cores. Read the set of receiving queues recorded before adjustment in the active core mapping table, remap each receiving queue in the set of receiving queues to each active physical core in the set of active physical cores after adjustment, obtain the updated correspondence between active physical core numbers and receiving queues, and write the updated correspondence between active physical core numbers and receiving queues into the active core mapping table.

5. The method according to claim 4, characterized in that, The step of remapping each receive queue in the receive queue set to each active physical core in the adjusted active physical core set, to obtain the updated correspondence between active physical core numbers and receive queues, includes: The total number of active physical cores within the adjusted set of active physical cores is counted. A preset modulo hash operation is performed based on the queue number of each receiving queue and the total number of active physical cores to obtain the target active physical core number corresponding to each receiving queue. Based on the target active physical core number, a binding relationship is established between each receiving queue in the receiving queue set and the active physical core corresponding to the number in the adjusted active physical core set, thereby generating the updated correspondence between the active physical core number and the receiving queue. Synchronize the updated correspondence between active physical core numbers and receiving queues with the active core mapping table.

6. The method according to claim 1, characterized in that, The process involves reading the hit status bitmap contained in the metadata to be processed through the target processing core, determining the plaintext sensitive fields marked by the hit status bitmap, performing character replacement processing based on preset masking rules on the plaintext sensitive fields, and combining them to generate a de-identified message, including: The target processing core separates the hit status bitmap and payload from the metadata to be processed, traverses the hit status bitmap, and converts the bit numbers of the hit status bitmap that are set into the corresponding byte offset positions in the payload to obtain the starting byte offset position and field byte length of the plaintext sensitive field. Based on the starting byte offset position and the field byte length, the complete byte content of the plaintext sensitive field is read from the payload, and the complete byte content is matched with a preset mask rule template to determine the character replacement method corresponding to the plaintext sensitive field; According to the character replacement method, a preset mask character filling operation is performed on the middle byte content after the preset prefix retention length and before the preset suffix retention length in the plaintext sensitive field to obtain the replaced desensitized field content; The replaced de-identified field content is written back to the original byte offset position corresponding to the plaintext sensitive field in the payload. The checksum is recalculated based on the written-back payload and the protocol header field of the input message. The updated checksum is written into the protocol header field. The protocol header field and the written-back payload are combined to generate the de-identified message.

7. The method according to claim 6, characterized in that, The method further includes: The downstream queue depth and processing latency corresponding to the target processing core are continuously monitored. When the downstream queue depth is not lower than a preset queue depth trigger threshold or the processing latency is not lower than a preset latency trigger threshold, the degradation trigger condition is determined to be met. When the degradation triggering condition is met, the payload of the metadata to be processed is divided into a sequence of byte blocks according to a preset block length. A preset secure hash algorithm is performed on each byte block in the sequence of byte blocks to obtain the hash digest value corresponding to each byte block. The original byte content in the payload is replaced with each of the hash digest values. The checksum is recalculated based on the replaced payload and the protocol header field and written into the protocol header field. The protocol header field and the replaced payload are combined to generate a downgraded and de-identified message.

8. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-7.

Images