Data security processing method and device, computer device and storage medium

Abstract

The application discloses a data security processing method and device, computer equipment and a storage medium, relates to the technical field of data processing, and comprises the following steps: acquiring to-be-processed data in initial data and first position information of the to-be-processed data; extracting the to-be-processed data and performing security processing on the to-be-processed data; determining second position information of processed data, and merging non-to-be-processed data in the initial data and the processed data into target data according to the first position information and the second position information. The problems of complex data security processing technology architecture, low data security processing efficiency and possible change of source data content can be solved. The method can quickly and accurately distribute data and determine to-be-processed data; the to-be-processed data can be quickly and accurately processed, and the efficiency of data security processing is improved; the processed data and the non-to-be-processed data in the initial data are efficiently merged according to address information, and the integrity and availability of the target data are ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data security processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] To ensure data security, confidential data such as user contact information, identity information, and address information must be securely processed. For example, confidential data can be processed through deformation, replacement, shielding, and format-preserving encryption (FPE) to achieve reliable data protection.

[0003] Current data security processing technologies can be categorized into static and dynamic processing. Static processing uses algorithms to securely process data. However, this technology alters the source data content and suffers from low data processing efficiency, making it difficult to meet the needs of low-latency, high-throughput businesses. Dynamic processing uses a data source proxy to parse SQL (Structured Query Language) statements to match security processing criteria, rewrite the query SQL, or intercept and protect the data before returning it to the application, achieving secure processing of confidential data. Dynamic processing requires proxy deployment, resulting in a complex architecture. The implementation process also requires SQL rewriting or interception, which complicates the process and reduces data security efficiency. Summary of the Invention

[0004] In view of this, the present application provides a data security processing method, apparatus, computer equipment and storage medium to solve the problems of complex data security processing technology architecture, low data security processing efficiency and possible changes to source data content.

[0005] In a first aspect, the present application provides a data security processing method, the method comprising:

[0006] When initial data to be securely processed is obtained, obtaining the data to be processed in the initial data and first position information of the data to be processed in the initial data;

[0007] Extracting the data to be processed from the initial data, and performing security processing on the data to be processed to obtain the processed data;

[0008] The second position information of the processed data is determined, and the data to be processed in the initial data is replaced with the corresponding processed data according to the first position information and the second position information to obtain the target data.

[0009] In a second aspect, the present application provides a data security processing device, the device comprising:

[0010] A data acquisition module, configured to, upon acquiring initial data to be securely processed, acquire the data to be processed in the initial data and first position information of the data to be processed in the initial data;

[0011] The data processing module is used to extract the data to be processed from the initial data, and perform security processing on the data to be processed to obtain the processed data;

[0012] The data replacement module is used to determine the second position information of the processed data, and replace the data to be processed in the initial data with the corresponding processed data according to the first position information and the second position information to obtain the target data.

[0013] In a third aspect, the present application provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the computer instructions to thereby execute the data security processing method of the first aspect or any corresponding embodiment thereof.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data security processing method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0015] In a fifth aspect, the present application provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the data security processing method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0016] Through this application, since the method first obtains the data to be processed and the first position information of the data to be processed in the initial data; extracts the data to be processed and performs security processing on the data to be processed; determines the second position information of the processed data, and merges the non-data to be processed in the initial data and the processed data into the target data based on the first position information and the second position information. This can solve the problems of complex data security processing technology architecture, low data security processing efficiency and possible changes to the source data content. This method can quickly and accurately divert data and determine the data to be processed; it can quickly perform security processing on the data to be processed and improve the efficiency of data security processing; based on the address information, the processed data is efficiently merged with the non-data to be processed in the initial data to ensure the integrity and availability of the target data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of this application or the technical solutions in related technologies, the following is a brief introduction to the drawings required for use in the specific implementation methods or related technical descriptions. Obviously, the drawings described below are some implementation methods of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 is a flowchart of data security processing according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of a data security processing framework based on high-speed computing link technology and field programmable gate array according to an embodiment of the present application;

[0020] Figure 3 is a flowchart of another data security processing method according to an embodiment of the present application;

[0021] Figure 4 is a structural block diagram of a data security processing device according to an embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of the hardware structure of the computer device of an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0024] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0025] The Flink streaming engine consists of two components: the JobManager and the TaskManager. The Flink engine utilizes a master-slave architecture. The JobManager runs on the master node and is responsible for receiving submitted tasks from clients, scheduling jobs, coordinating tasks, and managing checkpoints. It also manages task scheduling and resource management for the entire Flink cluster. The TaskManager runs on the slave nodes and is responsible for executing specific tasks and allocating and managing resources for each node. It executes specific tasks and manages multiple task slots. A task is a unit of work running on a task slot; a task slot is a resource-isolated unit that, by default, uses all memory. A dataflow is a logical diagram of the data flow between operators during the execution of a streaming job.

[0026] Currently, stream computing engines are widely used in fields such as real-time risk control for financial transactions and real-time analysis of medical data. However, the security and performance bottlenecks of their data processing are becoming increasingly prominent. Traditional stream data security processing technologies rely on software agents, resulting in security processing delays typically exceeding 50 microseconds. This process consumes excessive CPU and storage resources, making it difficult to meet the demands of low-latency, high-throughput services. Furthermore, traditional stream data security processing technologies are based on the TCP / IP (Transmission Control Protocol / Internet Protocol) data transmission method, resulting in a high rate of packet loss and retransmission during data fragmentation and reassembly, severely impacting data reliability. Data security processing technologies can be categorized as static security processing and dynamic decryption. Static security processing involves exporting production data to a target storage medium using algorithms such as deformation, substitution, masking, and format-preserving encryption (FPE). This exported, securely processed data actually alters the content of the source data. Dynamic data security processing uses a proxy on the data source to parse SQL (Structured Query Language) statements to match security processing conditions, rewrite data queries using SQL, or intercept and protect the data before returning it to the application, thereby achieving data security processing. Dynamic security processing uses a proxy deployment approach, requiring operations and maintenance personnel to access the database through the dynamic security processing device before data access results are securely processed according to system rules. Traditional dynamic data security processing systems require proxy deployment, resulting in a complex architecture. The implementation process also requires SQL rewriting or interception protection, which is complex and slows down data security processing.

[0027] Based on the above, embodiments of the present application provide a data security processing method. This method uses a Compute Express Link (CXL) device as the data storage medium for task management nodes in a stream computing engine. A new stream computing-oriented read / write interface is developed, enabling the stream computing engine to read and write data from the CXL device. A large-scale model-based data identification algorithm extracts and analyzes features of the stream data in the stream computing engine, identifying key words for secure processing. An FPGA (Field-Programmable Gate Array)-based computational microinstruction algorithm is developed for the stream engine to implement batch secure data processing within the CXL device. This method, based on the stream computing engine's stream batch secure processing framework, enables rapid data diversion, secure data processing via the FPGA, and merging of processed data with non-processed data using CXL technology and the Remote Direct Memory Access protocol. This method achieves rapid data diversion and efficient secure processing, meeting high-efficiency data security requirements.

[0028] According to an embodiment of the present application, a data security processing embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, for example: a computer, a server, etc., and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] In this embodiment, a data security processing method is provided. Figure 1 is a flow chart of a data security processing method according to an embodiment of the present application. Figure 1 As shown, the process includes the following steps:

[0030] Step S101 : when initial data to be securely processed is obtained, the data to be processed in the initial data and first position information of the data to be processed in the initial data are obtained.

[0031] Specifically, data security processing includes: desensitizing sensitive data; protecting key data to avoid malicious destruction, modification or leakage. This embodiment takes desensitizing sensitive data as an example for explanation. There are many ways for the hardware that executes the data security processing method to obtain the initial data to be securely processed, such as: reading the initial data from a fast computing link technology device; receiving a data packet sent by other devices, in which the initial data exists; or Figure 2As shown, the stream computing engine obtains initial data from the data source, and the stream computing engine starts sensitive data recognition through the LlmRecogDesensitization function, and receives initial data from the data source of the stream computing engine. The LlmRecogDesensitization function is a composite function that combines language model recognition and sensitive information desensitization processing. After calling this function, feature extraction is first performed, and the stream computing engine encapsulates the preprocessed data into a fixed format. Then, through the data write interface in CxlOperDriver, the initial data is written to the first memory device. The data source is a fast computing link technology Type 2 (Type 2) device, such as an accelerator card or an add-on card. CxlOperDriver is an operation driver for the fast computing link technology.

[0032] The large model is invoked to execute a pending data detection algorithm to detect the initial data in the first memory device and identify the pending data. The detection process includes: the large model performs a forward propagation calculation, setting the output dimension of the large model to 10, corresponding to 10 data types (such as ID card, bank card, email address, etc.). Based on the output of the large model, the data in the initial data is determined to be pending data. The pending data may include data that needs to be kept confidential or data that involves user privacy. The first location information of the pending data in the initial data is obtained.

[0033] Step S102: extracting the data to be processed from the initial data, and performing security processing on the data to be processed to obtain processed data.

[0034] Specifically, if Figure 2 As shown, the stream computing engine invokes the field programmable gate array (FPGA) to execute a data processing algorithm, securely processing the data and extracting the data to be processed from the initial data based on the first location information. Security processing strategies include, for example, string replacement, anonymization, and encryption. The FPGA securely processes the data to be processed, for example, by generating a data processing instruction set based on the security processing strategy, executing the data processing instruction set, processing each piece of data to be processed, and writing the processed data to the third memory device.

[0035] Step S103 , determining the second position information of the processed data, and replacing the data to be processed in the initial data with the corresponding processed data according to the first position information and the second position information to obtain target data.

[0036] Specifically, if Figure 2As shown, in the third memory device, second location information of the processed data is determined. The stream computing engine merges the processed data with data that does not need to be processed in the initial data, obtains non-to-be-processed data from the first memory device based on the first location information, obtains processed data from the third memory device based on the second location information, merges the non-to-be-processed data with the processed data, replaces the to-be-processed data in the initial data with the corresponding processed data, merges the data into target data, and writes the target data to the fourth memory device, facilitating external storage or downstream systems to obtain the target data in the fourth memory device.

[0037] The data security processing method provided in this embodiment obtains the data to be processed and the first location information of the data to be processed in the initial data; extracts the data to be processed and performs security processing on the data to be processed; determines the second location information of the processed data, and merges the non-data to be processed in the initial data with the processed data into the target data based on the first location information and the second location information. This method can quickly and accurately divert data and determine the data to be processed; it can quickly perform security processing on the data to be processed, improving the efficiency of data security processing; and based on the address information, efficiently merges the processed data with the non-data to be processed in the initial data to ensure the integrity and availability of the target data. This solves the problems of complex data security processing technology architecture, low data security processing efficiency, and the possibility of changing the source data content.

[0038] In this embodiment, another data security processing method applied to the local management system is provided. Figure 3 is a flow chart of another data security processing method applied to a local management system according to an embodiment of the present application. Figure 3 As shown, the process includes the following steps:

[0039] Step S301: When initial data to be securely processed is obtained, the data to be processed in the initial data and first position information of the data to be processed in the initial data are obtained.

[0040] For details on how to implement this step, see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0041] As an optional embodiment, the above step S301 includes steps S3011 to S3015.

[0042] Step S3011 , preprocessing the initial data, and encapsulating the preprocessed data into a first preset format to obtain a feature set, wherein the feature set includes a first preset number of features.

[0043] Step S3012: transmitting the feature set to a processor, wherein the processor is used to call a preset model.

[0044] Step S3013: Input the feature set into a preset model to obtain an output result, wherein the output result includes the probability that the feature belongs to a second preset number of preset types.

[0045] Step S3014: When the feature belongs to a target type, the feature is used as data to be processed, wherein the target type is a preset type with a probability greater than a preset threshold.

[0046] Step S3015: Determine the first position information of the data to be processed in the initial data.

[0047] Specifically, this embodiment identifies data to be processed from the initial data based on a preset model. For example, sensitive data in the initial data is identified as data to be processed. First, the stream computing engine preprocesses the initial data, including removing HTML (Hypertext Markup Language) tags and illegal characters.

[0048] The first preset format is, for example, JSON format, which uses key-value pairs to represent data. JSON format is a lightweight data exchange format. The preprocessed data is encapsulated into the first preset format to obtain a feature set. The feature set contains a first preset number of features, and the first preset number represents one or more. For example, a feature is a key-value pair, which represents a set of parameters in the initial data. The key represents the location information of the data, and the value is the numerical value of the parameter. After the stream computing engine completes the preprocessing of the initial data, it writes the feature set to the corresponding memory device through the data writing interface in the operation driver of the fast computing link technology, such as Figure 2 The first memory device in the memory. A processor, such as a graphics processing unit (GPU), can call a preset model. The feature set is transferred from the memory device to the processor, for example, by transferring the feature set to the GPU via the Remote Direct Memory Access (RDMA) protocol.

[0049] Preset models include classification models based on deep neural networks and pre-trained language models based on self-attention mechanisms. The processor calls the preset model, inputs the feature set into the preset model, calls the preset model to perform forward propagation calculations, and sets the output dimension of the preset model based on the preset type corresponding to the data. For example, setting the output dimension to 10 corresponds to 10 preset types, such as ID cards, bank cards, and email addresses. The output result of the preset model is obtained, and the output result is tensor data. The tensor data output by the large model is parsed to extract the preset type probability vector corresponding to each feature (Token). The probability vector is normalized using Softmax to ensure that the sum of all probability values ​​is 1. Based on the probability vector, the probability of the feature belonging to each preset type can be determined. The second preset number is one or more.

[0050] The probability of each feature is compared with a preset threshold. The preset type with a probability greater than the threshold is considered the target type for the corresponding feature. If the feature belongs to the target type, the feature is used as the data to be processed. The stream computing engine is used to determine the first position information of the data to be processed in the initial data.

[0051] In this embodiment, a detection algorithm for data to be processed based on a preset model is used to detect and screen the input feature set, identify the probability that different features in the feature set are data to be processed, and achieve accurate detection of the data to be processed.

[0052] As an optional embodiment, step S3015 includes steps A1 to A5.

[0053] Step A1: Determine first location information of data to be processed in a first memory device and a device identifier of a second memory device.

[0054] Step A2: Determine the data length and target type of the data to be processed.

[0055] Step A3: Generate pointer information of the first location information and encoding information of the target type.

[0056] Step A4: Generate triplet data according to the pointer information, the data length, the encoding information, and a third preset number of data bits, wherein the data bits are used to store the second position information of the processed data.

[0057] Step A5: convert the triplet data into a second preset format, and write the serialized data into a preset storage area of ​​the second memory device through the device identifier.

[0058] Specifically, a stream computing engine is used to determine first location information of the data to be processed in a first memory device, and a device management interface of an operation driver of a fast computing link technology is used to obtain a device handle of the memory device according to a device ID (Identity document) for storing the data to be processed, and the device handle is used as a device identifier of a second memory device.

[0059] The data length and target type of the data to be processed are determined, such as an ID card, bank card, email address, or mobile phone number. Pointer information for the first location information and encoding information for the target type are generated, where the pointer information points to the first location information in the initial data. A third predetermined number of data bits, such as 64 data bits, are used to record pointer information for the second address information. This pointer information can be used to determine the second location information of the processed data.

[0060] Based on the pointer information, data length, encoding information and a third preset number of data bits, triplet data is generated. For example, the features whose probability meets the preset threshold are constructed into a triplet data structure. The triplet data contains 64 bits of pointer information (pointing to the first position information in the initial data), 32 bits of data length, 16 bits of encoding information of the target type and 64 data bits. These data bits need to be initialized to 0. After completing the security processing of the data to be processed, the second position information of the processed data is determined, and the pointer information of the second address information is written into the above 64 data bits.

[0061] The second preset format is, for example, binary format or decimal format. The triple data is converted into the second preset format, and the serialized data is written to a preset storage area of ​​the second memory device through the device identifier. For example, the data is stored in the data storage area of ​​the second memory device through the append write interface of the device handle, and the address information of the triple data in the second memory device is returned to the stream computing engine.

[0062] In this embodiment, triplet data including pointer information, data length, encoding information and a third preset number of data bits is created, and the second position information of the processed data is stored using the data bits to achieve accurate matching of the processed data with the data to be processed in the initial data through the triplet data. At the same time, data merging technology is implemented to ensure the integrity and availability of the target data.

[0063] As an optional embodiment, after step S3013 "obtaining output results", steps B1 to B5 are also included.

[0064] Step B1: Obtain the data length and data format of the feature.

[0065] Step B2: determine whether the data length is equal to the preset length.

[0066] Step B3: If the data length is not equal to the preset length and the probability that the feature belongs to the preset type is greater than the preset value, the probability is modified to the preset value.

[0067] Step B4: determining whether the data format complies with a third preset format.

[0068] Step B5: If the data format does not conform to the third preset format and the probability that the feature belongs to the preset type is greater than a preset value, the probability is modified to the preset value.

[0069] Specifically, this embodiment introduces type filtering rules to check whether the probability of the feature is correct. Type filtering rules include: ID card verification rules, email verification rules, and other type rules.

[0070] Get the data length and data format of the feature. Determine whether the data length is equal to the preset length. The preset lengths are different for different preset types. For example, if the preset type is an ID card, the preset length is 15 or 18 digits; if the preset type is a mobile phone number, the preset length is 11 digits. If the data length is equal to the preset length, determine that the length of the feature meets the requirements. If the data length is not equal to the preset length, determine that the length of the feature does not meet the requirements, and then determine that the feature does not belong to the preset type. If the probability that the feature belongs to a mobile phone number in the preset type is greater than the preset value, modify the probability to the preset value, such as 0, 0.01, or other smaller values.

[0071] The third preset format may include, for example, an email address format, a mobile phone number format, or an IP (Internet Protocol) address format. Whether the data format conforms to the third preset format is determined, for example, by using a regular expression to check whether the feature conforms to the email address format. If the data format does not conform to the third preset format, and the probability that the feature belongs to the preset email address type is greater than a preset value, the probability is modified to the preset value.

[0072] Similarly, features can be verified based on other types of rules, including mobile phone number format, Internet Protocol address format, etc., to ensure the accuracy of the preset type probability.

[0073] In this embodiment, the features are verified based on the ID card verification rules, email verification rules and other types of rules to determine whether the features belong to a preset type and modify the probability based on the judgment result to ensure the accuracy of the preset type probability.

[0074] Step S302: extracting the data to be processed from the initial data, and performing security processing on the data to be processed to obtain processed data.

[0075] For details on how to implement this step, see Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0076] As an optional embodiment, the above step S302 includes steps S3021 to S3025.

[0077] Step S3021: Acquire triplet data and extract data to be processed based on the triplet data.

[0078] Step S3022: Utilize a fourth preset number of data processing units to perform security processing on the data to be processed in parallel to obtain processed data.

[0079] Step S3023: Allocate storage space in the third memory device according to the data capacity of the processed data, and write the processed data into the storage space in the third memory device.

[0080] Step S3024: Determine the second location information of the processed data in the storage space.

[0081] Step S3025: write the second position information into a third preset number of data bits in the triplet data.

[0082] Specifically, this embodiment uses a field programmable gate array to securely process the data to be processed. The field programmable gate array may include the following units: an interface unit, which implements address translation and virtual-to-real address conversion; a data pipeline unit, which implements data reading, data output, data verification, format parsing, instruction control, etc.; a computing array control unit, which implements DSP (Digital Signal Processor) unit data calculations, such as hash calculations, shift calculations, logical operations, encryption calculations, etc.; a security unit, which is mainly used to implement key generation, storage, update, distribution, verification, etc.

[0083] This embodiment uses data security processing as an example to illustrate the desensitization of sensitive data. The stream computing engine will call the data set desensitization function to perform security processing on the data to be processed. The parameters of the data set desensitization function consist of the data set size of the data to be processed, triple data, and data processing strategy. Through this function, the field programmable gate array can be used to implement the data desensitization function. The function will send an instruction to the controller of the field programmable gate array. The controller parses the instruction to obtain the triple data, performs address conversion on the data to be processed based on the triple data, obtains the first address information, and allocates storage space in the field programmable gate array. Then, based on the first address information, the data to be processed is extracted from the initial data and written to the field programmable gate array.

[0084] The data processing unit is, for example, a computing array control unit. The field programmable gate array includes a fourth preset number of data processing units, which can independently perform data security processing operations. The fourth preset number represents a plurality, and there is no specific quantity limit here. The data processing strategies in the data set desensitization function are, for example, digital string replacement, anonymization, encryption, etc. According to the desensitization strategy, the controller compilation algorithm generates a data desensitization operation instruction set for the data processing unit, and stores the data desensitization operation instruction set to the instruction controller of the field programmable gate array. Start the desensitization operation instruction set, use the fourth preset number of data processing units to execute the desensitization operation instruction set in parallel, perform security processing on each piece of data to be processed, obtain processed data, and write the processed data into the memory of the field programmable gate array.

[0085] Based on the data capacity of the processed data, the fast computing link technology operation driver is invoked to allocate storage space in the third memory device. The address of the space is returned to the field programmable gate array output instruction, which then invokes the data output instruction to write the processed data into the storage space in the third memory device. Second location information of the processed data in the storage space is determined, and the second location information is written into a third predetermined number of data bits in the triple data. In this process, the field programmable gate array interacts directly with the fast computing link technology device, eliminating the need for interaction with the central processing unit (CPU) via the system bus.

[0086] It should be noted that this embodiment combines fast computing link technology with field programmable gate arrays to scalably implement encryption of large amounts of data, and can also be extended to privacy computing of stream computing engines.

[0087] In this embodiment, multiple data processing units in a field programmable gate array (FPGA) are used to securely process data in parallel, significantly improving the efficiency of data security processing. Furthermore, this embodiment directly interacts with the FPGA and fast computing link technology devices, effectively reducing the CPU load and accelerating the data security processing process.

[0088] Step S303 , determining the second position information of the processed data, and replacing the data to be processed in the initial data with the corresponding processed data according to the first position information and the second position information to obtain target data.

[0089] For details on how to implement this step, see Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0090] As an optional embodiment, step S303 includes: step S3031 to step S3033.

[0091] Step S3031: Acquire triple data, acquire second position information of processed data in the triple data, and acquire the processed data from a third memory device according to the second position information.

[0092] Step S3032: Determine the processed data corresponding to the data to be processed based on the triple data, the first position information, and the second position information, and replace the data to be processed in the initial data with the corresponding processed data to obtain target data.

[0093] Step S3033: write the target data into the fourth memory device.

[0094] Specifically, the stream computing engine obtains triple data from a preset storage area of ​​the second memory device, or obtains triple data returned by the LlmRecogDesensitization function, obtains second location information of the processed data from the triple data, and obtains the processed data from the third memory device based on the second location information.

[0095] Determine, based on the triplet data and the first position information, the to-be-processed data corresponding to the processed data in the initial data. Replace the to-be-processed data in the initial data with the corresponding processed data according to the first position information in the triplet data, merge the non-to-be-processed data with the processed data to form target data, and write the target data into the fourth memory device.

[0096] As an optional embodiment, before step S301 of "obtaining the data to be processed in the initial data and the first position information of the data to be processed in the initial data", the method further includes steps C1 to C3.

[0097] Step C1: When a data read request is received, the data read request is parsed to obtain request information.

[0098] Step C2: Generate a read transaction packet containing request information, and convert the read transaction packet into a link control unit packet of a preset protocol using a root port.

[0099] Step C3, sending the link control unit packet to the target device to obtain response data, wherein the response data includes initial data, and the response data is obtained after the root port performs protocol conversion on the intermediate data, and the intermediate data is determined in the data stored in the target device based on the address information, and the address information is obtained after parsing the link control unit packet.

[0100] Specifically, this embodiment uses a fast computing link technology device as the storage medium of the task management node in the stream computing engine, and newly develops a read and write interface for the stream computing engine. The stream computing engine can implement data read and write operations on the fast computing link technology type 2 device.

[0101] The process by which the stream computing engine implements a read operation on a Fast Compute Link Technology Type 2 device includes the following: the stream computing engine's task management node sends a data read request through the Fast Compute Link Technology operation driver. Upon receiving the data read request, the operation driver parses the data read request and extracts the request information, including the device ID, address offset, and data size. It then generates a Fast Compute Link Technology protocol-based read transaction packet containing the request information. The root port then converts the read transaction packet into a link control unit (CXL flit) packet based on the pre-defined protocol. The CXL flit packet contains an operation code (OpCode = 0x01), an address field, a transaction ID, and flow control information.

[0102] A credit control mechanism manages data transmission over the Rapid Compute Link technology link, ensuring non-blocking communication. This link control unit packet is sent over the Rapid Compute Link technology link to the target device, such as a Rapid Compute Link technology Type 2 device. The target device's Rapid Compute Link technology controller parses the link control unit packet, extracts the address information, and maps it to a physical memory address via the Address Translation Unit (ATU). This triggers a burst read operation in the memory controller to retrieve the intermediate data. The intermediate data line is encapsulated according to the Rapid Compute Link technology response packet format, then converted to a PCIe (a high-speed serial computer expansion bus standard) completion packet via the root port and returned to the driver software. The stream computing engine then parses and uses this packet to obtain the response data.

[0103] In addition, the stream computing engine of this embodiment can also perform write operations on Fast Compute Link Technology Type 2 devices. The steps include: the write request is converted into a Fast Compute Link Technology storage instruction by the operation driver. The root port encapsulates the instruction into a link control unit packet and sends it to the device. Each link control unit packet generates a corresponding transaction ID. The device controller resolves the address and drives the memory controller to execute the write operation, returning a write confirmation response after completion. The underlying layer of the operation driver implements the following in the write step: data blocking: for data larger than the maximum transmission unit of the fast computing link technology, the driver software automatically splits it into multiple write transactions, each transaction contains a continuous address block; instruction generation: generates a fast computing link technology write transaction package, including the address, data payload and CRC (Cyclic redundancy check) check code, and adopts asynchronous write mode to improve throughput; flow control: the root port maintains the sending credit mechanism. When the fast computing link technology is ready to write data to the output device, it sends an available space query request to the device end. The device end returns the available space as the credit amount. When the credit amount is exhausted, it suspends sending to avoid link congestion; write confirmation mechanism: after the device completes the write, it returns a write completion response. The driver software matches the confirmation information according to the transaction ID to ensure the reliability of data write.

[0104] In this embodiment, by deploying fast computing link technology endpoint devices and developing dedicated driver software, an efficient read and write channel is constructed between the stream computing engine and the target device, significantly improving data processing performance; a credit control mechanism is used to manage data transmission of the fast computing link technology link to ensure non-blocking communication.

[0105] As an optional embodiment, before step C1 of “parsing the data read request”, the method further includes steps D1 to D4.

[0106] In step D1, a fifth preset number of root ports are configured on the system bus, and base address registers and internal memory information of device endpoints are acquired by using the root ports, wherein the root ports are connected to the device endpoints.

[0107] Step D2: registering a sixth preset number of memory devices in the node metadata of the task management node, wherein the device endpoints are endpoints of the memory devices, and the target devices are included in the memory devices.

[0108] Step D3, allocating address space for the base address register and internal memory information of the memory device in the preset memory space, and performing initialization processing and address space allocation on the memory device.

[0109] Step D4: Write the device information and address space data of the memory device into the node operating system configuration file.

[0110] Specifically, this embodiment uses Fast Compute Link Technology Type 2 devices as storage devices for the task management nodes of the stream computing engine. Fast Compute Link Technology is an industry-supported, cache-coherent, open interconnect protocol for processors, memory expansion, and accelerators. The Fast Compute Link Technology protocol includes three sub-protocols: CXL.io (read / write), CXL.cache (cache), and CXL.mem (memory). Three types of devices can be defined based on these sub-protocols: Type 1 devices, which are accelerator cards or add-in cards installed in PCIe slots. These cards can be integrated into existing systems and communicate directly with the central processing unit (CPU) via the Fast Compute Link Technology interface to provide faster data transfer speeds. They are used for cache devices such as network cards. Type 2 devices, which have all the functions of Type 1 devices, are typically used in scenarios with high-density computing, such as graphics processors. Type 3 devices, which are dedicated storage devices, communicate directly with the host processor and can use the Fast Compute Link Technology protocol to achieve low-latency, high-throughput data transfer. They serve as memory buffers to expand memory bandwidth and capacity. This embodiment develops task management nodes for the stream computing engine that support CXL.io, CXL.cache, and CXL.mem, and implements an operation driver for the stream computing engine that supports registration, memory space allocation, and read and write operation instructions of memory devices using the Fast Compute Link technology.

[0111] One or more Rapid Compute Link technology-based root ports (RPs) are developed and configured on the system bus of the cluster node's central processing unit. These root ports connect to Rapid Compute Link technology device endpoints (EPs), where the device endpoint type is a Rapid Compute Link technology type 2 memory device endpoint. The fifth preset number is an integer greater than one. Furthermore, the root ports enable bidirectional conversion between PCIe transactions and link control unit packets, supporting credit-based flow control to prevent data transmission congestion.

[0112] When the task management node of the stream computing engine starts, the kernel driver scans the Fast Compute Link technology root port through PCIe transactions to obtain the base address register (BAR) and internal memory (HDM) information of the device endpoint. For example, the node calls the kernel driver on the operating system to initiate PCIe transactions (PCIe transactions) to scan the base address register and internal memory information of the Fast Compute Link technology device to obtain the scan information.

[0113] The task management node registers a sixth preset number of memory devices in the node metadata based on the scan information. The sixth preset number represents one or more, and the target device is also a memory device. The preset memory space is the memory space reserved for the task management node. The preset memory space allocates address space for the base address register and internal memory information of the memory device, and adopts a 4KB page alignment strategy for address mapping. The memory device is initialized and the address space is allocated. The memory device initialization process is executed, including: parity bit setting, memory block health status detection, and establishing a mapping table between the system virtual address and the memory device through the address synchronization mechanism. The device information and address space data of the memory device are written into the node operating system configuration file for loading upon restart.

[0114] When the task management node restarts, it reads the configuration information from the operating system configuration file. Upon receiving a data read or write request, it uses instructions to access the internal memory of the memory device. The request is passed to the corresponding root port, which converts the request into a link control unit packet and sends it to the memory device using the Fast Compute Link Protocol. The controller in the memory device then parses the link control unit packet, converts the incoming address, and sends the converted request to the underlying memory controller, enabling the reading and writing of the memory device's memory resources.

[0115] In this embodiment, the fast computing link technology device serves as the storage device of the task management node of the stream computing engine. Each task management node is equipped with at least one root port, which is configured on the system bus of the stream computing task node for connecting to memory devices, realizing functions such as registering memory devices, memory management, read-write conversion, and address mapping.

[0116] As an optional embodiment, step S3012 “transmitting the feature set to the processor” includes steps E1 to E5.

[0117] Step E1: Determine the identifiers of a first memory device and a processor for storing a feature set.

[0118] Step E2: establishing a connection relationship between the first memory device and the processor through preset components and identifiers.

[0119] In step E3, based on the connection relationship, the memory area storing the feature set in the first memory device is registered as an accessible area, wherein the accessible area is a memory area that can be directly accessed by the processor through remote memory direct access technology.

[0120] In step E4, the feature set is transferred from the accessible area to the local memory of the processor through remote memory direct access technology.

[0121] Step E5: confirm whether to transmit the feature set to the processor according to the operation completion queue.

[0122] Specifically, communication between stream computing engine nodes utilizes conventional Ethernet, with network transmission using the TCP / IP protocol. During the operation of the stream computing engine, data may be leaked or incomplete due to network anomalies. Therefore, this embodiment transmits data to a processor, such as a graphics processor, using the Remote Memory Direct Access (RMA) protocol. RMA is a new direct memory access technology that allows computers to directly access the memory of other computers without requiring processor processing. Technically, RMA is a high-speed, remote memory access technology with a fully optimized smart network card (Smart NIC) and software architecture. This high-performance remote direct data access is achieved by integrating the RMA protocol into hardware (such as a network card) and supporting both zero-copy and kernel bypass. RMA data transmission does not require CPU intervention, allowing applications to access the remote host's memory without consuming any CPU resources. The preset components include, for example, the RdmaLlmGpuWrite component, which is a mechanism or software module specifically designed to utilize remote memory direct access technology for high-speed data writing between a graphics processor and a remote system.

[0123] The remote memory direct access connection is initialized, the first memory device used to store the feature set is determined, and the representation of the processor is determined, for example: the target GPU ID is 0. Through the preset component, a remote memory direct access connection relationship is established between the first memory device and the processor according to the identifier. Based on the established connection relationship, the memory area used to store the feature set in the first memory device is registered as an accessible area for the remote memory direct access technology, and the local key (Local Key) and remote key (Remote Key) of the memory area are obtained. The local key and remote key are used to ensure data security. The processor can directly access the data in the accessible area through the remote memory direct access technology.

[0124] Perform a data transfer operation to transfer the feature set from the accessible region to the processor's local memory using Remote Memory Direct Access (RMDA). This involves constructing a Remote Memory Direct Access (RMDA) write request, specifying the processor's buffer address, the local memory region's key, and the data length. This transfer uses zero-copy techniques to transfer data directly from the accessible region to the processor's memory using RDMA. The RDMA operation completion queue is polled to confirm the RDMA operation's completion, ensuring the feature set has been successfully transferred to the GPU.

[0125] In this embodiment, the zero-copy technology of the remote memory direct access protocol is used to achieve high-speed transmission of the feature set from the accessible area to the processor, further optimizing the efficiency of data security processing, and ensuring the integrity of the transmitted data and that the data will not be leaked during the transmission process.

[0126] The data security processing method provided in this embodiment obtains the data to be processed and the first location information of the data to be processed in the initial data; extracts the data to be processed and performs security processing on the data to be processed; determines the second location information of the processed data, and merges the non-data to be processed in the initial data with the processed data into the target data based on the first location information and the second location information. This method can quickly and accurately divert data and determine the data to be processed; it can quickly perform security processing on the data to be processed, improving the efficiency of data security processing; and based on the address information, efficiently merges the processed data with the non-data to be processed in the initial data to ensure the integrity and availability of the target data. This solves the problems of complex data security processing technology architecture, low data security processing efficiency, and the possibility of changing the source data content.

[0127] As an optional embodiment, in step S3022, "using a fourth preset number of data processing units to perform security processing on the data to be processed in parallel to obtain processed data", the security processing process may include steps F1 to F3.

[0128] Step F1: Import the data to be processed into a data security processing environment. The data to be processed includes data identification and data characteristics.

[0129] Step F2: Encrypt the data identifier of each data using a data processing algorithm in a data security processing environment.

[0130] In step F3, the multiple data imported into the data security processing environment are obfuscated to achieve data de-identification.

[0131] Specifically, the de-identified data identifier and its corresponding data feature still have a mapping relationship. The data to be processed includes multiple data, which are composed of (data identifier, data feature), for example, (data identifier = ID1, data feature = feature 1), (data identifier = ID2, data feature = feature 2). Among them, the data identifier may include private data such as the user's ID number, mobile phone number, and the company's business license number. If the data identifier is used directly, it may lead to the leakage of private data and cause data security issues.

[0132] In this embodiment, the data to be processed is imported into a data security processing environment. The characteristic of the data security processing environment is that the encryption key generation and data encryption process are carried out in the environment, and the outside world cannot directly obtain the key and encryption logic in the environment. In the data security processing environment, the data identifier of each data is encrypted using a data processing algorithm to obtain an encrypted data identifier. For example, ID1 is encrypted and converted to ID1', and ID2 is encrypted and converted to ID2', thereby anonymizing the above-mentioned private data. At the same time, the corresponding relationship between the data characteristics of each data and the encrypted data identifier continues to be maintained. For example, reference Figure 2 , (data identifier = ID1', data feature = feature 1), (data identifier = ID2', data feature = feature 2); at the same time, the order of the multiple data imported into the data security processing environment is disrupted through the obfuscation method, making it impossible to establish a corresponding relationship between the multiple data originally input into the data security processing environment and the multiple data after de-identification. For example, the order of the multiple data input into the data security processing environment is ID1, ID2, while the order of the multiple data after de-identification is ID2', ID1', which is different.

[0133] In addition, different data encryption keys can be used to encrypt multiple data imported into the data security processing environment in batches; each batch includes one or more data, and different batches of data correspond to different data encryption keys. When using a data processing algorithm to encrypt multiple data, in order to improve encryption efficiency, a batch encryption method can be adopted, and different data encryption keys can be generated to encrypt different batches of data. One batch corresponds to one data encryption key. Since different data encryption keys are used, the security of private data is further improved. A batch can also contain only one data, so that the security of private data is better. The above-mentioned data encryption keys corresponding to different batches can be the same or different, and can be specifically determined based on the security level requirements of the business scenario.

[0134] In the embodiment of the present application, sample de-identification is achieved, thereby anonymizing the private data of the sample identification, and it is also impossible to correspond the de-identified sample to the original sample, achieving a better security processing effect.

[0135] In this embodiment, a data security processing device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. Details that have already been described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0136] This embodiment provides a data security processing device, such as Figure 4 As shown, including:

[0137] The data acquisition module 401 is used to acquire the data to be processed in the initial data and the first position information of the data to be processed in the initial data when the initial data to be processed is acquired;

[0138] The data processing module 402 is used to extract the data to be processed from the initial data and perform security processing on the data to be processed to obtain processed data;

[0139] The data replacement module 403 is configured to determine the second position information of the processed data, and replace the data to be processed in the initial data with the corresponding processed data according to the first position information and the second position information to obtain the target data.

[0140] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0141] The data security processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0142] The present application also provides a computer device having the above Figure 4 The data security processing device shown.

[0143] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present application. Figure 5 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0144] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include an integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0145] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0146] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0148] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0149] The embodiments of the present application also provide a computer-readable storage medium. The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0150] Part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes but is not limited to a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0151] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations shall fall within the scope defined by the present application.

Claims

1. A data security processing method, characterized in that: The method comprises: When initial data to be securely processed is obtained, obtaining data to be processed in the initial data and first position information of the data to be processed in the initial data; Extracting the data to be processed from the initial data, and performing security processing on the data to be processed to obtain processed data; Determining the data length and target type of the data to be processed; generating pointer information of the first location information and encoding information of the target type; generating a triplet of data according to the pointer information, the data length, the encoding information, and a third preset number of data bits, wherein the data bits are used to store second position information of the processed data; The second position information of the processed data is determined, and according to the first position information and the second position information, the data to be processed in the initial data is replaced with the corresponding processed data to obtain target data.

2. The method according to claim 1, characterized in that The obtaining of the data to be processed in the initial data and first position information of the data to be processed in the initial data includes: Preprocessing the initial data and encapsulating the preprocessed data into a first preset format to obtain a feature set, wherein the feature set includes a first preset number of features; transmitting the feature set to a processor, wherein the processor is configured to call a preset model; Inputting the feature set into the preset model to obtain an output result, wherein the output result includes a probability that the feature belongs to a second preset number of preset types; In the case where the feature belongs to a target type, the feature is used as the data to be processed, wherein the target type is a preset type with a probability greater than a preset threshold; Determine the first position information of the data to be processed in the initial data.

3. The method according to claim 2, characterized in that The determining the first position information of the data to be processed in the initial data includes: Determine the first location information of the data to be processed in the first memory device and the device identifier of the second memory device; The triplet data is converted into a second preset format, and the serialized data is written into a preset storage area of ​​the second memory device through the device identifier.

4. The method according to claim 3, characterized in that The step of extracting the data to be processed from the initial data and performing security processing on the data to be processed to obtain processed data includes: Acquire the triplet data, and extract the data to be processed according to the triplet data; Using a fourth preset number of data processing units to perform security processing on the data to be processed in parallel to obtain the processed data; allocating storage space in a third memory device according to the data capacity of the processed data, and writing the processed data into the storage space in the third memory device; Determining the second location information of the processed data in the storage space; The second position information is written into a third preset number of the data bits in the triplet data.

5. The method according to claim 4, characterized in that The step of replacing the to-be-processed data in the initial data with the corresponding processed data according to the first position information and the second position information to obtain target data includes: Acquire the triple data, acquire the second position information of the processed data in the triple data, and acquire the processed data from the third memory device according to the second position information; Determine the processed data corresponding to the data to be processed according to the triple data, the first position information, and the second position information, and replace the data to be processed in the initial data with the corresponding processed data to obtain target data; The target data is written into the fourth memory device.

6. The method according to claim 2, characterized in that After obtaining the output result, the method further includes: Obtaining the data length and data format of the feature; Determining whether the data length is equal to a preset length; If the data length is not equal to the preset length, and the probability that the feature belongs to the preset type is greater than a preset value, modifying the probability to the preset value; Determining whether the data format conforms to a third preset format; If the data format does not conform to the third preset format and the probability that the feature belongs to the preset type is greater than the preset value, the probability is modified to the preset value.

7. The method according to claim 1, characterized in that Before obtaining the data to be processed in the initial data and the first position information of the data to be processed in the initial data, the method further includes: When a data read request is received, the data read request is parsed to obtain request information; Generate a read transaction packet containing the request information, and convert the read transaction packet into a link control unit packet of a preset protocol using a root port; The link control unit packet is sent to the target device to obtain response data, wherein the response data includes the initial data, and the response data is obtained after the root port performs protocol conversion on the intermediate data, and the intermediate data is determined in the data stored in the target device based on the address information, and the address information is obtained after parsing the link control unit packet.

8. The method according to claim 7, characterized in that Before parsing the data read request, the method further includes: Developing and configuring a fifth preset number of the root ports on the system bus, and using the root ports to obtain base address registers and internal memory information of device endpoints, wherein the root ports are connected to the device endpoints; registering a sixth preset number of memory devices in the node metadata of the task management node, wherein the device endpoints are endpoints of the memory devices and the target devices are included in the memory devices; Allocating address space for the base address register and the internal memory information of the memory device in a preset memory space, and performing initialization processing and address space allocation on the memory device; Write the device information and address space data of the memory device into the node operating system configuration file.

9. The method according to claim 2, characterized in that The transmitting the feature set to a processor comprises: determining an identifier of a first memory device and a processor for storing the feature set; Establishing a connection relationship between the first memory device and the processor through a preset component and the identifier; Based on the connection relationship, registering a memory area storing the feature set in the first memory device as an accessible area, wherein the accessible area is a memory area that is directly accessible to the processor through remote memory direct access technology; transferring the feature set from the accessible area to the local memory of the processor using the remote memory direct access technology; Determine whether to transmit the feature set to the processor according to the operation completion queue.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data security processing method according to any one of claims 1 to 9 by executing the computer instructions.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data security processing method according to any one of claims 1 to 9.

12. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the data security processing method according to any one of claims 1 to 9.

Images