A data processing method, apparatus, device, and medium

Abstract

The application discloses a data processing method and device, equipment and medium, and relates to the technical field of data processing, and the method comprises the steps that a preset data processing environment is deployed, a trusted environment container cluster is formed based on the data processing environment, a user-submitted data processing task is acquired, security review is performed, a Pod is created for the data processing task in the case that the security review is passed, and original data and a data processing script are stored in a temporary storage directory; the temporary storage directory is mounted by using a data sandbox container, the original data is downloaded by the data sandbox container and the original data is encrypted, the temporary storage directory is mounted by using a logic sandbox container, and the data processing script is run by the logic sandbox container, the original data is decrypted, and the original data is processed by using the data processing script. The application provides a full-link and full-process secure data processing scheme, and ensures that data injection, encrypted storage, secure calculation and result output are all under strict protection.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a data processing method, apparatus, device, and medium. Background Technology

[0002] With the rapid development of big data and cloud computing technologies, data has become one of the core assets of enterprises. Analyzing and mining massive amounts of data can greatly improve decision-making efficiency, optimize operational processes, and create new value. However, releasing the value of data is often accompanied by severe security and compliance challenges, especially when dealing with sensitive data such as personal privacy information, trade secrets, financial data, and medical and health information. Any leakage could cause irreparable damage. Currently, enterprises need to ensure the confidentiality and integrity of sensitive data during storage, transmission, and processing, and strictly limit the flow of data to unauthorized environments or external entities.

[0003] To meet the requirements of data security and compliance, the industry has proposed various privacy-preserving computing technologies, such as federated learning, secure multi-party computation, and Trusted Execution Environments (TEEs). TEEs create an isolated, protected memory region through hardware, ensuring the confidentiality and integrity of code and data even if the operating system kernel or virtual machine monitor is compromised. This provides a hardware-level foundation for secure cloud data processing. Containerization technologies such as Docker and container orchestration platforms such as Kubernetes provide lightweight, portable, and elastic application packaging and runtime methods. By combining TEEs with containerization technologies, secure data processing services can be delivered in a cloud-native manner, offering excellent scalability and resource utilization.

[0004] However, current data processing solutions that combine TEE and containers still have the following problems:

[0005] It may focus only on the security of the computation process within the TEE, but neglect the security of the entire data chain from input to output. If the links such as how data is safely entered into the TEE, how it is stored outside the TEE, and how the processing results are safely output are not protected, and at the same time, there is a lack of pre-security review and dynamic behavioral constraints on the computation tasks submitted by users, especially the code, malicious or vulnerable code may endanger the TEE environment itself or cause data anomalies.

[0006] Therefore, how to provide end-to-end, end-to-end secure data processing solutions that can provide efficient and flexible data processing services while meeting security and compliance requirements is an important issue that the industry urgently needs to address. Summary of the Invention

[0007] In view of this, embodiments of the present invention provide a data processing method, apparatus, device, and medium to solve the problems that TEE and container data processing schemes neglect the end-to-end security of data from input to output and lack prior security review and dynamic behavioral constraints on user-submitted computing tasks, especially code.

[0008] According to a first aspect, embodiments of the present invention provide a data processing method, the method comprising:

[0009] Deploy a pre-defined data processing environment, and form a trusted environment container cluster based on the data processing environment; a security injection webhook is registered in the trusted environment container cluster;

[0010] Retrieve user-submitted data processing tasks; data processing tasks include data task requests, raw data, data processing scripts, and user authentication tokens.

[0011] Security audits are performed using secure injection webhooks. If the security audit passes, a Pod consisting of a data sandbox container and a logic sandbox container is created for the data processing task, and the raw data and data processing scripts are stored in a temporary storage directory.

[0012] A temporary storage directory is mounted using a data sandbox container. The data sandbox container downloads and encrypts the original data. After the data sandbox container finishes execution, a logic sandbox container mounts the temporary storage directory and runs a data processing script to decrypt and process the original data.

[0013] After the logical sandbox process is complete, delete the Pod and its associated temporary storage directory.

[0014] In conjunction with the first aspect, in the first embodiment of the first aspect, the step of using secure injection Webhook for security auditing, and creating a Pod consisting of a data sandbox container and a logic sandbox container for the data processing task if the security audit passes, and storing the raw data and data processing scripts in a temporary storage directory, specifically includes:

[0015] Determine the temporary network address for accessing the original data and verify that the temporary access URL is valid.

[0016] If the temporary network address for accessing the original data is confirmed to be valid, static validation is performed on the code of the data processing script to determine whether there are any errors in the data processing script.

[0017] If the data processing script is confirmed to be error-free, a unique task number and task creation timestamp are generated for the data processing task. The user authentication token is then parsed by the security injection webhook to obtain the user's unique number.

[0018] Store the raw data and data processing scripts in a temporary storage directory and generate a unique storage number for the temporary storage directory;

[0019] The encryption key for the data processing task is generated by hashing the user's unique ID, the task's unique ID, and the task's creation timestamp using an encryption algorithm.

[0020] The encryption key, user authentication token, temporary access network address, storage unique number, and data processing script are populated into the preset configuration description file to generate the Pod creation request and the Pod's container configuration file;

[0021] Based on the Pod creation request, a security audit is performed by a security injection webhook, and if the security audit passes, the Pod is created according to the container configuration file.

[0022] In conjunction with the first embodiment of the first aspect, in the second embodiment of the first aspect, the step of performing a security audit by a security-injected webhook based on the Pod creation request, and creating the Pod according to the container configuration file if the security audit passes, specifically includes:

[0023] Based on the Pod creation request, a security-injected Webhook performs identity authentication using the unique identifier.

[0024] Once identity authentication is confirmed to be successful, a security-injected Webhook performs permission verification based on the user's unique ID, the task's unique ID, and the storage's unique ID.

[0025] Once the authorization is verified, the trusted environment identifier in the container configuration file is read by the security injection webhook; the trusted environment identifier is a preset annotation of the Pod.

[0026] Without reading the trusted environment identifier, the container configuration file is parsed and the Pod is created from the container configuration file;

[0027] Upon reading the trusted environment identifier, add the environment variables and volume mounts required for the trusted environment container cluster to the container configuration file, parse the modified container configuration file, and create the Pod from the modified container configuration file.

[0028] In conjunction with the second implementation of the first aspect, in the third implementation of the first aspect, the step of adding the environment variables and volume mounts required for the trusted environment container cluster to the container configuration file upon reading the trusted environment identifier, parsing the modified container configuration file, and creating the Pod from the modified container configuration file specifically includes:

[0029] Upon reading the trusted environment identifier, the security injection webhook modifies the container configuration file, adding environment variables and volume mounts to the container configuration file;

[0030] Parse the modified container configuration file to obtain the configuration parsing results;

[0031] Based on the configuration resolution results, a temporary storage directory is mounted and a data sandbox container is created. The data sandbox container then downloads and encrypts the original data.

[0032] Once it is determined that the creation of a data sandbox container to download the raw data and encrypt the raw data are complete, a temporary storage directory is mounted and a logical sandbox container is created. The logical sandbox container constructs a virtual environment for data processing scripts and uses the data processing scripts to process the raw data.

[0033] In conjunction with the third implementation of the first aspect, in the fourth implementation of the first aspect, the step of mounting a temporary storage directory and creating a data sandbox container based on the configuration parsing result, and having the data sandbox container download and encrypt the original data, specifically includes:

[0034] Create a data sandbox container in the Pod based on the configuration resolution results;

[0035] Create a corresponding encrypted directory using a data sandbox container;

[0036] The data sandbox container receives the encryption key and uses it to mount the temporary storage directory to the encrypted directory;

[0037] Based on the temporary access network address, the data sandbox container downloads the raw data required for the data processing task;

[0038] The original data is written to an encrypted directory, and the original data is encrypted using an encryption algorithm when it is written to the encrypted directory.

[0039] In conjunction with the fourth implementation of the first aspect, in the fifth implementation of the first aspect, the step of determining that the creation of a data sandbox container to download the original data and encrypt the original data is complete, and then, based on the configuration parsing result, mounting a temporary storage directory and creating a logical sandbox container, wherein the logical sandbox container constructs a virtual environment for data processing scripts and uses the data processing scripts to process the original data, specifically includes:

[0040] Once it is confirmed that the creation of the data sandbox container for downloading the raw data and encrypting the raw data are complete, the logical sandbox container in the Pod is created based on the configuration parsing result;

[0041] Create the corresponding decryption directory using a logic sandbox container;

[0042] The logic sandbox container receives the encryption key and uses it to mount the temporary storage directory to the decrypted directory.

[0043] Based on the secure isolation image converted from the Docker image carrying the dependency libraries and the trusted environment container cluster, the logical sandbox container creates a virtual environment;

[0044] The logic sandbox container uses a virtual environment to run data processing scripts and reads raw data from an encrypted directory.

[0045] The raw data is encrypted and processed using a data processing script to obtain the data processing result.

[0046] The data processing results are written to a preset data source in an encrypted manner.

[0047] In conjunction with the first aspect, in the sixth embodiment of the first aspect, the deployment of a preset data processing environment, and the formation of a trusted environment container cluster based on the data processing environment, specifically includes:

[0048] Deploy a central processing unit that supports trusted execution environment technology on at least one physical server, and deploy Kubernetes on the physical server;

[0049] Deploy a security injection webhook on each physical server, register the security injection webhook with Kubernetes, and set a preset matching rule and service address for the security injection webhook; the preset matching rule is to only listen for Pod creation.

[0050] Configure security review rules for the security injection webhook; the security review rules are used to perform security reviews on the raw data, data processing scripts, identity information and permission information when a Pod needs to be created. If the security review is passed, the Pod is created, and the security injection webhook injects environment variables and volume mounts into the Pod according to the annotations in the Pod configuration.

[0051] Convert a Docker image containing dependency libraries into a GSC secure isolated image;

[0052] Deploy an HTTP service in a Kubernetes cluster; the HTTP service is used to control the submission and execution of data processing tasks.

[0053] Collect logs of the entire data processing task process.

[0054] According to a second aspect, embodiments of the present invention also provide a data processing apparatus, the apparatus comprising:

[0055] The environment deployment module is used to deploy a pre-defined data processing environment, and a trusted environment container cluster is formed based on the data processing environment; a security injection webhook is registered in the trusted environment container cluster.

[0056] The task submission module is used to obtain data processing tasks submitted by users; data processing tasks include data task requests, raw data, data processing scripts, and user authentication tokens.

[0057] The security audit module is used to perform security audits using secure injection webhooks. If the security audit passes, it creates a Pod for the data processing task, consisting of a data sandbox container and a logic sandbox container, and stores the raw data and data processing scripts in a temporary storage directory.

[0058] The data processing module is used to mount a temporary storage directory using a data sandbox container, download the raw data and encrypt it. After the data sandbox container finishes execution, it uses a logic sandbox container to mount the temporary storage directory, and the logic sandbox container runs a data processing script to decrypt and process the raw data.

[0059] The container deletion module is used to delete Pods and their associated temporary storage directories after the logical sandbox process is complete.

[0060] According to a third aspect, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the data processing methods described above.

[0061] According to a fourth aspect, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the data processing methods described above.

[0062] The data processing method, apparatus, device, and medium of this invention, by deploying a trusted environment container cluster and registering a security injection webhook in the trusted environment container cluster, ensures that the core computational logic of the data processing task is executed in a hardware TEE such as Intel SGX. Even cloud service providers or system administrators cannot snoop on memory data and code logic, providing a higher level of security. Through a dual-sandbox processing architecture of data sandbox containers and logic sandbox containers in the Pod, with the data sandbox containers and logic sandbox containers responsible for separate functions, the raw data is encrypted from the moment it enters the Pod, and is only briefly decrypted for computation when the data processing script performs data processing. Simultaneously, the dual-sandbox processing architecture ensures that the corresponding encryption keys only appear briefly in the memory of the two task-related sandbox containers, with a short lifespan, thus forming a fully encrypted data stream and achieving high end-to-end security. It combines network isolation, data encryption, TEE, and other multi-layered security mechanisms of the dual-sandbox processing structure. This constitutes a defense-in-depth system, which can minimize the probability of information leakage even in the most extreme cases, greatly raising the threshold for attackers. The entire data processing task takes place in a TEE controlled by the enterprise. The raw data and the corresponding data processing results meet the data processing compliance requirements. Furthermore, every key step from task submission, identity authentication, authorization, security review, key generation, data access to computation completion is recorded in detailed and non-repudiable logs, providing a solid data foundation for compliance audits. The trusted environment container cluster is built on Kubernetes, which allows for mixed deployment with other non-secure workloads. Kubernetes' automatic scaling mechanism can also cope with traffic fluctuations, avoiding the resource silos and waste problems of traditional security hardware solutions. New data processing tasks submitted by users are quickly started in Pod units, and resources are immediately released after the data processing task is completed, realizing on-demand use of computing resources.

[0063] This invention provides a secure data processing solution covering the entire chain and process, ensuring that every step from data injection, encrypted storage, secure computation to result output is under strict protection. It also establishes a comprehensive security review and audit mechanism to conduct multi-dimensional control and traceability of tasks, code, and user behavior, meeting the compliance requirements of data processing. Attached Figure Description

[0064] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:

[0065] Figure 1 A flowchart illustrating the data processing method provided by the present invention is shown;

[0066] Figure 2 This diagram illustrates the process of task submission in the data processing method provided by the present invention.

[0067] Figure 3 This diagram illustrates the process of container creation in the data processing method provided by the present invention.

[0068] Figure 4 This diagram illustrates the execution flow of the data sandbox container in the data processing method provided by the present invention.

[0069] Figure 5 This diagram illustrates the execution flow of the logic sandbox container in the data processing method provided by the present invention.

[0070] Figure 6 A schematic diagram of the data processing apparatus provided by the present invention is shown;

[0071] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] With the rapid development of big data and cloud computing technologies, data has become one of the core assets of enterprises. Analyzing and mining massive amounts of data can greatly improve decision-making efficiency, optimize operational processes, and create new value. However, releasing the value of data is often accompanied by severe security and compliance challenges, especially when dealing with sensitive data such as personal privacy information, trade secrets, financial data, and medical and health information. Any leakage could cause irreparable damage. Currently, enterprises need to ensure the confidentiality and integrity of sensitive data during storage, transmission, and processing, and strictly limit the flow of data to unauthorized environments or external entities.

[0074] For example, building an independent, closed network environment disconnected from the internet can fundamentally block the channels for data leakage. This method is highly secure, but extremely expensive, with complex infrastructure construction and maintenance, low resource utilization, poor scalability, and difficulty in adapting to rapid business changes. At the network level, firewalls and virtual private networks (VPNs) can be used to divide zones, and at the application level, strict user authentication and role-based access control can be used to restrict data access. This method relies on software policies and is susceptible to configuration errors, abuse of privileges by internal personnel, or bypassing by advanced persistent threats. Alternatively, data stored in databases or file servers can be encrypted. However, conventional text masking methods using simple replacement rules are easily cracked by pattern matching techniques. Furthermore, even encryption systems cannot be absolutely reliable, especially when facing professional hacker groups with powerful computing capabilities.

[0075] To meet the requirements of data security and compliance, the industry has proposed various privacy-preserving computing technologies, such as federated learning, secure multi-party computation, and TEE (Transparent Environment Execution Environment). TEE uses hardware to create an isolated, protected memory region, ensuring that the code and data within it maintain confidentiality and integrity even if the operating system kernel or virtual machine monitor is compromised. This provides a hardware-level foundation for secure cloud data processing. Containerization technologies such as Docker and container orchestration platforms such as Kubernetes provide lightweight, portable, and elastic application packaging and runtime methods. By combining TEE with containerization technologies, secure data processing services can be delivered in a cloud-native manner, offering good scalability and resource utilization.

[0076] However, current data processing solutions that combine TEE and containers still have the following problems:

[0077] It may focus only on the security of the computation process within the TEE, but neglect the security of the entire data chain from input to output. If the links such as how data is safely entered into the TEE, how it is stored outside the TEE, and how the processing results are safely output are not protected, and at the same time, there is a lack of pre-security review and dynamic behavioral constraints on the computation tasks submitted by users, especially the code, malicious or vulnerable code may endanger the TEE environment itself or cause data anomalies.

[0078] In conclusion, how to provide end-to-end, fully secure data processing solutions that meet security and compliance requirements while offering efficient and flexible data processing services is a crucial issue that the industry urgently needs to address.

[0079] Due to the aforementioned technical problems, this invention provides a data processing method aimed at offering a secure data processing solution across the entire chain and process. This ensures that every step, from data injection, encrypted storage, secure computation to result output, is under strict protection. A comprehensive security review and auditing mechanism is established to manage and trace tasks, code, and user behavior from multiple dimensions, meeting the compliance requirements of data processing. This data processing method can be used in electronic devices, including but not limited to computers and mobile terminals. Figure 1 This is a flowchart illustrating a data processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the method may include the following steps:

[0080] S101. Deploy the pre-defined data processing environment and form a trusted container cluster based on Kubernetes based on the data processing environment.

[0081] In this embodiment of the invention, the trusted environment container cluster deployed on the server, i.e., the data processing environment, includes:

[0082] TEE Integrated Containers: These serve as the server-side hardware and infrastructure layer. A TEE integrated container consists of at least one physical server equipped with a CPU that supports TEE technology, such as a physical server supporting Intel SGX. On these physical servers, a standard Kubernetes container orchestration platform is deployed, enabling these physical servers to constitute at least one execution compute node.

[0083] Subsequently, a security injection webhook was deployed on each physical server where Kubernetes was deployed, and the security injection webhook was deployed as a combination of Deployment and Service. Next, the security injection webhook was registered to the Kubernetes cluster through the Kubernetes MutatingWebhookConfiguration resource, and the preset matching rules and the service address of the security injection webhook were set. The preset matching rule can be: listen only for Pod creation.

[0084] In this embodiment of the invention, the security injection webhook is a callback service registered with the Kubernetes cluster. When a user creates a Pod (the minimum deployment container of Kubernetes) in the Kubernetes cluster, the physical server deploying the Kubernetes cluster first sends the Pod creation request to the security injection webhook for review and modification. The security injection webhook determines whether TEE-related configurations need to be enabled for the Pod based on specific annotations in the Pod configuration, such as `tee-enabled: "true"`. If so, the webhook dynamically injects necessary elements into the Pod configuration. For example, it mounts the Intel SGX device driver files (` / dev / sgx / enclave`, ` / dev / sgx / provision`) into the container as a hostPath volume, and requests the SGX device in the container's security context, setting specific environment variables (such as `SGX_AESM_ADDR`) to connect to the SGX service component (AESM). This automatically injects the necessary environment variables and volume mounts for the security mechanism.

[0085] This process is completely transparent to the user, thus enabling automated injection supported by TEE through secure injection of Webhook.

[0086] TEE, such as Intel SGX, requires that the code running must undergo special signing and encapsulation. In this embodiment of the invention, the Gramine open-source library is used to build a secure image. The Gramine toolchain is used to convert the Docker image carrying dependent libraries into a Gramine Shielded Container (GSC) secure isolation image. The GSC secure isolation image contains a minimalist, lightweight operating system protected by Gramine, as well as the user's application. When this image runs in a Kubernetes Pod, the Gramine library is responsible for loading and running the user program inside the Intel SGX enclave, thereby protecting its code logic and processed data from external snooping.

[0087] In this embodiment of the invention, by deploying a secure injection webhook and the converted GSC secure isolation image in the Kubernetes cluster, the Kubernetes cluster enables TEE support. After special configuration, the Kubernetes cluster can not only identify and support TEE-related resource requests, providing a running foundation for upper-layer secure containers, but the TEE integrated container is also responsible for underlying resource scheduling, network communication and storage provisioning.

[0088] After completing TEE support, security auditing capabilities will also be deployed in the Kubernetes cluster to achieve fine-grained permission management and automated audit trail functionality. Whenever a new data processing task's data processing request is issued to the Kubernetes cluster, a validity review will be performed first. Only after the review is passed will a Pod be created and the data processing task executed.

[0089] In this embodiment of the invention, security review rules are configured for the security injection Webhook. The security review rules are used to perform security reviews on the original data, data processing scripts, identity information and permission information when a Pod needs to be created. The Pod is created when the original data and data processing scripts have passed the review. The security injection Webhook injects environment variables and volume mounts into the Pod according to the annotations in the Pod configuration.

[0090] Secondly, throughout the entire lifecycle of the data processing task, comprehensive logs are collected through integration with the Kubernetes API, container runtime logs, and self-implanted log points. These logs include, but are not limited to: task logs, task submission time, user authentication, data source information, security audit results, Pod creation events, container start time, container stop time, gocryptfs mount events (recording key fingerprints), data access, data download volume, script execution output, final task status (success, failure, error messages), and permission verification. These logs are centrally stored in a secure and tamper-proof logging system for compliance reporting and post-event traceability.

[0091] This integrates secure containers and security auditing capabilities into the Kubernetes cluster.

[0092] By securely injecting Webhook and encapsulating Gramine images, users do not need to learn the complex SGX SDK programming model. They can enjoy hardware-level security protection simply by writing ordinary data processing scripts, which greatly reduces the development threshold.

[0093] Task Controller: Deploys an HTTP service in the Kubernetes cluster through Deployment, Service, and Ingress. This HTTP service is used to control the submission and execution of data processing tasks.

[0094] The system initialization and environment deployment are completed in step S101. The operating system and Kubernetes are installed on the hardware server that supports Intel SGX, the security injection webhook service is deployed and registered to the Kubernetes cluster, security audit capabilities are integrated, and finally the task controller is deployed and its network ingress is configured.

[0095] S102. Obtain the data processing task submitted by the user. The data processing task includes: data task request, raw data, data processing script, and user authentication token. That is, the data processing task contains the following parameters: data processing request, raw data, data processing script, and user authentication token.

[0096] In this embodiment of the invention, the raw data provided by the user is uploaded to an object storage or HTTP server and a temporary access network address for the data is obtained, which is the temporary access Uniform Resource Locator (URL) for the raw data. For example, the user uploads the raw data to be processed to a trusted, HTTP-accessible storage service such as an enterprise internal object storage and obtains a temporary access URL.

[0097] Data processing scripts are responsible for processing data according to certain processing logic. Data input and output can be clearly defined through data processing scripts.

[0098] After obtaining the raw data and data processing script prepared by the user, both the raw data and the data processing script can be stored in a preset database to form a data source. Whenever the user submits a new data task request through the API call task controller, the raw data and data processing script corresponding to the current data processing request are retrieved from the data source, i.e., the database, and assembled with the user authentication token to form the data processing task submitted by the user, thereby completing the task preparation for the client.

[0099] S103. Perform security audit using secure injection Webhook. If the security audit passes, create a Pod for the data processing task consisting of a data sandbox container and a logic sandbox container, and store the raw data and data processing scripts in a temporary storage directory.

[0100] In this embodiment of the invention, a security injection webhook integrated in a trusted environment container cluster is used to perform security audits on the raw data and data processing scripts. If the security audit passes, a Pod creation request and a Pod container configuration file are generated, and the raw data and data processing scripts are stored in a temporary storage directory.

[0101] The secure injection webhook verifies the validity of the raw data and the data processing script, as well as checks for errors in the code. This performs a security audit on the raw data and the data processing script. After both the raw data and the data processing script pass the security audit, a Pod creation request and a container configuration file are generated.

[0102] Based on the Pod creation request, a security injection webhook is used to perform a security audit on the container configuration file. If it is determined that the data processing task requires a trusted execution environment, environment variables and volume mounts are injected into the container configuration file. The security injection webhook serializes the Pod creation request into AdmissionReview format and sends it to the webhook. The Pod's authentication token configuration is parsed into a user ID for user authentication. Then, based on the user ID, task ID, storage ID, and other information, user permissions are authenticated. Through this authentication and permission verification, it is ensured that the user and the corresponding data processing task have the relevant permissions for task execution and data access, guaranteeing the legitimacy of task execution.

[0103] S104. Use the data sandbox container to mount a temporary storage directory, download the original data and encrypt the original data. After the data sandbox container finishes execution, use the logic sandbox container to mount a temporary storage directory, run the data processing script, decrypt the original data and process the original data using the data processing script.

[0104] In this embodiment of the invention, a dual-sandbox processing architecture is formed in the Pod. This architecture decomposes a single data processing task into two physically strictly isolated stages: data sandbox container processing and data sandbox container processing. These stages are each carried by two independent sandbox containers located within the same Pod to achieve close collaboration. After the data sandbox container completes its execution, it exits and enters another logical sandbox container within the Pod. If the data sandbox container has not yet completed its execution, it waits for it to finish before exiting and entering another logical sandbox container within the Pod. This ensures forced isolation between the two through execution order and permission restrictions. Specifically, the dual-sandbox processing architecture includes:

[0105] Data Sandbox Container: This container is responsible for data input and encryption. It creates a dedicated temporary storage directory for encrypted data. All data entering this directory must undergo strict format verification and encryption processes beforehand, ensuring that only authorized programs or services can access the plaintext. The data sandbox container is primarily responsible for:

[0106] Allocate storage: Use PersistentVolume (PV) in Kubernetes to create a controlled temporary storage directory for storing encrypted data, ensuring data security at the storage layer.

[0107] Key generation: Each time a data processing task is created, a unique, temporary encryption key is generated for each data processing task by hashing the task parameters such as user ID, task ID, and task creation timestamp, and then using a lightweight hash function such as the MD5 algorithm.

[0108] Data encryption: The user-space encrypted file system tool gocryptfs is used. gocryptfs receives the encryption key generated above and mounts the temporary storage directory with the encryption key. At the moment the file is written, it automatically encrypts the data using the AES-256 symmetric encryption algorithm.

[0109] Data sandbox containers can only access specified sources, which also enables network isolation and provides a more secure data processing environment. This allows users to enjoy the elasticity and efficiency of cloud computing while meeting the most stringent data security and compliance requirements at an affordable cost, which has significant practical value and broad market prospects.

[0110] Logical Sandbox Container: This container sets up an independent virtual environment to host various data processing scripts uploaded by users. The basic dependencies of these scripts, responsible for processing data according to specific logic, need to be pre-packaged into a standard Docker image and allocated to a designated TEE environment for execution. Throughout the entire lifecycle of the data processing task, no external request can directly obtain a copy of its memory state, nor can it use network requests for any form of data reporting. In other words, this container is responsible for executing user logic in an absolutely secure environment. The logical sandbox container runs within a secure container based on the Gramine GSC image, which is protected by an Intel SGX enclave. The logical sandbox container is primarily responsible for:

[0111] Container isolation: User-uploaded data processing scripts undergo the aforementioned security review to ensure they do not contain malicious code or vulnerabilities. They are then executed in an isolated virtual environment to ensure that the processing logic corresponding to the data processing scripts is not spied on or tampered with.

[0112] Access restrictions: Multi-dimensional restrictions are imposed on system permissions and dependent libraries to ensure secure data processing and prevent security issues such as unauthorized access or data leakage.

[0113] S105. After the logical sandbox is processed, delete the Pod and its associated temporary storage directory.

[0114] In this embodiment of the invention, after both the data sandbox container and the logic sandbox container in the Pod have completed execution, the Pod enters the Completed state. The task controller listens for the Pod's state change, extracts complete logs from the Pod logs and its own audit database, generates a final task audit report, and stores it in the database. Simultaneously, the controller triggers cleanup logic, deleting the Pod and its associated temporary storage directory to ensure the one-time use and complete destruction of encrypted data. Finally, the task completion status is returned to the user.

[0115] Since the trusted environment container cluster is built on Kubernetes, it can be deployed in combination with other non-security workloads. Kubernetes' automatic scaling mechanism can also cope with traffic fluctuations, avoiding the resource silos and waste problems of traditional security hardware solutions. With this setting, newly submitted data processing tasks are quickly started in units of Pods, and resources are released immediately after the data processing tasks are completed, realizing on-demand use of computing resources.

[0116] For example, tightly coupling multiple independent service units to form a unified application stack for users to choose and call, while leveraging the inherent advantages of the multi-tenant shared model and advanced intelligent scheduling algorithms to utilize idle resources, can help enterprises significantly reduce expenditures and other related maintenance costs.

[0117] The data processing method of this invention deploys a trusted environment container cluster and registers a security injection webhook within the cluster. This ensures that the core computational logic of the data processing task is executed within a hardware TEE such as an Intel SGX, preventing cloud service providers or system administrators from accessing memory data and code logic, thus providing a higher level of security. Through a dual-sandbox processing architecture—comprising a data sandbox container and a logic sandbox container within the Pod—with the data and logic sandbox containers responsible for separate functions, raw data is encrypted from the moment it enters the Pod and only briefly decrypted for computation during data processing by the data processing script. This dual-sandbox architecture also ensures that the corresponding encryption keys only appear briefly in the memory of the two task-related sandbox containers, resulting in a short lifespan and forming a fully encrypted data stream, achieving high end-to-end security. Combining network isolation, data encryption, and TEE security mechanisms within the dual-sandbox processing structure, a multi-layered security system is constructed, maximizing the prevention of information leakage even in the most extreme circumstances. The probability significantly raises the attack threshold for attackers; the entire data processing task takes place in an enterprise-controlled TEE, and the raw data and corresponding data processing results meet the data processing compliance requirements. Furthermore, every key step from task submission, identity authentication, authorization verification, security review, key generation, data access to computation completion is recorded in detailed and non-repudiable logs, providing a solid data foundation for compliance audits; the trusted environment container cluster is built on Kubernetes, allowing for mixed deployment with other non-security workloads. Kubernetes' automatic scaling mechanism can also handle traffic fluctuations, avoiding the resource silos and waste problems of traditional security hardware solutions. Newly submitted data processing tasks are quickly started in Pod units, and resources are immediately released after the data processing task is completed, achieving on-demand use of computing resources.

[0118] This invention provides a secure data processing solution covering the entire chain and process, ensuring that every step from data injection, encrypted storage, secure computation to result output is under strict protection. It also establishes a comprehensive security review and audit mechanism to conduct multi-dimensional control and traceability of tasks, code, and user behavior, meeting the compliance requirements of data processing.

[0119] Please see Figure 2 In this embodiment of the invention, step S103 specifically includes:

[0120] S1031. Determine the temporary access network address for the original data and verify the validity of the temporary access URL via a HEAD request. That is, verify the validity of the data link for the original data via a HEAD request.

[0121] If the temporary access URL for the original data is determined to be valid, the data processing script will continue to be validated; conversely, if the temporary access URL for the original data is determined to be invalid, a first error message indicating that the original data has an error will be generated and returned to the corresponding client of the user.

[0122] S1032. If the temporary access URL of the original data is found to be valid, perform static validation on the code of the data processing script, and determine whether there are errors such as permission out of bounds, invalid dependencies, code compilation errors, and code complexity errors in the data processing script based on the validation results obtained from the static validation.

[0123] In this embodiment of the invention, static verification is used to perform security audits on data processing scripts. It is also the first security audit in each data processing task. For example, static verification can determine whether the code of the data processing script attempts to execute system commands, attempts to import dangerous modules in the blacklist, has obvious syntax errors, and whether the code complexity is within the allowable range. Static verification can effectively intercept malicious code and code with serious vulnerabilities.

[0124] If the verification results indicate that the data processing script is error-free, the corresponding steps will continue to be executed. Conversely, if the verification results indicate that the data processing script contains errors, the server will also generate a second error message indicating that the data processing script is faulty and return the second error message to the corresponding client of the user.

[0125] It should be noted that regardless of whether the first error message or the second error message is generated, the subsequent process will not continue, meaning that the data processing task for that instance will end immediately.

[0126] S1033. If it is determined that there are no errors in the data processing script, generate the task ID and task creation timestamp of the data processing task, and use the security injection Webhook to parse the user authentication token to obtain the user ID.

[0127] S1034. Create a temporary storage directory, store the original data and data processing scripts in the temporary storage directory, and generate the storage ID of the temporary storage directory.

[0128] In this embodiment of the invention, a temporary storage directory is applied for and created using PersistentVolume, and a corresponding storage ID is generated for the temporary storage directory.

[0129] S1035. Hash the user ID, task ID, and task creation timestamp using an encryption algorithm to generate an encryption key for the data processing task.

[0130] S1036. Populate the encryption key, user authentication token, temporary access URL, storage ID, and data processing script into the predefined YAML attribute configuration description file to generate the Pod creation request and the Pod's container configuration file. The configuration description file includes:

[0131] The first configuration information is the configuration information of the data sandbox container. The image of the data sandbox container is a dedicated image with built-in gocryptfs and data download tools, and a temporary storage directory is mounted.

[0132] The second configuration information is the configuration information for the logical sandbox container. The image of the logical sandbox container is a pre-prepared GSC image, mounted in the same temporary storage directory, and configured to wait for the data sandbox container to complete before further configuration.

[0133] Add a default annotation to the Pod, such as tee-enabled: "true" as mentioned above.

[0134] It should be noted that by filling in information into a pre-defined configuration description file to generate a container configuration file, users only need to focus on the core data processing logic, enabling rapid development, testing, and deployment of data processing tasks.

[0135] In this embodiment of the invention, steps S1034, S1035 and S1036 are executed in parallel, that is, creating a temporary storage directory, generating an encryption key and building a container configuration file are executed in parallel.

[0136] S1037. Based on the Pod creation request, the security injection webhook performs a security review of the user's identity and permission information. If the security review is passed, a Pod integrating a data sandbox container and a logic sandbox container is created based on the container configuration file.

[0137] In this embodiment of the invention, after generating the container configuration file for the Pod, the task controller submits the Pod creation request to the trusted environment container cluster via the Kubernetes API. The Pod creation request is first intercepted by the security injection webhook, which performs a security review on the data processing task. This is the second security review during each data processing task. After the security review is passed, the Pod is created according to the container configuration file. The Pod contains both a data sandbox container and a logical sandbox container.

[0138] Please see Figure 2 and Figure 3 More specifically, step S1037 includes:

[0139] S10371. Based on the Pod creation request, the identity ID is authenticated by a security injection Webhook.

[0140] In this embodiment of the invention, after the task controller submits the Pod creation request to the trusted environment container cluster through the Kubernetes API, a secure Webhook is injected to authenticate the identity ID and determine the user corresponding to the identity ID and whether the user can perform the task processing.

[0141] If the identity authentication is successful, the authorization process will continue; conversely, if the identity authentication fails, a third error message indicating a problem with the identity information will be generated and returned to the user's corresponding client.

[0142] S10372. If the identity authentication is successful, the security injection Webhook will perform permission authentication on the user's unique ID, the task's unique ID, and the storage's unique ID to ensure that the data processing task has the relevant permissions for task execution and data access.

[0143] If the authorization is successful, the trusted environment identifier will continue to be read; conversely, if the authorization fails, a fourth error message indicating a problem with the authorization information will be generated and returned to the user's corresponding client.

[0144] It should be noted that whether a third or fourth error message is generated, the subsequent process will not continue, meaning that the data processing task for that instance will end directly.

[0145] S10373. If the authentication is successful, the trusted environment identifier in the container configuration file is read by the security injection webhook. The trusted environment identifier is the default Pod annotation, namely tee-enabled: "true" mentioned above.

[0146] In this embodiment of the invention, the Trusted Environment (TEE) is determined to be enabled for the current data processing task by using the annotations preset in the Pod.

[0147] S10374. If the trusted environment identifier is not read, parse the container configuration file and create the Pod from the container configuration file.

[0148] S10375. Upon reading the trusted environment identifier, add the environment variables and volume mounts required by the trusted environment container cluster to the container configuration file, parse the modified container configuration file, and create the Pod from the modified container configuration file.

[0149] If the trusted environment identifier cannot be read, the container configuration file is started directly to create the Pod. Alternatively, the container configuration file is modified first by a security injection webhook to add the environment variables and volume mounts required by the trusted environment container cluster to the container configuration file. The security injection webhook recognizes the pod's tee-enabled annotation and adds SGX device mounts, environment variables and other configurations to the pod to ensure that the pod can run on the SGX node. The modified container configuration file is returned to the physical server, and then the container configuration file is started to create the Pod.

[0150] Taking step S10375 as an example, this step more specifically includes:

[0151] Upon receiving a trusted environment identifier, a security-injected webhook modifies the container configuration file, adding environment variables and volume mounts. The modified configuration file is then parsed to obtain the configuration resolution result. Based on this result, a temporary storage directory is mounted, and a data sandbox container is created. This data sandbox container downloads and encrypts the raw data. Finally, after confirming that the creation of the data sandbox container, downloading, and encryption of the raw data are complete, a temporary storage directory is mounted, and a logical sandbox container is created. This logical sandbox container constructs a virtual environment for data processing scripts and uses these scripts to process the raw data.

[0152] The process of creating data sandbox containers and logic sandbox containers also applies to step S10374.

[0153] Please see Figure 2 as well as Figure 4 Taking step S10375 as an example, the data sandbox container downloads the original data and encrypts it in the following way:

[0154] S201. Based on the configuration parsing results, create a data sandbox container in the Pod.

[0155] In this embodiment of the invention, the Kubernetes scheduler assigns the Pod to a node with SGX capability and starts the Pod, starting first the data sandbox container.

[0156] S202. Create a corresponding encrypted directory using the data sandbox container. Whenever a data sandbox container is created, a new directory is generated for that data sandbox container. This new directory is the encrypted directory corresponding to the data sandbox container. Each new data processing task will create a new data sandbox container and a new encrypted directory.

[0157] S203. The data sandbox container receives the encryption key and uses the encryption key to mount the temporary storage directory to the encrypted directory.

[0158] In this embodiment of the invention, the data sandbox container uses gocryptfs to receive the encryption key and uses gocryptfs and the encryption key to mount the temporary storage directory to the encrypted directory.

[0159] Specifically, the data sandbox container uses the mount command in gocryptfs to initialize the temporary storage directory as an encrypted file system with an encryption key and mount it to the encrypted directory.

[0160] S204. Based on the temporary access URL, the data sandbox container downloads the raw data required for the data processing task.

[0161] S205. Write the original data to the encrypted directory, and encrypt the original data using an encryption algorithm while writing it to the encrypted directory. The entire data writing process is completely transparent to the user.

[0162] The data sandbox container will decompress the downloaded raw data to an encrypted directory, thereby writing the raw data into the encrypted directory. At the moment the raw data is written, the data sandbox container automatically encrypts the raw data using encryption algorithms such as AES-256 symmetric encryption.

[0163] Please see Figure 2 as well as Figure 5 The logic sandbox container is created through the following steps:

[0164] S301. After confirming that the creation of the data sandbox container has been completed, the original data has been downloaded and encrypted, and the logical sandbox container in the Pod is created according to the configuration parsing result.

[0165] After the data sandbox container finishes execution, it will automatically exit. After the data sandbox container is detected to have exited successfully, Kubernetes starts the second container in the Pod, namely the logical sandbox container.

[0166] S302. Create a corresponding decryption directory using a logical sandbox container. Whenever a logical sandbox container is created, a new directory is generated for that logical sandbox container. This new directory is the decryption directory corresponding to the logical sandbox container. Each new data processing task will create a new logical sandbox container and a new decryption directory.

[0167] S303. The logic sandbox container receives the encryption key and uses the encryption key to mount the temporary storage directory to the decryption directory.

[0168] In this embodiment of the invention, the logic sandbox container also uses gocryptfs and the same encryption key to mount the temporary storage directory to the encrypted directory.

[0169] S304. Based on the secure isolation image converted from the Docker image carrying the dependency library and the trusted environment container cluster, the logical sandbox container creates a virtual environment. The virtual environment provides hardware-level security based on the secure isolation image and the trusted environment container cluster. Data processing scripts are executed within the virtual environment to perform data processing.

[0170] S305. The logic sandbox container uses a virtual environment to run data processing scripts and reads raw data from the decryption directory.

[0171] S306. Decrypt the read raw data and process the raw data using a data processing script to obtain the data processing result.

[0172] The logical sandbox container is created only after the data sandbox container has finished executing. The logical sandbox container then processes the data. Since the temporary storage directory is mounted to the decryption directory by the logical sandbox container, the logical sandbox container can continuously read the original data from the decryption directory containing the original data. During the original data reading process, the logical sandbox container will automatically decrypt the encrypted original data.

[0173] S307. Write the data processing results into the preset data source in an encrypted manner.

[0174] After the data processing script processes the raw data, it will obtain the data processing result. The data processing result will be automatically encrypted when written to the data source, or it can be directly POSTed back to a specified, secure callback address through a pre-built, strictly authenticated and encrypted HTTP client within the enclave.

[0175] The data processing apparatus provided in the embodiments of the present invention will be described below. The data processing apparatus described below and the data processing method described above can be referred to in correspondence.

[0176] Due to the aforementioned technical problems, this invention also provides a data processing device, which aims to provide a secure data processing solution for the entire chain and process, ensuring that every link from data injection, encrypted storage, secure computation to result output is under strict protection, and establishing a sound security review and audit mechanism to conduct multi-dimensional control and traceability of tasks, code, and user behavior, so as to meet the compliance requirements of data processing. Figure 6 This is a schematic diagram of the structure of a data processing method according to an embodiment of the present invention, such as... Figure 6 As shown, the device may include:

[0177] Environment deployment module 10 is used to deploy a preset data processing environment, and form a trusted environment container cluster based on Kubernetes based on the data processing environment.

[0178] By completing system initialization and environment deployment, the operating system and Kubernetes are installed on a hardware server that supports Intel SGX, a secure injection webhook service is deployed and registered to the Kubernetes cluster, security audit capabilities are integrated, and finally, a task controller is deployed and its network ingress is configured.

[0179] The task submission module 20 is used to obtain data processing tasks submitted by users. These data processing tasks include: a data task request, raw data, a data processing script, and a user authentication token. In other words, a data processing task contains these parameters: a data processing request, raw data, a data processing script, and a user authentication token.

[0180] In this embodiment of the invention, the raw data provided by the user is uploaded to an object storage or HTTP server and a temporary access network address for the data is obtained, which is the temporary access URL of the raw data. For example, the user uploads the raw data to be processed to a trusted, HTTP-accessible storage service such as an enterprise internal object storage and obtains a temporary access URL.

[0181] Data processing scripts are responsible for processing data according to certain processing logic. Data input and output can be clearly defined through data processing scripts.

[0182] After obtaining the raw data and data processing script prepared by the user, both the raw data and the data processing script can be stored in a preset database to form a data source. Whenever the user submits a new data task request through the API call task controller, the raw data and data processing script corresponding to the current data processing request are retrieved from the data source, i.e., the database, and assembled with the user authentication token to form the data processing task submitted by the user, thereby completing the task preparation for the client.

[0183] The security audit module 30 is used to perform security audits using secure injection Webhook. If the security audit passes, it creates a Pod for the data processing task, consisting of a data sandbox container and a logic sandbox container, and stores the raw data and data processing scripts in a temporary storage directory.

[0184] In this embodiment of the invention, a security injection webhook integrated in a trusted environment container cluster is used to perform security audits on the raw data and data processing scripts. If the security audit passes, a Pod creation request and a Pod container configuration file are generated, and the raw data and data processing scripts are stored in a temporary storage directory.

[0185] The secure injection webhook verifies the validity of the raw data and the data processing script, as well as checks for errors in the code. This performs a security audit on the raw data and the data processing script. After both the raw data and the data processing script pass the security audit, a Pod creation request and a container configuration file are generated.

[0186] Based on the Pod creation request, a security injection webhook is used to perform a security audit on the container configuration file. If it is determined that the data processing task requires a trusted execution environment, environment variables and volume mounts are injected into the container configuration file. The security injection webhook serializes the Pod creation request into AdmissionReview format and sends it to the webhook. The Pod's authentication token configuration is parsed into a user ID for user authentication. Then, based on the user ID, task ID, storage ID, and other information, user permissions are authenticated. Through this authentication and permission verification, it is ensured that the user and the corresponding data processing task have the relevant permissions for task execution and data access, guaranteeing the legitimacy of task execution.

[0187] The data processing module 40 is used to mount a temporary storage directory using a data sandbox container, download the raw data and encrypt the raw data. After the data sandbox container finishes execution, it uses a logic sandbox container to mount a temporary storage directory, run a data processing script, decrypt the raw data and process the raw data using the data processing script.

[0188] In this embodiment of the invention, a dual-sandbox processing architecture is formed in the Pod. This architecture decomposes a single data processing task into two logically and physically strictly isolated stages, each carried by a separate sandbox container. These two sandbox containers reside within the same Pod to achieve close collaboration, but are also subject to mandatory isolation through execution order and permission restrictions. Specifically, the dual-sandbox processing module includes:

[0189] Data Sandbox Container: This container is responsible for data input and encryption. The data sandbox container creates a temporary storage directory specifically for storing encrypted data. All data entering this temporary storage directory must undergo strict format verification and encryption processes beforehand to ensure that only authorized programs or services can obtain the plaintext.

[0190] Logical Sandbox Container: This container sets up an independent virtual environment to host various data processing scripts uploaded by users. The basic dependencies of these scripts, responsible for processing data according to specific logic, need to be pre-packaged into a standard Docker image and allocated to a designated TEE environment for execution. Throughout the entire lifecycle of the data processing task, no external request can directly obtain a copy of its memory state, nor can it use network requests for any form of data reporting. In other words, this container is responsible for executing user logic in an absolutely secure environment. The logical sandbox container runs within a secure container based on the Gramine GSC image, which is protected by an Intel SGX enclave. The logical sandbox container is primarily responsible for:

[0191] The container deletion module 50 is used to delete the Pod and its associated temporary storage directory after the logical sandbox has been processed.

[0192] In this embodiment of the invention, after both the data sandbox container and the logic sandbox container in the Pod have completed execution, the Pod enters the Completed state. The task controller listens for the Pod's state change, extracts complete logs from the Pod logs and its own audit database, generates a final task audit report, and stores it in the database. Simultaneously, the controller triggers cleanup logic, deleting the Pod and its associated temporary storage directory to ensure the one-time use and complete destruction of encrypted data. Finally, the task completion status is returned to the user.

[0193] The data processing device of this invention deploys a trusted environment container cluster and registers a security injection webhook within the cluster. This ensures that the core computational logic of the data processing task is executed within a hardware TEE such as an Intel SGX, preventing cloud service providers or system administrators from accessing memory data and code logic, thus providing a higher level of security. Through a dual-sandbox processing architecture—comprising a data sandbox container and a logic sandbox container within the Pod—with the data and logic sandbox containers responsible for separate functions, raw data is encrypted from the moment it enters the Pod, only briefly decrypted for computation during data processing by the data processing script. This dual-sandbox architecture also ensures that the corresponding encryption keys only appear briefly in the memory of the two task-related sandbox containers, resulting in a short lifespan and forming a fully encrypted data stream, achieving high end-to-end security. Combining network isolation, data encryption, and TEE security mechanisms within the dual-sandbox processing structure, a multi-layered security system is constructed, maximizing the prevention of information leakage even in the most extreme circumstances. The probability of attack is greatly increased, raising the threshold for attackers. The entire data processing task takes place in a TEE controlled by the enterprise. The raw data and the corresponding data processing results meet the data processing compliance requirements. Detailed and non-repudiable logs record every key step from task submission, identity authentication, authorization, security review, key generation, data access to computation completion, providing a solid data foundation for compliance auditing. The trusted environment container cluster is built on Kubernetes, which allows for mixed deployment with other non-security workloads. Kubernetes' automatic scaling mechanism can also handle traffic fluctuations, avoiding the resource silos and waste problems of traditional security hardware solutions. Newly submitted data processing tasks are quickly started in Pod units, and resources are immediately released after the data processing task is completed, realizing on-demand use of computing resources.

[0194] This invention provides a secure data processing solution covering the entire chain and process, ensuring that every step from data injection, encrypted storage, secure computation to result output is under strict protection. It also establishes a comprehensive security review and audit mechanism to conduct multi-dimensional control and traceability of tasks, code, and user behavior, meeting the compliance requirements of data processing.

[0195] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical commands stored in the memory 730 to execute a data processing method, which includes:

[0196] Deploy a pre-defined data processing environment, and form a trusted environment container cluster based on the data processing environment; a security injection webhook is registered in the trusted environment container cluster;

[0197] Retrieve user-submitted data processing tasks; data processing tasks include data task requests, raw data, data processing scripts, and user authentication tokens.

[0198] Security audits are performed using secure injection webhooks. If the security audit passes, a Pod consisting of a data sandbox container and a logic sandbox container is created for the data processing task, and the raw data and data processing scripts are stored in a temporary storage directory.

[0199] A temporary storage directory is mounted using a data sandbox container. The data sandbox container downloads and encrypts the original data. After the data sandbox container finishes execution, a logic sandbox container mounts the temporary storage directory and runs a data processing script to decrypt and process the original data.

[0200] After the logical sandbox process is complete, delete the Pod and its associated temporary storage directory.

[0201] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as a software functional container and, when sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0202] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the data processing method provided by the above methods, the method comprising:

[0203] Deploy a pre-defined data processing environment, and form a trusted environment container cluster based on the data processing environment; a security injection webhook is registered in the trusted environment container cluster;

[0204] Retrieve user-submitted data processing tasks; data processing tasks include data task requests, raw data, data processing scripts, and user authentication tokens.

[0205] Security audits are performed using secure injection webhooks. If the security audit passes, a Pod consisting of a data sandbox container and a logic sandbox container is created for the data processing task, and the raw data and data processing scripts are stored in a temporary storage directory.

[0206] A temporary storage directory is mounted using a data sandbox container. The data sandbox container downloads and encrypts the original data. After the data sandbox container finishes execution, a logic sandbox container mounts the temporary storage directory and runs a data processing script to decrypt and process the original data.

[0207] After the logical sandbox process is complete, delete the Pod and its associated temporary storage directory.

[0208] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the data processing methods provided above, the method comprising:

[0209] Deploy a pre-defined data processing environment, and form a trusted environment container cluster based on the data processing environment; a security injection webhook is registered in the trusted environment container cluster;

[0210] Retrieve user-submitted data processing tasks; data processing tasks include data task requests, raw data, data processing scripts, and user authentication tokens.

[0211] Security audits are performed using secure injection webhooks. If the security audit passes, a Pod consisting of a data sandbox container and a logic sandbox container is created for the data processing task, and the raw data and data processing scripts are stored in a temporary storage directory.

[0212] A temporary storage directory is mounted using a data sandbox container. The data sandbox container downloads and encrypts the original data. After the data sandbox container finishes execution, a logic sandbox container mounts the temporary storage directory and runs a data processing script to decrypt and process the original data.

[0213] After the logical sandbox process is complete, delete the Pod and its associated temporary storage directory.

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method, characterized by, The method includes: Deploy a pre-defined data processing environment, and form a trusted environment container cluster based on the data processing environment; a security injection webhook is registered in the trusted environment container cluster; Retrieve user-submitted data processing tasks; data processing tasks include data task requests, raw data, data processing scripts, and user authentication tokens. Security audits are performed using secure injection webhooks. If the security audit passes, a Pod consisting of a data sandbox container and a logic sandbox container is created for the data processing task, and the raw data and data processing scripts are stored in a temporary storage directory. A temporary storage directory is mounted using a data sandbox container. The data sandbox container downloads and encrypts the original data. After the data sandbox container finishes execution, a logic sandbox container mounts the temporary storage directory and runs a data processing script to decrypt and process the original data. After the logical sandbox process is complete, delete the Pod and its associated temporary storage directory.

2. The data processing method according to claim 1, characterized in that, The process involves using secure injection webhooks for security auditing. If the audit passes, a Pod consisting of a data sandbox container and a logic sandbox container is created for the data processing task. The raw data and data processing scripts are then stored in a temporary storage directory. Specifically, this includes: Determine the temporary network address for accessing the original data and verify that the temporary access URL is valid. If the temporary network address for accessing the original data is confirmed to be valid, static validation is performed on the code of the data processing script to determine whether there are any errors in the data processing script. If the data processing script is found to be error-free, a unique task number and task creation timestamp are generated for the data processing task. The user authentication token is then parsed by a security injection Webhook to obtain the user's unique number. Store the raw data and data processing scripts in a temporary storage directory and generate a unique storage number for the temporary storage directory; The encryption key for the data processing task is generated by hashing the user's unique ID, the task's unique ID, and the task's creation timestamp using an encryption algorithm. The encryption key, user authentication token, temporary access network address, storage unique number, and data processing script are populated into the preset configuration description file to generate the Pod creation request and the Pod's container configuration file; Based on the Pod creation request, a security audit is performed by a security injection webhook, and if the security audit passes, the Pod is created according to the container configuration file.

3. The data processing method according to claim 2, characterized in that, The process of performing a security audit using a security-injected webhook based on the Pod creation request, and creating the Pod according to the container configuration file if the security audit passes, specifically includes: Based on the Pod creation request, a security-injected Webhook performs identity authentication using the unique identifier. Once identity authentication is confirmed to be successful, a security injection webhook will perform permission verification based on the user's unique ID, the task's unique ID, and the storage's unique ID. Once the authorization is verified, the trusted environment identifier in the container configuration file is read by the security injection webhook; the trusted environment identifier is a preset annotation of the Pod. Without reading the trusted environment identifier, the container configuration file is parsed and the Pod is created from the container configuration file; Upon reading the trusted environment identifier, add the environment variables and volume mounts required for the trusted environment container cluster to the container configuration file, parse the modified container configuration file, and create the Pod from the modified container configuration file.

4. The data processing method according to claim 3, characterized in that, Upon receiving the trusted environment identifier, the process involves adding the necessary environment variables and volume mounts for the trusted environment container cluster to the container configuration file, parsing the modified container configuration file, and creating the Pod from the modified configuration file. Specifically, this includes: Upon reading the trusted environment identifier, the security injection webhook modifies the container configuration file, adding environment variables and volume mounts to the container configuration file; Parse the modified container configuration file to obtain the configuration parsing results; Based on the configuration resolution results, a temporary storage directory is mounted and a data sandbox container is created. The data sandbox container then downloads and encrypts the original data. Once it is confirmed that the creation of a data sandbox container to download and encrypt the raw data is complete, a temporary storage directory is mounted and a logical sandbox container is created based on the configuration parsing results. The logical sandbox container then constructs a virtual environment for the data processing scripts and uses the scripts to process the raw data.

5. The data processing method according to claim 4, characterized in that, The process of mounting a temporary storage directory and creating a data sandbox container based on the configuration resolution results, followed by the data sandbox container downloading and encrypting the original data, specifically includes: Based on the configuration resolution results, create a data sandbox container in the Pod; Create a corresponding encrypted directory using a data sandbox container; The data sandbox container receives the encryption key and uses it to mount the temporary storage directory to the encrypted directory; Based on the temporary access network address, the data sandbox container downloads the raw data required for the data processing task; The original data is written to an encrypted directory, and the original data is encrypted using an encryption algorithm when it is written to the encrypted directory.

6. The data processing method according to claim 5, characterized in that, Once it is determined that the creation of the data sandbox container for downloading and encrypting the original data is complete, based on the configuration parsing result, a temporary storage directory is mounted and a logical sandbox container is created. The logical sandbox container constructs a virtual environment for data processing scripts and uses these scripts to process the original data. Specifically, this includes: Based on the configuration parsing results, create a logical sandbox container in the Pod; Create the corresponding decryption directory using a logic sandbox container; The logic sandbox container receives the encryption key and uses it to mount the temporary storage directory to the decrypted directory. Based on the secure isolation image converted from the Docker image carrying the dependency libraries and the trusted environment container cluster, the logical sandbox container creates a virtual environment; The logic sandbox container uses a virtual environment to run data processing scripts and reads raw data from the decryption directory. The raw data is decrypted and processed using a data processing script to obtain the data processing result. The data processing results are written to a preset data source in an encrypted manner.

7. The data processing method of claim 1, wherein, The deployment of the preset data processing environment, and the formation of a trusted environment container cluster based on the data processing environment, specifically includes: Deploy a central processing unit that supports trusted execution environment technology on at least one physical server, and deploy Kubernetes on the physical server; Deploy a security injection webhook on each physical server, register the security injection webhook with Kubernetes, and set a preset matching rule and service address for the security injection webhook; the preset matching rule is to only listen for Pod creation. Configure security review rules for the security injection webhook; the security review rules are used to perform security reviews on the raw data, data processing scripts, identity information and permission information when a Pod needs to be created. If the security review is passed, the Pod is created, and the security injection webhook injects environment variables and volume mounts into the Pod according to the annotations in the Pod configuration. Convert a Docker image containing dependency libraries into a secure, isolated image; Deploy an HTTP service in a Kubernetes cluster; the HTTP service is used to control the submission and execution of data processing tasks. Collect logs of the entire data processing task process.

8. A data processing apparatus, characterized by, The device includes: The environment deployment module is used to deploy a pre-defined data processing environment, and a trusted environment container cluster is formed based on the data processing environment; a security injection webhook is registered in the trusted environment container cluster. The task submission module is used to obtain data processing tasks submitted by users; data processing tasks include data task requests, raw data, data processing scripts, and user authentication tokens. The security audit module is used to perform security audits using secure injection webhooks. If the security audit passes, it creates a Pod for the data processing task, consisting of a data sandbox container and a logic sandbox container, and stores the raw data and data processing scripts in a temporary storage directory. The data processing module is used to mount a temporary storage directory using a data sandbox container, download the raw data and encrypt it. After the data sandbox container finishes execution, it uses a logic sandbox container to mount the temporary storage directory, and the logic sandbox container runs a data processing script to decrypt and process the raw data. The container deletion module is used to delete Pods and their associated temporary storage directories after the logical sandbox process is complete.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the data processing method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the data processing method as described in any one of claims 1 to 7.

Images