Data desensitization processing method and device, electronic equipment, storage medium and product
By encapsulating data anonymization capabilities into an independent script, and utilizing deep learning models and Gaussian blurring methods, cross-platform data anonymization processing is achieved, solving the problems of poor reusability and portability in existing technologies, and improving processing efficiency and applicability.
Patent Information
- Application Number
- CN202511553936.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-28
- Publication Date
- 2026-01-27
AI Technical Summary
Existing data anonymization methods have poor reusability and portability, making them difficult to deploy across platforms. Furthermore, they are tightly coupled with business logic, impacting efficiency and flexibility.
The data anonymization processing capability is encapsulated into an independent script. The target script is called by executing the script with configuration parameters to perform anonymization processing on the data to be processed. It can process big data in parallel, support cross-platform deployment and loosely coupled integration, and use deep learning models to detect sensitive information and perform Gaussian blur processing.
It improves the versatility and reusability of data anonymization processing, reduces the R&D cycle, enhances flexibility and efficiency, supports multiple data formats and scenarios, and adapts to different business needs.
Smart Images
Figure CN121413005A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a data desensitization processing method, apparatus, electronic device, storage medium, and product. Background Technology
[0002] Typically, due to privacy compliance requirements in different scenarios, some information needs to be anonymized. The anonymization methods in related technologies mainly embed anonymization capabilities as a service into business systems or platforms. This approach is closely linked to actual business logic and has poor reusability and portability. Summary of the Invention
[0003] Therefore, this disclosure provides a data anonymization processing method, apparatus, electronic device, storage medium, and product.
[0004] Firstly, this disclosure provides a data anonymization method, which includes:
[0005] Obtain a data processing request sent by the business side. The data processing request includes script execution configuration parameters and data to be processed or data storage path to be processed. The script execution configuration parameters are used to indicate the values of variable parameters in the target script. The target script is used to implement data desensitization processing.
[0006] Based on the script execution configuration parameters, the target script is executed to perform de-identification processing on the data to be processed, and the de-identification processing result is obtained.
[0007] The de-identification processing result is sent to the business terminal or stored in the target storage location.
[0008] In one possible embodiment, the step of performing de-identification processing on the data to be processed by executing the target script based on the script execution configuration parameters to obtain the de-identification processing result includes:
[0009] According to the data processing request, obtain the data to be processed;
[0010] If the size of the data to be processed meets the splitting condition, the data to be processed is split into multiple sub-data.
[0011] Based on the script execution configuration parameters, the target script is executed to perform desensitization processing on multiple sub-data in parallel, and multiple sub-desensitization processing results are obtained.
[0012] The multiple sub-desensitization processing results are merged to obtain the desensitization processing result of the data to be processed.
[0013] In one possible embodiment, obtaining the data processing request sent by the service terminal includes:
[0014] The data processing request sent by the business terminal is obtained from the message queue, wherein the message queue stores the data processing request sent by the business terminal, and the node device used to execute the data desensitization processing method is determined based on the current processing resources of multiple node devices in the node system.
[0015] In one possible embodiment, the data processing request is triggered by the business terminal in response to a target script invocation instruction, wherein the target script invocation instruction is associated with a script invocation method.
[0016] In one possible embodiment, after sending the de-identification processing result to the business terminal or storing it in the target storage location, the method further includes:
[0017] Add the de-identification label information to the data to be processed;
[0018] In response to triggering a data deletion task, the data to be processed is deleted.
[0019] In one possible embodiment, the method further includes:
[0020] When a de-identification processing progress query request is received from the business terminal, the current de-identification processing completion status corresponding to the data to be processed is determined, and the current de-identification processing completion status is returned to the business terminal so that the business terminal can display the current de-identification processing completion status on the interface.
[0021] Secondly, this disclosure provides a data desensitization processing apparatus, which includes:
[0022] The acquisition module is used to acquire data processing requests sent by the business end. The data processing requests include script execution configuration parameters and data to be processed or data storage path to be processed. The script execution configuration parameters are used to indicate the values of variable parameters in the target script. The target script is used to implement data desensitization processing.
[0023] The processing module is used to execute the script based on the configuration parameters, and perform de-identification processing on the data to be processed by executing the target script to obtain the de-identification processing result;
[0024] The sending module is used to send the de-identification processing result to the business terminal or store it in the target storage location.
[0025] Thirdly, this disclosure provides an electronic device comprising:
[0026] One or more processors; a memory storing one or more programs that, when executed by the one or more processors, enable the one or more processors to implement the data anonymization processing method described above; and one or more I / O interfaces connected between the processors and the memory, configured to enable information interaction between the processors and the memory.
[0027] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described data desensitization processing method.
[0028] Fifthly, this disclosure provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device performs the above-described data desensitization processing method.
[0029] The data anonymization processing method provided in this disclosure encapsulates data anonymization processing capabilities into an independent script, which can be used as an independent capability and supports cross-platform deployment and operation. In the node system, the node device receives a data processing request sent by the business end, and based on the script execution configuration parameters and the indicated data to be processed in the data processing request, calls and executes the target script to perform anonymization processing on the data to be processed, obtains the anonymization processing result, and then sends the anonymization processing result to the business end or stores it in a target location. In this way, the anonymization processing capability does not need to be embedded in the business process of the business end, and the values of variable parameters in the target script can be instantiated through the script execution configuration parameters, which meets the data anonymization processing requirements. It has stronger versatility, is easy to port and reuse, and is also easy to update and optimize the anonymization processing capability without having to consider the entire business process, thus improving efficiency.
[0030] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0031] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:
[0032] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;
[0033] Figure 2 A flowchart of a data desensitization method provided in this disclosure embodiment;
[0034] Figure 3 A flowchart of another data desensitization method provided in this disclosure embodiment;
[0035] Figure 4 A block diagram of a data desensitization device provided in an embodiment of this disclosure;
[0036] Figure 5 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation
[0037] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.
[0038] As used in this disclosure, the term "and / or" includes any and all combinations of one or more of the related enumerated entries.
[0039] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. As used in this disclosure, the singular forms “a” and “the” are also intended to include the plural forms, unless the context clearly indicates otherwise.
[0040] When the terms “comprising” and / or “made of” are used in this disclosure, they specify the presence of the said feature, integral, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or groups thereof.
[0041] Unless otherwise specified, all terms used in this disclosure (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so specified in this disclosure.
[0042] In fields such as connected vehicles, compliance and privacy requirements often necessitate information anonymization. For example, data such as images and videos collected at the vehicle, road, and cloud levels may require anonymization. The anonymization methods in related technologies mainly embed anonymization capabilities as a service into business systems or platforms. This approach is closely linked to actual business logic. For instance, it requires combining actual business needs, conducting requirements analysis and technical options, adapting to the existing business system's technical architecture and workflow, and implementing the function after determining the solution options. However, this approach suffers from poor reusability and portability.
[0043] To address this, this disclosure provides a data anonymization method that encapsulates data anonymization capabilities into an independent script. This script can be used as a standalone capability and supports cross-platform deployment. In use, a node device in the node system receives a data processing request from the business end. Based on the script execution configuration parameters and the indicated data to be processed in the request, the node calls and executes the target script to anonymize the data, obtaining the anonymization result. This result can then be sent to the business end or stored in a target location. In this way, the anonymization capability does not need to be embedded in the business process of the business end, supporting multiple inter-process calls to achieve loosely coupled integration. Furthermore, the values of variable parameters in the target script can be instantiated through the script execution configuration parameters, meeting the data anonymization requirements. This method is more versatile, easier to port and reuse, and facilitates updates and optimizations of the anonymization capability without considering the entire business process, thus improving efficiency and reducing the development cycle.
[0044] Figure 1 The diagram illustrates an application scenario of the data desensitization method and apparatus provided in the embodiments of this disclosure.
[0045] like Figure 1 As shown, the application scenario of this disclosure embodiment may include a service terminal 110 and a node system 120, and the node system may include multiple node devices 121. The service terminal 110 and the node devices 121 can communicate through a network, which may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.
[0046] Users can use the service terminal 110 to interact with the node device 121 via the network to receive or send messages, etc. Various client applications can be installed on the service terminal, such as web browser applications, search applications, instant messaging tools, social platform software, etc. (for example only).
[0047] The business terminal 110 can be any electronic device with a display screen and web browsing support, including but not limited to smartphones, tablets, laptops, and desktop computers.
[0048] Node device 121 can be an electronic device or server that provides various services. For example, in this embodiment of the present disclosure, a user can input script execution configuration parameters through the interface of the service terminal 110, and then send a data processing request to the node device 121 based on the target script call instruction. After the node device 121 receives the data processing request, it performs data desensitization processing based on the target script and the script execution configuration parameters, and can return the desensitized data to the service terminal 110. The service terminal 110 can also display the desensitized data to the user through the interface.
[0049] It should be noted that the data anonymization processing method and apparatus provided in this disclosure embodiment can be executed by node device 121. Accordingly, the data anonymization processing method and apparatus provided in this disclosure embodiment can be disposed in node device 121. The data anonymization processing method and apparatus provided in this disclosure embodiment can also be executed by electronic devices or electronic device clusters that are different from node device 121 and are capable of communicating with service terminal 110 and / or node device 121. Accordingly, the data anonymization processing method and apparatus provided in this disclosure embodiment can also be disposed in electronic devices or electronic device clusters that are different from node device 121 and are capable of communicating with service terminal 110 and / or node device 121.
[0050] It should be understood that Figure 1 The number of service terminals and node devices shown is merely illustrative. Depending on the implementation requirements, there can be any number of terminal devices, networks, and servers.
[0051] It should be noted that, in this embodiment of the disclosure, there are no restrictions on the data that has undergone anonymization processing. It can be any of the following: text data, image data, voice data, and video data. There are also no restrictions on the scenario or field to which the data belongs. For example, it can be a customer service consultation scenario, a product after-sales scenario, an intelligent question-and-answer scenario, or a vehicle networking scenario. In addition, there are no restrictions on the data format. For example, it supports data formats such as JPG, JPEG, PNG, and MP4, which improves the applicability of business scenarios.
[0052] Figure 2 A flowchart illustrating a data anonymization method provided in this disclosure. This method is applied to node devices, as shown below. Figure 2 The method includes:
[0053] Step S210: Obtain the data processing request sent by the business end. The data processing request includes script execution configuration parameters and the data to be processed or the data storage path to be processed. The script execution configuration parameters are used to indicate the values of variable parameters in the target script. The target script is used to implement data desensitization processing.
[0054] In this embodiment of the disclosure, the target script can be understood as a program template for data anonymization processing, and the values of the variable parameters in the target script can be dynamically modified. The variable parameters can represent parameters used to indicate the anonymization processing, such as the method for detecting the target area, the file format for saving the anonymization processing results, etc. In addition, in order to ensure the correct operation of the target script, the variable parameters can be preset with default values. When the target script does not need to be modified based on business requirements, the anonymization processing can be performed based on the default values. When the values of certain variable parameters need to be adjusted based on business requirements, they can be modified according to the obtained values.
[0055] The business side can obtain script execution configuration parameters and supports multiple ways to call the script. The script execution configuration parameters can represent configuration information related to variable parameters in the target script, and the script calling method can be related to the application scenario or requirements of the business side.
[0056] For example, in one possible implementation, the user can input the required script execution configuration parameters through an interface provided by the business side, which supports user-defined configurations and offers greater flexibility. Alternatively, the business side can automatically determine the script execution configuration parameters based on business needs, resulting in higher efficiency.
[0057] In one possible embodiment, the data processing request is triggered by the business side in response to a target script invocation instruction, wherein the target script invocation instruction is associated with the script invocation method.
[0058] For example, the business side can combine actual application scenarios and adopt various script calling methods such as Python module import, command line call, installable package, application programming interface (API) service, and graphical user interface (GUI) application. For example, a command line representing the target script call can be entered, and executing the command line will generate the target script call instruction, and then the business side can send data processing requests.
[0059] Furthermore, data processing requests can be stored in a message queue, and node devices can obtain data processing requests through the message queue. In one possible embodiment, obtaining a data processing request sent by the business terminal includes: obtaining a data processing request sent by the business terminal from the message queue, wherein the message queue stores data processing requests sent by the business terminal, and the node device used to execute the data desensitization processing method is determined based on the current processing resources of multiple node devices in the node system.
[0060] In this embodiment of the disclosure, there may be multiple data processing requests. Multiple data processing requests are maintained through a message queue. Based on the current processing resources of multiple node devices in the node system and the resource requirements corresponding to the data anonymization in the data processing request, the node device that meets the data processing request can be determined from multiple node devices. Then, the determined node device performs the data anonymization processing. This can achieve decoupling and load balancing. Furthermore, the node device obtains and processes the data processing request from the message queue, which can reduce the situation of instantaneous high load and improve the processing efficiency of the data processing request.
[0061] In one possible embodiment, this disclosure also supports dynamic adjustment of node devices in the node system, which can increase or decrease node devices, expand or upgrade the performance of the node system, and improve flexibility and scenario requirements.
[0062] Step S220: Based on the script execution configuration parameters, execute the target script to perform de-identification processing on the data to be processed and obtain the de-identification processing result.
[0063] For example, by assigning script execution configuration parameters to the corresponding variable parameters in the target script, a personalized and more targeted target script can be obtained. Running the target script can then perform data anonymization processing on the data to be processed.
[0064] Step S230: Send the de-identification processing result to the business end or store it in the target storage location.
[0065] For example, the results of the de-identification process can include the de-identified data, the de-identification progress, etc. A unique access identifier can be generated for the de-identification results to facilitate subsequent viewing and downloading.
[0066] In this embodiment of the disclosure, after the node device obtains the de-identification processing result, it can return it to the business end, which can then display it. Alternatively, it can store it in the target storage location and send it to the business end when the business end requests it. This embodiment of the disclosure does not impose any restrictions on this.
[0067] In this embodiment of the disclosure, the data anonymization processing capability can be encapsulated into a target script. Then, the target script can be called through specific script execution configuration parameters to achieve the anonymization processing of the data to be processed. In this way, the target script can run independently, is plug-and-play, improves portability and reusability, can be loosely coupled with specific business systems, and is also easy to upgrade or update, with lower maintenance costs, meets cross-language and cross-platform functions, is more convenient to use, and has higher applicability.
[0068] The data desensitization processing method of the present disclosure embodiments will be described in detail below.
[0069] In one possible embodiment, regarding the script execution configuration parameters in step S220 above, the data to be processed is de-identified by executing the target script to obtain the de-identification result, including:
[0070] S221: Obtain the data to be processed according to the data processing request.
[0071] For example, when a data processing request includes data to be processed, the data to be processed can be obtained directly from the data processing request.
[0072] For example, when a data processing request includes a data storage path to be processed, the data to be processed can be retrieved from the corresponding storage location based on that data storage path.
[0073] S222: If the size of the data to be processed meets the splitting condition, split the data to be processed into multiple sub-data.
[0074] In this embodiment of the disclosure, the splitting condition can be that the data size exceeds a threshold, and the threshold can be preset, for example, determined based on the parallel processing capability and computing resources of the node device. This embodiment of the disclosure does not impose any restrictions on this.
[0075] S223: Based on the script execution configuration parameters, the target script is executed to perform de-identification processing on multiple sub-data in parallel, and multiple sub-de-identification processing results are obtained.
[0076] In this embodiment of the disclosure, a parallel processing architecture can be adopted, and a multi-threaded approach can be used to perform desensitization processing on each sub-data in parallel, which can improve efficiency and reduce processing time.
[0077] S224: Merge the results of multiple sub-desensitization processes to obtain the desensitization result of the data to be processed.
[0078] For example, if the data to be processed is video, the video can be split into multiple video segments, and each video segment can be de-identified in parallel by multiple threads in the node device. Then, the segments are merged to obtain the final de-identified video.
[0079] In this embodiment of the disclosure, by splitting the data to be processed, processing each data in parallel by executing a target script, and then merging the data, multiple sub-data can be processed efficiently, and resources can be fully utilized to reduce processing time.
[0080] In one possible embodiment, for example, if the data to be processed is video, for large amounts of data to be processed, streaming processing technology can be used to analyze the video frame by frame to avoid loading the entire video into memory, reduce excessive memory and computing resource consumption, and improve processing efficiency. In addition, in this embodiment, hardware acceleration and adaptive bitrate adjustment for video can also be supported. For example, video can be decoded and encoded in real time using preset tools to reduce frequent read and write storage, which can further improve processing efficiency and real-time performance and reduce latency.
[0081] This disclosure provides a specific method for desensitization processing, and also provides possible implementation methods. In one possible implementation, a target script is executed to schedule a deep learning model to detect sensitive information in the data to be processed, and the detected sensitive information is desensitized based on the Gaussian blurring method.
[0082] The deep learning model can be pre-trained, and the network structure of the deep learning model is not limited in this embodiment. The Gaussian blurring method primarily achieves desensitization by blurring the information.
[0083] For example, if the data to be processed is an image or video, a deep learning model can be used to detect target objects in the image or video frame. The target objects are pre-defined objects that need to be desensitized, such as faces, license plates, card numbers, ID cards, etc. Then, based on the Gaussian blur method, the pixels in the detected target object area are subjected to weighted average blur processing to achieve desensitization.
[0084] Thus, in this embodiment of the disclosure, the detection of sensitive information based on a deep learning model is more accurate and can adapt to different lighting, role and occlusion conditions. Furthermore, the blurring based on the Gaussian blur method can effectively eliminate sensitive information and improve the efficiency and accuracy of the desensitization process.
[0085] Of course, other methods can also be used for sensitive information detection and blurring in this embodiment, and other methods can also be used for desensitization. This embodiment does not limit these methods.
[0086] In some possible embodiments, after sending the de-identification processing result to the business end or storing it in the target storage location, the method further includes: adding a completed de-identification identifier to the data to be processed; and deleting the data to be processed in response to triggering a data deletion task.
[0087] For example, a scheduled task can be set up to trigger the data deletion task after the data to be processed has been stored for a certain period of time. This allows the data to be deleted. In this way, after the data has been anonymized, it is automatically marked as having completed the anonymization process and then deleted after a certain period of storage. This can reduce storage costs and privacy risks and ensure compliance.
[0088] In some possible embodiments, the progress of the de-identification process can be recorded in a database, and the completion status of the de-identification process can be confirmed. An API interface can be provided for the business side to query. Specifically, the embodiments of this disclosure provide possible implementation methods. When a de-identification process progress query request is received from the business side, the current completion status of the de-identification process corresponding to the data to be processed is determined, and the current completion status of the de-identification process is returned to the business side so that the business side can display the current completion status of the de-identification process on the interface.
[0089] The current status of the desensitization process can be displayed in various ways, such as text, progress bar, or percentage. This embodiment does not impose any restrictions on these methods.
[0090] For example, the business side provides a progress query function option through the interface. Users can trigger the progress query function option by clicking, and then the business side can obtain the current de-identification processing status from the node device and display it through the interface.
[0091] In one possible embodiment, the business terminal disclosed herein can also support functions such as viewing and downloading historical records and anonymized data. Users can access or determine which anonymized data to download and retrieve by using the data anonymization task identifier, thereby improving the user experience. For example, users can trigger the download function of anonymized data through the interface, call the storage API based on the web interface, obtain the anonymized data, and display it. For example, for video data, a thumbnail of the anonymized video can be rendered or the streaming media can be played.
[0092] The data desensitization processing method in the embodiments of this disclosure will be described below in conjunction with specific application scenarios.
[0093] Figure 3 A flowchart of a data desensitization processing method provided in this disclosure embodiment is shown below. Figure 3 As shown, the data desensitization processing method in this embodiment includes a business terminal and node devices.
[0094] Part 1: Business Side.
[0095] 1) The business side can obtain custom script execution configuration parameters and data to be processed.
[0096] For example, script execution configuration parameters may include: whether to use a Graphics Processing Unit (GPU) for accelerated processing; the confidence threshold for the deep learning model's judgment of sensitive information; whether to run the deep learning model for detection (e.g., when the parameter representation is true, the deep learning model can be started to detect the data to be processed); whether to store the detection results of the deep learning model as a JavaScript Object Notation (JSON) file, for example, when the data to be processed is video, the detection results of each frame of the video data (e.g., the bounding box coordinates, confidence, and category identifier of the target object) can be serialized and saved to a JSON file; and parameters indicating blurring processing, such as the intensity value of blurring processing, which can be an odd number, and the larger the value, the stronger the blurring effect. Of course, in this embodiment of the disclosure, there are no restrictions on the script execution configuration parameters, and for script execution configuration parameters not transmitted by the business side, the default values of the variable parameters in the target script can also be used when executing the target script.
[0097] 2) The business side can support multiple script calling methods, and based on the supported script calling methods, obtain the target script calling instruction, and then respond to the target script calling instruction to trigger data processing requests.
[0098] In this embodiment of the disclosure, a configuration file can be generated based on the data to be processed and the script execution configuration parameters, and the target script call instruction can be obtained through system commands or other calling methods.
[0099] In this embodiment of the disclosure, the business side calls the target script through the target script call instruction to perform de-identification processing, which is more flexible and supports asynchronous processing to reduce the blocking of the main business logic in the business side.
[0100] Part Two: Node Devices
[0101] In this embodiment of the disclosure, the node device can obtain data processing requests from the message queue based on load balancing, thereby performing de-identification processing. For example, a deep learning model can be used to detect target information in the data to be processed, and the detected target information can be blurred using a Gaussian blurring method to achieve de-identification processing. The de-identification processing result is then stored in the target storage location.
[0102] Furthermore, in this embodiment, the de-identification processing progress can be updated to facilitate querying and downloading by the business side. It can also delete the data to be processed that has completed de-identification processing by triggering a data deletion task at regular intervals, thereby reducing the occupation of storage resources and improving security.
[0103] In some possible implementations, large amounts of data to be processed can be split into smaller parts. For example, for large video files, streaming technology can be used to desensitize each video frame separately, improving processing efficiency and reducing problems such as insufficient memory that may result from loading large files into memory.
[0104] In some possible embodiments, simultaneous transmission and parallel processing of multiple files can also be supported. For example, the video can be split into multiple video segments, and the multiple video segments can be de-identified in parallel through multi-threading, and then merged to obtain the de-identified result of the video, which can reduce processing time and improve efficiency.
[0105] Thus, in this embodiment of the disclosure, the node device can split and process large amounts of data to be processed, and can also optimize the parallel processing of multiple files, thereby supporting the batch de-identification processing of large amounts of data to be processed and multiple files. In addition, more efficient hardware can be used to accelerate the processing, thereby improving the processing performance of the node device, increasing the processing throughput and reducing latency.
[0106] In this embodiment of the disclosure, the business side configures the script execution configuration parameters and the data to be processed, so that the node device can perform de-identification processing on the data to be processed by executing the target script based on the script execution configuration parameters. The target script can call relevant deep learning models, Gaussian blurring methods, etc. to achieve de-identification processing of the data to be processed, realizing the automation of the de-identification process. Moreover, the de-identification processing capability can be encapsulated as a script for independent operation, making the application more flexible and improving portability and reusability.
[0107] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.
[0108] Figure 4 This is a block diagram of a data desensitization processing apparatus provided in an embodiment of the present disclosure.
[0109] Reference Figure 4 This disclosure provides a data desensitization processing apparatus, which includes:
[0110] The acquisition module 41 is used to acquire the data processing request sent by the business terminal. The data processing request includes script execution configuration parameters and data to be processed or data storage path to be processed. The script execution configuration parameters are used to indicate the values of variable parameters in the target script. The target script is used to implement data desensitization processing.
[0111] Processing module 42 is used to execute the target script based on the script execution configuration parameters to perform de-identification processing on the data to be processed and obtain the de-identification processing result.
[0112] The sending module 43 is used to send the de-identification processing result to the business terminal or store it in the target storage location.
[0113] In one possible embodiment, when the processing module 42 performs de-identification processing on the data to be processed by executing the target script based on the script execution configuration parameters to obtain the de-identification processing result, the processing module 42 is used to:
[0114] According to the data processing request, obtain the data to be processed;
[0115] If the size of the data to be processed meets the splitting condition, the data to be processed is split into multiple sub-data.
[0116] Based on the script execution configuration parameters, the target script is executed to perform desensitization processing on multiple sub-data in parallel, and multiple sub-desensitization processing results are obtained.
[0117] The multiple sub-desensitization processing results are merged to obtain the desensitization processing result of the data to be processed.
[0118] In one possible embodiment, when obtaining the data processing request sent by the service terminal, the acquisition module 41 is used to:
[0119] The data processing request sent by the business terminal is obtained from the message queue, wherein the message queue stores the data processing request sent by the business terminal, and the node device used to execute the data desensitization processing method is determined based on the current processing resources of multiple node devices in the node system.
[0120] In one possible embodiment, the data processing request is triggered by the business terminal in response to a target script invocation instruction, wherein the target script invocation instruction is associated with a script invocation method.
[0121] In one possible embodiment, after sending the de-identification processing result to the business terminal or storing it in the target storage location, the method further includes a deletion module 44, used for:
[0122] Add the de-identification label information to the data to be processed;
[0123] In response to triggering a data deletion task, the data to be processed is deleted.
[0124] In one possible embodiment, it further includes an update module 45, configured to:
[0125] When a de-identification processing progress query request is received from the business terminal, the current de-identification processing completion status corresponding to the data to be processed is determined, and the current de-identification processing completion status is returned to the business terminal so that the business terminal can display the current de-identification processing completion status on the interface.
[0126] The apparatus provided in this disclosure can have functions or include modules that can be used to execute the methods described in the above embodiments. The specific implementation and technical effects can be referred to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.
[0127] Each module in the aforementioned data anonymization processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0128] Reference Figure 5 This disclosure provides an electronic device, which includes:
[0129] One or more processors 501;
[0130] The memory 502 stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement any of the above-mentioned data desensitization processing methods.
[0131] One or more I / O interfaces 503 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.
[0132] Among them, processor 501 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 502 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 503 is connected between processor 501 and memory 502, and can realize information interaction between processor 501 and memory 502, including but not limited to data bus (Bus).
[0133] In some embodiments, the processor 501, memory 502, and I / O interface 503 are interconnected via a bus, and thus connected to other components of the computing device.
[0134] This embodiment also provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the data desensitization processing method provided in this embodiment. To avoid repetition, the specific steps of the data desensitization processing method will not be repeated here.
[0135] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device performs the above-described data desensitization processing method.
[0136] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses in the methods, systems, and apparatuses described above can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.
[0137] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0138] Those skilled in the art will understand that although some embodiments described herein include certain features that are included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this embodiment and form different embodiments.
[0139] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.
Claims
1. A data anonymization processing method, characterized in that, Node devices used in node systems include: Obtain a data processing request sent by the business side. The data processing request includes script execution configuration parameters and data to be processed or data storage path to be processed. The script execution configuration parameters are used to indicate the values of variable parameters in the target script. The target script is used to implement data desensitization processing. Based on the script execution configuration parameters, the target script is executed to perform de-identification processing on the data to be processed, and the de-identification processing result is obtained. The de-identification processing result is sent to the business terminal or stored in the target storage location.
2. The method according to claim 1, characterized in that, The process of de-identifying the data to be processed by executing the target script based on the script execution configuration parameters to obtain the de-identification result includes: According to the data processing request, obtain the data to be processed; If the size of the data to be processed meets the splitting condition, the data to be processed is split into multiple sub-data. Based on the script execution configuration parameters, the target script is executed to perform desensitization processing on multiple sub-data in parallel, and multiple sub-desensitization processing results are obtained. The multiple sub-desensitization processing results are merged to obtain the desensitization processing result of the data to be processed.
3. The method according to claim 1, characterized in that, The process of obtaining the data processing request sent by the service terminal includes: The data processing request sent by the business terminal is obtained from the message queue, wherein the message queue stores the data processing request sent by the business terminal, and the node device used to execute the data desensitization processing method is determined based on the current processing resources of multiple node devices in the node system.
4. The method according to any one of claims 1-3, characterized in that, The data processing request is triggered by the business terminal in response to the target script call instruction, wherein the target script call instruction is associated with the script call method.
5. The method according to claim 1, characterized in that, After sending the de-identification processing result to the business terminal or storing it in the target storage location, the method further includes: Add the de-identification label information to the data to be processed; In response to triggering a data deletion task, the data to be processed is deleted.
6. The method according to claim 1, characterized in that, The method further includes: When a de-identification processing progress query request is received from the business terminal, the current de-identification processing completion status corresponding to the data to be processed is determined, and the current de-identification processing completion status is returned to the business terminal so that the business terminal can display the current de-identification processing completion status on the interface.
7. A data desensitization processing device, characterized in that, include: The acquisition module is used to acquire data processing requests sent by the business end. The data processing requests include script execution configuration parameters and data to be processed or data storage path to be processed. The script execution configuration parameters are used to indicate the values of variable parameters in the target script. The target script is used to implement data desensitization processing. The processing module is used to execute the script based on the configuration parameters, and perform de-identification processing on the data to be processed by executing the target script to obtain the de-identification processing result; The sending module is used to send the de-identification processing result to the business terminal or store it in the target storage location.
8. An electronic device, characterized in that, include: One or more processors; A memory having stored one or more programs thereon, which, when executed by one or more processors, cause the one or more processors to implement the data desensitization processing method according to any one of claims 1-6; One or more I / O interfaces are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data desensitization processing method as described in any one of claims 1-6.
10. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the data desensitization processing method as described in any one of claims 1-6.