Data transmission method and device
By deploying custom functions in multiple distributed engines, the problem of low data desensitization and adaptability is solved, and data desensitization and adaptability in different engines is improved.
Patent Information
- Application Number
- PCT/CN2024/125830
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-11-23
- Filing Date
- 2024-10-18
- Publication Date
- 2025-05-30
AI Technical Summary
In the prior art, the data desensitization method has low adaptability and cannot be applied to other types of distributed data engines other than spark.
Provides a data transmission method, by receiving query requests from the target account, selecting pre-registered custom functions to desensitize the target data set, and sending the desensitized data set to the target account, and deploying the custom functions in multiple versions of the distributed engine.
It realizes the adaptability of custom functions in multiple distributed engines, and improves the adaptability of data desensitization.
Smart Images

Figure CN2024125830_30052025_PF_FP_ABST
Abstract
Description
Data transmission method and device
[0001] This application claims priority to a Chinese patent application filed with the China Patent Office on November 23, 2023, with application number 202311578890X and application name “Data Transmission Method and Device,” the entire contents of which are incorporated herein by reference. Technical Field
[0002] The present application relates to the field of computer technology, and in particular to a data transmission method and device. Background Art
[0003] Desensitizing data before transmission is a common practice in the data security field. In related technologies, when business systems access data using Spark (a distributed computing engine), Spark is used to access desensitized data from the database. While this solution can desensitize Spark calculations, it is not applicable to other types of distributed data engines.
[0004] Summary of the Invention
[0005] The embodiments of the present application provide a data transmission method and device to at least solve the technical problem of low adaptability of data desensitization methods in related technologies.
[0006] According to one aspect of an embodiment of the present application, a data transmission method is provided, comprising: receiving a query request sent by a target account, the query request including at least a target data set that the target account requests to access; in the case where sensitive data exists in the target data set, selecting a target function from pre-registered custom functions, and performing a desensitizing operation on the target data set using the target function to obtain a desensitized target data set, wherein the custom function is deployed in multiple versions of distributed engines; and sending the desensitized target data set to the target account.
[0007] In some embodiments, a target function is selected from pre-registered custom functions, and the target function is used to perform a desensitization operation on the target data set to obtain a desensitized target data set, including: obtaining sensitive data in the target data set; selecting a target function with the same name as a target desensitization rule from the custom function, and the target desensitization rule is used to perform a desensitization operation on the sensitive data; using the target function to perform a desensitization operation on the sensitive data in the target data set to obtain the desensitized target data set.
[0008] In some embodiments, obtaining sensitive data in the target data set includes: selecting each data unit in the target data set in turn as a data unit to be identified; identifying the data unit to be identified according to a predetermined identification rule to determine whether the data unit to be identified is sensitive data, until all data units in the target data set are identified to obtain the sensitive data; marking multiple data units in the sensitive data set as multiple categories of sensitive data according to a predetermined classification standard, and each category of sensitive data corresponds to different plaintext permissions.
[0009] In some embodiments, the method further includes: after obtaining the desensitized target data set, receiving a viewing request initiated by the target account for the sensitive data; extracting the sensitive data field to be viewed from the viewing request, and when the target account passes the verification, adding a mapping relationship between the target account and the sensitive data field to be viewed in a preset mapping relationship table, wherein the mapping relationship is used to indicate that the target account has plain text permission for the sensitive data field to be viewed.
[0010] In some embodiments, the method further includes: when the target account has plaintext permission for the sensitive data field to be viewed, determining that the desensitization rule corresponding to the sensitive data field to be viewed is an empty value; when the target account does not have plaintext permission for the sensitive data field to be viewed, selecting the desensitization rule corresponding to the sensitive data field to be viewed from a preset desensitization rule mapping relationship table.
[0011] In some embodiments, the method further includes: in the event that the plaintext permissions of the target account change, determining the changed plaintext permissions of the target account as the target permissions; traversing all sensitive data in the target data set, determining the sensitive data that needs to be desensitized under the target permissions as the first target data, and determining the sensitive data that does not need to be desensitized under the target permissions as the second target data; setting the desensitization rules of the first target data to a null value, setting the desensitization rules of the second target data to a null value, and determining the desensitization operation plan for the target data set.
[0012] In some embodiments, the method further includes: packaging the pre-registered custom function into an extension package in a target format; determining reference parameters according to a version type of the distributed engine, and deploying the extension package in the target format through the reference parameters.
[0013] According to another aspect of an embodiment of the present application, a data transmission device is also provided, including: a receiving module for receiving a query request sent by a target account, the query request including at least a target data set that the target account requests to access; a desensitizing module for selecting a target function from pre-registered custom functions when sensitive data exists in the target data set, and using the target function to perform a desensitizing operation on the target data set to obtain a desensitized target data set, wherein the custom function is deployed in multiple versions of distributed engines; and a sending module for sending the desensitized target data set to the target account.
[0014] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which stores a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned data transmission method.
[0015] According to another aspect of the embodiments of the present application, a computer device is provided, including: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the above-mentioned data transmission method is executed when the program is run.
[0016] In an embodiment of the present application, a query request sent by a target account is received, and the query request includes at least a target data set that the target account requests to access; when sensitive data exists in the target data set, a target function is selected from pre-registered custom functions, and the target data set is desensitized using the target function to obtain a desensitized target data set, wherein the custom function is deployed in multiple versions of distributed engines; the desensitized target data set is sent to the target account, and the target data set is desensitized using the target function in the pre-registered custom function, thereby achieving the purpose of using custom functions to adapt to multiple types of distributed computing engines, realizing the technical effect of improving the adaptability of data desensitization, and thus solving the technical problem of low adaptability of data desensitization in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0018] FIG1 is a hardware structure block diagram of a computer terminal (or mobile device) for a data transmission method according to an embodiment of the present application;
[0019] FIG2 is a flow chart of a data transmission method according to the present application;
[0020] FIG3 is a schematic diagram of an optional sensitive data identification process according to an embodiment of the present application;
[0021] FIG4 is a schematic diagram of an optional sensitive rule acquisition flow chart according to an embodiment of the present application;
[0022] FIG5 is a schematic diagram of an optional desensitization execution plan change flow chart according to an embodiment of the present application;
[0023] FIG6 is a schematic structural diagram of an optional data transmission device according to an embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.
[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0026] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0027] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a data transmission method. As shown in Figure 1, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used in the figure to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that the structure shown in Figure 1 is only illustrative and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may also include more or fewer components than those shown in Figure 1, or have a configuration different from that shown in Figure 1.
[0028] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).
[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data transmission method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above-mentioned data transmission method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0030] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.
[0031] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).
[0032] In the above operating environment, an embodiment of the present application provides a data transmission method, as shown in FIG2 , which includes the following steps:
[0033] Step S202: receiving a query request sent by a target account, where the query request at least includes a target dataset that the target account requests to access;
[0034] Step S204: If sensitive data exists in the target dataset, a target function is selected from pre-registered custom functions, and the target dataset is desensitized using the target function to obtain a desensitized target dataset, wherein the custom function is deployed in multiple versions of distributed engines;
[0035] Step S206: Send the desensitized target data set to the target account.
[0036] Through the above steps, a query request sent by a target account can be received, where the query request includes at least a target data set that the target account requests to access; when sensitive data exists in the target data set, a target function is selected from pre-registered custom functions, and the target data set is desensitized using the target function to obtain a desensitized target data set, wherein the custom function is deployed in multiple versions of distributed engines; the desensitized target data set is sent to the target account, and the target data set is desensitized using the target function in the pre-registered custom function, thereby achieving the purpose of using custom functions to adapt to multiple types of distributed computing engines, realizing the technical effect of improving the adaptability of data desensitization, and thus solving the technical problem of low adaptability of data desensitization in related technologies.
[0037] It should be noted that the user-defined function is a UDF (User-Defined Function).
[0038] The following describes steps S202 to S206 in detail through an embodiment.
[0039] In step S204, a target function is selected from pre-registered custom functions, and the target data set is desensitized using the target function. The specific method for obtaining the desensitized target data set is as follows: obtaining the sensitive data in the target data set; selecting a target function with the same name as the target desensitization rule from the custom function, and the target desensitization rule is used to desensitize the sensitive data; and using the target function to desensitize the sensitive data in the target data set to obtain the desensitized target data set.
[0040] In an optional manner, before obtaining the sensitive data in the target data set, the sensitive data in the target data set needs to be identified first. The specific identification process is: selecting each data unit in the target data set as the data unit to be identified in turn; identifying the data unit to be identified according to a predetermined identification rule to determine whether the data unit to be identified is sensitive data, until all data units in the target data set are identified to obtain the sensitive data; marking multiple data units in the sensitive data set as multiple categories of sensitive data according to a predetermined classification standard, and each category of sensitive data corresponds to different plaintext permissions.
[0041] Figure 3 shows a sensitive data identification process. As shown in Figure 3, a sensitive data marking module is constructed to record the mapping relationship between the model's sensitive data and sensitive rules. First, the preset desensitizing identification rules and the classification and grading standards for sensitive data are loaded, and then the data in the target data set are traversed. Sensitive data is identified and marked according to the sensitive data identification rules. The predetermined identification rules at least include: identifying sensitive data using regular expressions, identifying sensitive data according to predetermined sensitive keywords, etc.
[0042] It should be noted that different plaintext permissions can display sensitive data of different levels or categories in plaintext.
[0043] After receiving a viewing request for the sensitive data initiated by the target account, the sensitive data field to be viewed is extracted from the viewing request, and if the target account passes the verification, a mapping relationship between the target account and the sensitive data field to be viewed is added to a preset mapping relationship table, where the mapping relationship is used to indicate that the target account has plain text permission for the sensitive data field to be viewed.
[0044] Figure 4 shows a sensitive rule acquisition flow chart. As shown in Figure 4, when the target account has the plaintext permission for the sensitive data field to be viewed, the desensitization rule corresponding to the sensitive data field to be viewed is determined to be an empty value; when the target account does not have the plaintext permission for the sensitive data field to be viewed, the desensitization rule corresponding to the sensitive data field to be viewed is selected from the preset desensitization rule mapping relationship table.
[0045] In some embodiments of the present application, a sensitive data authorization and authentication module can be constructed. This module includes a sensitive data access authorization submodule, which is used to authorize target accounts to access sensitive data. A target account requiring plaintext permissions initiates a request for plaintext permission to view sensitive data. If the request passes verification, the plaintext permission for the corresponding sensitive data is granted.
[0046] It's important to note that the preset mapping table records whether an account has cleartext permissions for sensitive data in a specific field of a specific table. Furthermore, the sensitive data authorization and authentication module also includes an authentication submodule, which determines whether a user has cleartext permissions for the data. If so, the desensitization rules for that data are set to null. Otherwise, the sensitive data field name and its desensitization rule mapping are returned.
[0047] In the event that the plaintext permissions of the target account are changed, the changed plaintext permissions of the target account are determined as the target permissions; all sensitive data in the target data set are traversed, and the sensitive data that needs to be desensitized under the target permissions are determined as the first target data, and the sensitive data that does not need to be desensitized under the target permissions are determined as the second target data; the desensitization rule of the first target data is set to a null value, and the desensitization rule of the second target data is set to a null value, and a desensitization operation plan for the target data set is determined.
[0048] In some embodiments, a desensitizing extension module can be constructed to extend the execution plan conversion function of the distributed computing engine. The specific desensitizing execution plan change process is shown in Figure 5. The desensitizing function of the execution plan is extended and UDF is introduced to desensitize the target data set. Iterate the existing query plan, obtain the metadata information of the target data set (including the query user, the table to which the data belongs, the field name of the query, etc.) and request the sensitive data authentication module to determine whether the corresponding data needs to be desensitized. If desensitization is required, the execution plan is rewritten according to the desensitization rules returned by the authentication module, otherwise the execution plan remains unchanged. When using the desensitizing rule to rewrite the execution plan, register the UDF function with the same name as the desensitizing rule in the big data engine, and when the execution engine starts, pre-register the UDF so that the user does not need to manually register the UDF. To translate the desensitizing rule into UDF, you only need to match the UDF with the same name as the desensitizing rule in the engine UDF registration list.
[0049] For distributed query engines that support extensions, such as Spark and Hive, this module can be used as an out-of-the-box extension package to implement data desensitization in a pluggable manner. An optional method is to package the pre-registered custom functions into an extension package in the target format; determine the reference parameters according to the version type of the distributed engine, and deploy the extension package in the target format through the reference parameters.
[0050] It is understandable that the target format extension package may be in jar format.
[0051] The data transmission method provided by this application is not restricted by the type of data source. Compared with the method of capturing data packets and parsing sql data and rewriting sql data in related technologies, data desensitization is performed dynamically without matching different types of sql data syntax. At the same time, a UDF automatic registration module is built in, which is applicable to but not limited to mainstream big data query engines such as spark, hive and flink. This application directly uses the original data without the need for additional storage space, which has a higher utilization rate of storage and is more economical and efficient. In addition, because there is no additional storage, there is no need to maintain the mapping relationship between the original data and the static desensitized data, and the system complexity is greatly reduced.
[0052] The embodiment of the present application provides a data transmission device, as shown in FIG6 , including:
[0053] A receiving module 60 is configured to receive a query request sent by a target account, wherein the query request includes at least a target dataset that the target account requests to access;
[0054] a desensitization module 62 configured to, when sensitive data exists in the target dataset, select a target function from pre-registered custom functions and perform a desensitization operation on the target dataset using the target function to obtain a desensitized target dataset, wherein the custom function is deployed in multiple versions of distributed engines;
[0055] The sending module 64 is configured to send the desensitized target data set to the target account.
[0056] The desensitization module 62 includes: a desensitization sub-module, used to obtain the sensitive data in the target data set; select a target function with the same name as the target desensitization rule from the custom function, and the target desensitization rule is used to perform a desensitization operation on the sensitive data; use the target function to perform a desensitization operation on the sensitive data in the target data set to obtain the desensitized target data set.
[0057] The desensitizing submodule includes: an acquisition unit, which is used to select each data unit in the target data set in turn as a data unit to be identified; identify the data unit to be identified according to a predetermined identification rule to determine whether the data unit to be identified is sensitive data, until all data units in the target data set are identified to obtain the sensitive data; mark multiple data units in the sensitive data set as multiple categories of sensitive data according to a predetermined classification standard, and each category of sensitive data corresponds to different plaintext permissions.
[0058] The acquisition unit includes: a receiving subunit and a determining subunit, the receiving subunit is used to receive a viewing request initiated by the target account for the sensitive data; extract the sensitive data field to be viewed from the viewing request, and when the target account passes the verification, add a mapping relationship between the target account and the sensitive data field to be viewed in a preset mapping relationship table, wherein the mapping relationship is used to indicate that the target account has the plain text permission for the sensitive data field to be viewed.
[0059] The determination subunit is used to determine that the desensitization rule corresponding to the sensitive data field to be viewed is an empty value when the target account has the plain text permission for the sensitive data field to be viewed; and to select the desensitization rule corresponding to the sensitive data field to be viewed from the preset desensitization rule mapping relationship table when the target account does not have the plain text permission for the sensitive data field to be viewed.
[0060] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, in which a program is stored. When the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned data transmission method.
[0061] According to another aspect of the embodiments of the present application, a computer device is also provided, including: receiving a query request sent by a target account, the query request including at least a target data set that the target account requests to access; in the case where sensitive data exists in the target data set, selecting a target function from pre-registered custom functions, and using the target function to perform a desensitizing operation on the target data set to obtain a desensitized target data set, wherein the custom function is deployed in multiple versions of distributed engines; and sending the desensitized target data set to the target account.
[0062] It should be noted that the various modules in the above-mentioned interface interaction device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.
[0063] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0064] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0065] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0066] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.
[0067] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0068] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0069] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A data transmission method, comprising: receiving a query request sent by a target account, wherein the query request at least includes a target data set requested to be accessed by the target account; In the case where sensitive data exists in the target data set, a target function is selected from pre-registered custom functions, and the target data set is desensitized using the target function to obtain a desensitized target data set, wherein the custom function is deployed in multiple versions of distributed engines; The desensitized target data set is sent to the target account.
2. The method according to claim 1, wherein: Selecting a target function from pre-registered custom functions, and performing a desensitization operation on the target data set using the target function to obtain a desensitized target data set, including: Acquire sensitive data in the target data set; Selecting a target function with the same name as a target desensitization rule from the custom functions, wherein the target desensitization rule is used to perform a desensitization operation on the sensitive data; The objective function is used to perform a desensitization operation on the sensitive data in the target data set to obtain the desensitized target data set.
3. The method according to claim 2, wherein: Acquire sensitive data in the target data set, including: Selecting each data unit in the target data set in turn as a data unit to be identified; Identify the data unit to be identified according to a predetermined identification rule to determine whether the data unit to be identified is sensitive data, until all data units in the target data set are identified to obtain the sensitive data; The multiple data units in the sensitive data set are marked as multiple categories of sensitive data according to a preset classification standard, and each category of sensitive data corresponds to different plaintext permissions.
4. The method according to claim 3, further comprising: After obtaining the desensitized target data set, Receiving a request for viewing the sensitive data initiated by the target account; The sensitive data field to be viewed is extracted from the viewing request, and when the target account passes the verification, a mapping relationship between the target account and the sensitive data field to be viewed is added to a preset mapping relationship table, wherein the mapping relationship is used to indicate that the target account has plain text permissions for the sensitive data field to be viewed.
5. The method according to claim 4, further comprising: In the case where the target account has the plain text permission for the sensitive data field to be viewed, determining that the desensitization rule corresponding to the sensitive data field to be viewed is a null value; When the target account does not have the plain text permission for the sensitive data field to be viewed, a desensitization rule corresponding to the sensitive data field to be viewed is selected from a preset desensitization rule mapping relationship table.
6. The method according to claim 2, further comprising: In the event that the plaintext permissions of the target account are changed, Determine the changed plaintext permission of the target account as the target permission; Traversing all sensitive data in the target data set, determining the sensitive data that needs to be desensitized under the target authority as the first target data, and determining the sensitive data that does not need to be desensitized under the target authority as the second target data; The desensitization rule of the first target data is set to a null value, the desensitization rule of the second target data is set to a null value, and a desensitization operation plan for the target data set is determined.
7. The method according to claim 1, further comprising: Packing the pre-registered custom functions into an extension package in a target format; The reference parameters are determined according to the version type of the distributed engine, and the extension package in the target format is deployed through the reference parameters.
8. The method according to claim 1, further comprising: After receiving the query request sent by the target account, load the preset desensitization identification rules and classification and grading standards of sensitive data; Traversing the data in the target data set, and identifying and marking sensitive data according to predetermined sensitive data identification rules; The predetermined sensitive data identification rules at least include: identifying sensitive data using regular expressions, or identifying sensitive data based on predetermined sensitive keywords.
9. A data transmission device, comprising: A receiving module, configured to receive a query request sent by a target account, wherein the query request at least includes a target data set requested to be accessed by the target account; A desensitization module, used for selecting a target function from pre-registered custom functions when sensitive data exists in the target data set, and performing a desensitization operation on the target data set using the target function to obtain a desensitized target data set, wherein the custom function is deployed in distributed engines of multiple versions; A sending module is used to send the desensitized target data set to the target account.
10. A non-volatile storage medium storing a program, wherein: When the program is running, the device where the non-volatile storage medium is located is controlled to execute the data transmission method according to any one of claims 1 to 8.
11. A computer device comprising: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the data transmission method according to any one of claims 1 to 8 is executed when the program is run.
Citation Information
Patent Citations
Data desensitization method and device
CN108446570A
Dynamic desensitization method and apparatus
CN108595979A
Data access method, device and system
CN113535754A
Desensitization data display method and device, intelligent wearable equipment and augmented reality glasses
CN116302275A
Data transmission method and device
CN117633878A
Cited By
Real-time data desensitization method and system based on context awareness
CN120750643A
System security and data protection mechanism method
CN120930171A