Real-time data desensitization management and data distribution system and method
By using a real-time data desensitization management and data distribution system, and leveraging shadow desensitization and data distribution components, the system solves the problem of increased costs and timelines associated with rule-based coding in existing technologies, and achieves efficient data security management and configurable data security in real-time computing scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHERY AUTOMOBILE CO LTD
- Filing Date
- 2022-12-27
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies use rule coding based on different business needs, which increases implementation costs and time, reduces R&D efficiency, and fails to meet the data security and configurability requirements of real-time computing scenarios.
A real-time data masking management and data distribution system is adopted, including a message bus, shadow channel, shadow masking component, data distribution component, and data security gateway. The shadow masking component pushes the masked data in the message bus to the shadow channel according to the masking configuration, and the data distribution component distributes the masked data to the corresponding data application parties.
It reduced implementation costs and time, improved R&D efficiency, and effectively met the data security and configurability requirements in real-time computing scenarios.
Smart Images
Figure CN116248353B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data security management technology, and in particular to a real-time data de-identification management and data distribution system and method. Background Technology
[0002] In related technologies, real-time data processing systems encode rules based on different business needs to meet the requirements of different business scenarios. Data security definitions are stored in the database to meet offline business security requirements. For online business applications, security rules are obtained through periodic polling of online business data. Each change to the security rules requires a new round of requirements, development, testing, and release.
[0003] However, the rule coding based on different business needs in related technologies increases implementation costs and time, reduces R&D efficiency, and fails to meet the data security and configurability requirements in real-time computing scenarios, which urgently needs to be addressed. Summary of the Invention
[0004] This application provides a real-time data anonymization management and data distribution system and method to solve the problems in related technologies, such as rule coding based on different business needs, which increases implementation costs and implementation cycle, reduces R&D efficiency, and fails to meet the data security and configurability requirements in real-time computing scenarios.
[0005] The first aspect of this application provides a real-time data anonymization management and data distribution system, including: a message bus and a shadow channel; a shadow anonymization component, used to anonymize data in the message bus and push it to the shadow channel according to the anonymization configuration; and a data distribution component, used to distribute the anonymized data in the shadow channel to the corresponding data application party.
[0006] Optionally, in one embodiment of this application, the real-time data de-identification management and data distribution system further includes: a data de-identification configuration component, used to generate the de-identification configuration.
[0007] Optionally, in one embodiment of this application, the real-time data anonymization management and data distribution system further includes: a data security gateway, which communicates with the data distribution component and the corresponding data application party respectively.
[0008] Optionally, in one embodiment of this application, the real-time data de-identification management and data distribution system further includes: a data security authorization component, used to generate configuration requirements so that the data distribution component can send the de-identified data through the data security gateway under the configuration requirements.
[0009] A second aspect of this application provides a real-time data anonymization management and data distribution method, employing the real-time data anonymization management and data distribution system described above. The method includes the following steps: anonymizing data in the message bus according to the anonymization configuration to obtain anonymized data; pushing the anonymized data to the shadow channel; and distributing the anonymized data in the shadow channel to the corresponding data application.
[0010] Optionally, in one embodiment of this application, before desensitizing the data in the message bus according to the desensitization configuration, the method further includes: generating the desensitization configuration.
[0011] Optionally, in one embodiment of this application, before distributing the de-identified data in the shadow channel to the corresponding data application party, the method further includes: generating configuration requirements to send the de-identified data under the configuration requirements.
[0012] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the real-time data de-identification management and data distribution method as described in the above embodiments.
[0013] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described real-time data desensitization management and data distribution method.
[0014] This application embodiment can push data in the message bus to the shadow channel after it has been anonymized according to the anonymization configuration through the shadow anonymization component. The anonymized data in the shadow channel is then distributed to the corresponding data application party through the data distribution component. This reduces implementation costs and implementation cycle, improves R&D efficiency, and effectively meets the data security and configurability requirements in real-time computing scenarios. Thus, it solves the problems in related technologies, such as rule encoding based on different business needs, which increases implementation costs and implementation cycle, reduces R&D efficiency, and fails to meet the data security and configurability requirements in real-time computing scenarios.
[0015] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0016] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0017] Figure 1 This is a schematic diagram of the structure of a real-time data anonymization management and data distribution system according to an embodiment of this application;
[0018] Figure 2 This is a schematic diagram of the structure of a real-time data anonymization management and data distribution system according to a specific embodiment of this application;
[0019] Figure 3 This is a schematic diagram of the shadow desensitization structure according to a specific embodiment of this application;
[0020] Figure 4 This is a schematic diagram of the structure of a data security gateway according to a specific embodiment of this application;
[0021] Figure 5 This is a schematic diagram of the structure of a data distribution task according to a specific embodiment of this application;
[0022] Figure 6 This is a flowchart illustrating a real-time data anonymization management and data distribution method according to an embodiment of this application;
[0023] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0024] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0025] The following is a schematic diagram of the structure of a real-time data anonymization management and data distribution system proposed according to an embodiment of this application, with reference to the accompanying drawings. Figure 1 This is a schematic diagram of the structure of the real-time data anonymization management and data distribution system according to an embodiment of this application.
[0026] like Figure 1 As shown, the real-time data anonymization management and data distribution system 10 includes: a message bus 100, a shadow channel 200, a shadow anonymization component 300, and a data distribution component 400.
[0027] Specifically, message bus 100 and shadow channel 200.
[0028] In actual implementation, this application embodiment sets up a message bus 100 and a shadow channel 200. The message bus 100 can be a distributed message subscription system that can store the original message data and the desensitized shadow message data. The shadow channel 200 can be an automatically sensing message channel to improve the executability of data desensitization.
[0029] The shadow desensitization component 300 is used to push the desensitized data in the message bus 100 to the shadow channel 200 according to the desensitization configuration.
[0030] As one possible approach, the shadow desensitization component 300 can push the desensitized data in the message bus 100 to the shadow channel 200 according to the desensitization configuration in the following steps. Furthermore, the shadow desensitization component 300 and the shadow channel 200 can automatically sense the message channel and desensitization rule definition, and can complete data desensitization and storage. They can automatically create data desensitization tasks according to the upstream message channel to complete the data desensitization work, thereby effectively reducing the implementation cost of data processing and improving the efficiency of data desensitization work.
[0031] For example, such as Figure 2 As shown, the shadow desensitization component 300 can automatically detect messages in the message channel. If there is a message in the message channel [msg_channel01], the shadow desensitization component 300 can automatically create a data desensitization task [task_channel01]. The data security officer can configure security desensitization rules through the following steps. The data desensitization task completes the data desensitization in [msg_channel01] according to the security desensitization rules and pushes the data to the shadow channel 200 [sec_msg_channel01]. The data in the shadow channel 200 consists of two parts: the original message and the desensitized message.
[0032] The data distribution component 400 is used to distribute the de-identified data in the shadow channel 200 to the corresponding data application party.
[0033] In some embodiments, the data distribution component 400 can distribute the de-identified data in the shadow channel 200 to the corresponding data application parties based on the pull concept. Specifically, the data distribution component 400 can distribute the data distribution task to different data application parties through the data security gateway 600 in the following steps, following the configuration requirements of the data security authorization component 700 in the following steps, thereby effectively improving the applicability and reliability of the system.
[0034] For example, such as Figure 3 As shown, data subscribers can subscribe to data [msg_channel01] through the data distribution platform and register a push channel. The data distribution service requests the data gateway service, which will be described in detail in the following steps, and starts a data consumption task [task_gateway_channel01]. The data consumption task consumes the data in the [sec_msg_channel01] channel and completes secure encapsulation. The encapsulated data is then distributed and pushed through the channel registered by the subscriber in the data distribution component 400.
[0035] Optionally, in one embodiment of this application, the real-time data desensitization management and data distribution system 10 further includes: a data desensitization configuration component 500.
[0036] Among them, the data desensitization configuration component 500 is used to generate desensitization configurations.
[0037] In some embodiments, such as Figure 4 As shown, this application embodiment can set a data desensitization configuration component 500, which can generate desensitization configurations. Data security personnel can use the data desensitization configuration component 500 to define desensitization rules for sensitive attributes and distribute desensitization events to the message middleware, thereby effectively reducing implementation costs and implementation cycle and improving the real-time performance of data desensitization management.
[0038] Optionally, in one embodiment of this application, the real-time data desensitization management and data distribution system 10 further includes a data security gateway 600.
[0039] The data security gateway 600 communicates with the data distribution component 400 and the corresponding data application party.
[0040] In actual implementation, this application embodiment can set up a data security gateway 600 to communicate with the data distribution component 400 and the corresponding data application party. The data security gateway 600 can redefine the distributed data according to the security requirements based on the data security level and data anonymization rules of the data consumer user, and finally distribute it to the corresponding data application party, thereby effectively meeting the data security configurability requirements in real-time computing scenarios.
[0041] For example, such as Figure 5 As shown, the data security gateway 600 can be composed of three parts: data consumption, data security encapsulation, and gateway service. It also works in conjunction with the data security authorization component 700 in the following steps to complete data security control. External applications obtain real-time data through the data security gateway 600. The data security gateway 600 automatically creates a consumption task [task_gateway_channl01] according to the request. [task_gateway_channl01] consumes [sec_msg_channel01]. Since the channel contains original message data and encrypted data, the consumption task completes the real-time encapsulation of the de-identified fields according to the data security authorization and pushes the data to the gateway service.
[0042] Optionally, in one embodiment of this application, the real-time data desensitization management and data distribution system 10 further includes a data security authorization component 700.
[0043] Among them, the data security authorization component 700 is used to generate configuration requirements so that the data distribution component 400 can send de-identified data through the data security gateway 600 under the configuration requirements.
[0044] In some embodiments, the present application may set a data security authorization component 700, which can generate configuration requirements so that the data distribution component 400 can send de-identified data through the data security gateway 600 under the configuration requirements. The data security authorization component 700 can also allocate different user access permissions according to the user's data security level, mainly defining whether the data is de-identified, thereby improving the real-time performance and reliability of data de-identification management and meeting the data security configurability requirements in real-time computing scenarios.
[0045] For example, the working principle of the embodiments of this application will be described in detail below with a specific example.
[0046] In this application embodiment, the data governance team can set de-identification rules for message data based on security requirements. In this application embodiment, data de-identification can be completed according to the de-identification rules.
[0047] First, the data governance team completes the allocation of data security permissions based on the data access security level of data consumers. Then, the data computing module in this embodiment can automatically sense the message channel based on Flink technology and automatically create the corresponding de-identification pipeline, thereby completing the de-identification of the original message data according to the de-identification rules.
[0048] Secondly, the shadow desensitization component can repackage the desensitized data and the original message. The packaged message contains the original message part rawData and the desensitized message part secData. The packaged message is pushed to the shadow channel with the prefix sec_*.
[0049] Furthermore, the message bus in this embodiment can store the original message data and the de-identified shadow message data.
[0050] Finally, in this embodiment of the application, the data security gateway is an access proxy layer for data distribution and message bus. The distribution system can access the message middleware through the consumer proxy layer. The message proxy layer encapsulates the obtained messages into data that can be accessed by the visitor's security level according to the data security authorization rules. In other words, the data security gateway can complete the data splicing based on shadow messages according to the data security of the data consumer and provide the data that complies with the security rules to the consumer.
[0051] In summary, the embodiments of this application can realize real-time data desensitization management and data allocation, effectively meeting the data security and configurability requirements in real-time computing scenarios.
[0052] The real-time data desensitization management and data distribution system proposed in this application can push desensitized data in the message bus to the shadow channel according to the desensitization configuration through the shadow desensitization component, and distribute the desensitized data in the shadow channel to the corresponding data application party through the data distribution component. This reduces implementation costs and implementation cycle, improves R&D efficiency, and effectively meets the data security and configurability requirements in real-time computing scenarios. Thus, it solves the problems in related technologies, such as rule coding based on different business needs, which increases implementation costs and implementation cycle, reduces R&D efficiency, and fails to meet the data security and configurability requirements in real-time computing scenarios.
[0053] in, Figure 6 This is a flowchart illustrating a real-time data anonymization management and data distribution method provided in an embodiment of this application.
[0054] like Figure 6 As shown, this real-time data anonymization management and data distribution method includes the following steps:
[0055] In step S601, the data in the message bus is de-identified according to the de-identification configuration to obtain de-identified data.
[0056] Optionally, in one embodiment of this application, before desensitizing the data in the message bus according to the desensitization configuration, the method further includes: generating the desensitization configuration.
[0057] In step S602, the de-identified data is pushed into the shadow channel.
[0058] In step S603, the de-identified data in the shadow channel is distributed to the corresponding data application party.
[0059] Optionally, in one embodiment of this application, before distributing the de-identified data in the shadow channel to the corresponding data application party, the method further includes: generating configuration requirements to send the de-identified data under the configuration requirements.
[0060] It should be noted that the foregoing explanation of the real-time data desensitization management and data distribution system embodiment also applies to the real-time data desensitization management and data distribution method of this embodiment, and will not be repeated here.
[0061] The real-time data desensitization management and data distribution method proposed in this application can push desensitized data in the message bus to the shadow channel through the shadow desensitization component according to the desensitization configuration. The desensitized data in the shadow channel is then distributed to the corresponding data application party through the data distribution component. This reduces implementation costs and implementation cycle, improves R&D efficiency, and effectively meets the data security and configurability requirements in real-time computing scenarios. Thus, it solves the problems in related technologies, such as rule coding based on different business needs, which increases implementation costs and implementation cycle, reduces R&D efficiency, and fails to meet the data security and configurability requirements in real-time computing scenarios.
[0062] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:
[0063] The memory 701, the processor 702, and the computer program stored on the memory 701 and executable on the processor 702.
[0064] When the processor 702 executes the program, it implements the real-time data desensitization management and data distribution method provided in the above embodiments.
[0065] Furthermore, electronic devices also include:
[0066] Communication interface 703 is used for communication between memory 701 and processor 702.
[0067] The memory 701 is used to store computer programs that can run on the processor 702.
[0068] The memory 701 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0069] If the memory 701, processor 702, and communication interface 703 are implemented independently, then the communication interface 703, memory 701, and processor 702 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0070] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.
[0071] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.
[0072] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described real-time data desensitization management and data distribution method.
[0073] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0074] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0075] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0076] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0077] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0078] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0079] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0080] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.
Claims
1. A real-time data anonymization management and data distribution system, characterized in that, include: Message bus and shadow channel; The shadow desensitization component is used to desensitize the data in the message bus and push it to the shadow channel according to the desensitization configuration; as well as A data distribution component is used to distribute the de-identified data in the shadow channel to the corresponding data application party; Also includes: A data anonymization configuration component is used to generate the anonymization configuration; Also includes: A data security gateway, which communicates with the data distribution component and the corresponding data application party respectively; The data security gateway redefines the distributed data according to security requirements based on the data security level and data anonymization rules of the data consumer, and finally distributes it to the corresponding data application party. Also includes: A data security authorization component is used to generate configuration requirements so that the data distribution component can send the de-identified data through the data security gateway under the configuration requirements.
2. A method for real-time data anonymization management and data distribution, characterized in that, The real-time data anonymization management and data distribution system as described in claim 1, wherein the method includes the following steps: The data in the message bus is de-identified according to the de-identification configuration to obtain de-identified data; Push the de-identified data to the shadow channel; and Distribute the de-identified data in the shadow channel to the corresponding data application parties; Before desensitizing the data in the message bus according to the desensitization configuration, the method further includes: Generate the desensitization configuration; Before distributing the anonymized data in the shadow channel to the corresponding data application party, the process also includes: Generate configuration requirements to send the de-identified data under the specified configuration requirements.
3. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the memory storing the program to implement the real-time data anonymization management and data distribution method as described in claim 2.
4. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the real-time data anonymization management and data distribution method as described in claim 2.