A sensitive data processing method, system, storage medium, and electronic device

Abstract

This invention relates to a method, system, storage medium, and electronic device for processing sensitive data. The method includes: acquiring network data and extracting keywords from the network data; if the keywords in the network data are sensitive data, and if the keywords are not stored in a first database, adding the keywords to the first database to obtain an updated first database; using the updated first database to perform sensitive data matching on data stored in a second database; and if a sensitive data match is successful, performing data anonymization processing on the sensitive data in the second database. This invention enables rapid processing of sensitive data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a sensitive data processing method, system, storage medium, and electronic device. Background Technology

[0002] With the rapid development of the internet, the importance of monitoring and regulating network data is gradually increasing. When network data is sensitive, it is essential to know how to quickly process sensitive data and reduce its adverse effects. Summary of the Invention

[0003] This invention provides a sensitive data processing method, system, storage medium, and electronic device that can process sensitive data quickly.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] This invention provides a sensitive data processing method, comprising:

[0006] Acquire network data and extract keywords from the network data;

[0007] If the keywords of the network data are sensitive data, and if the keywords of the network data are not stored in the first database, then the keywords of the network data are added to the first database to obtain an updated first database; wherein, the first database stores sensitive data.

[0008] The updated first database is used to perform sensitive data matching on the data stored in the second database; wherein, the second database stores user-uploaded data;

[0009] If a sensitive data match is successful, the sensitive data in the second database is desensitized.

[0010] Optional, also includes:

[0011] Retrieve the changed data stored in the second database;

[0012] Determine the data type of the changed data;

[0013] Based on the data type of the changed data, keywords are extracted from the changed data to obtain the keywords of the changed data;

[0014] Based on the data type of the changed data, the keywords of the changed data are matched with sensitive data in the first database;

[0015] If a match is found, the changed data will be anonymized.

[0016] Optionally, the step of extracting keywords from the changed data based on its data type to obtain keywords for the changed data includes:

[0017] If the changed data is text data, then the changed data is preprocessed using natural language processing methods to obtain preprocessed text, and the preprocessed text is processed using a text classification algorithm to obtain the keywords of the changed data;

[0018] If the changed data is of image type, then feature extraction is performed on the changed data to obtain data features, and the data features are processed using a text classification algorithm to obtain the keywords of the changed data.

[0019] Optionally, matching the keywords of the changed data with sensitive data in the first database according to the data type of the changed data includes:

[0020] If the changed data is text data, then keyword matching or object detection method is used to perform keyword matching on the changed data;

[0021] If the changed data is image data, then a machine vision algorithm is used to perform keyword matching on the changed data.

[0022] Optionally, after acquiring network data and extracting keywords from the network data, the method further includes:

[0023] Obtain the popularity information of the network data;

[0024] Based on the popularity information of the network data, determine whether the keywords of the network data are sensitive data;

[0025] If the keywords of the network data are sensitive data, then obtain the keyword tags of the network data based on the network data;

[0026] If the keywords of the network data are not stored in the first database, the keyword tags of the network data are stored in the first database.

[0027] Optionally, the data anonymization process for sensitive data in the second database includes:

[0028] The sensitive data in the second database is taken offline from the network, and the sensitive data in the second database is deleted from the second database.

[0029] The present invention also provides a sensitive data processing system, comprising:

[0030] The first extraction module is configured to acquire network data and extract keywords from the network data.

[0031] The keyword adding module is configured to add the keywords of the network data to the first database if the keywords of the network data are sensitive data and are not stored in the first database, thereby obtaining an updated first database; wherein, the first database stores sensitive data.

[0032] The first matching module is configured to perform sensitive data matching on the data stored in the second database using the updated first database; wherein, the second database stores user-uploaded data;

[0033] The first desensitization module is configured to perform data desensitization on sensitive data in the second database when the sensitive data is successfully matched.

[0034] Optional, also includes:

[0035] The change data acquisition module is configured to acquire change data stored in the second database;

[0036] The data type determination module is configured to determine the data type of the changed data;

[0037] The second extraction module is configured to extract keywords from the changed data based on the data type of the changed data, thereby obtaining the keywords of the changed data;

[0038] The second matching module is configured to match the keywords of the changed data with sensitive data in the first database based on the data type of the changed data.

[0039] The second desensitization module is configured to perform data desensitization processing on the changed data if a match is successful.

[0040] The present invention also provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, it implements the sensitive data processing method described above.

[0041] The present invention also provides an electronic device, comprising:

[0042] At least one processor, and at least one memory and bus connected to the processor;

[0043] The processor and the memory communicate with each other via the bus; the processor is used to call program instructions in the memory to execute the sensitive data processing method described above.

[0044] As can be seen from the above technical solutions, the present invention discloses a sensitive data processing method, system, storage medium, and electronic device. When the sensitive data in the first database storing sensitive data changes, a sensitive data matching is performed on the second database storing user-uploaded data. This allows for a re-matching of sensitive data in the data content already stored in the second database that does not contain sensitive data, thereby discovering the sensitive data contained in the second database. This enables rapid processing of the sensitive data and reduces the adverse effects caused by the sensitive data.

[0045] Of course, any product or method implementing this invention does not necessarily need to achieve all of the above advantages at the same time. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of a sensitive data processing method provided in an embodiment of the present invention;

[0048] Figure 2 A flowchart of another sensitive data processing method provided in an embodiment of the present invention;

[0049] Figure 3 A structural diagram of a sensitive data processing system provided in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] This invention provides a sensitive data processing method, such as... Figure 1 As shown, the method includes:

[0053] Step 101: Obtain network data and extract keywords from the network data.

[0054] This online data can be message data from social networking sites or news websites, and this message data can be ranking list data. The data can be text-based or image-based, and the methods for extracting keywords can differ depending on the data type. For text-based online data, natural language processing methods such as part-of-speech tagging and entity recognition can be used to preprocess the text, and then text classification algorithms such as Naive Bayes and Support Vector Machines can be used to classify the text. Classification types can include celebrities, politics, etc., to obtain the keywords from the online data. For image-based data, computer vision techniques can be used, such as convolutional neural networks to extract features from the images, and then classification algorithms such as Support Vector Machines can be used for classification.

[0055] As an optional implementation, after acquiring network data and extracting keywords from the network data, the method further includes:

[0056] Obtain information on the popularity of online data;

[0057] Based on the popularity information of online data, determine whether the keywords in the online data are sensitive data;

[0058] If the keywords in the network data are sensitive data, then obtain the keyword tags for the network data based on the network data;

[0059] If the keywords of the network data are not stored in the first database, then the keyword tags of the network data will be stored in the first database.

[0060] To determine whether keywords in online data are sensitive, one can rely on the data's popularity. If the keywords are trending search terms, they may be considered sensitive. To improve accuracy, another method is to check if the keywords are found in a sensitive database. If so, the keywords are indeed sensitive. In practice, even if the keywords are trending but not stored in a sensitive database, manual verification is still possible.

[0061] If the keywords in the online data are sensitive data, and these keywords are not stored in the first database, then tags can be added to the sensitive data. These tags can include, but are not limited to, time, location, people, events, and impact. After adding tags, these keyword tags can be stored in the first database, which is the database for storing sensitive data.

[0062] Step 102: If the keywords in the network data are sensitive data, and if the keywords in the network data are not stored in the first database, then add the keywords in the network data to the first database to obtain the updated first database; wherein, the first database stores sensitive data.

[0063] Because network data updates rapidly, in order to ensure the richness and timeliness of sensitive data in the first database and improve the processing speed of sensitive data, this invention can acquire network data in real time and extract keywords from the network data. When the keyword is not stored in the first database, the data in the first database is updated in a timely manner so as to process sensitive data quickly and reduce the adverse effects of sensitive data.

[0064] Step 103: Perform sensitive data matching on the data stored in the second database using the updated first database; wherein, the second database stores user-uploaded data.

[0065] Before storing user-uploaded data in the second database, sensitive data detection can be performed on the uploaded data to prevent it from containing sensitive information. If no sensitive data is detected or sensitive data is detected and de-identified, the user-uploaded data can be stored in the second database.

[0066] However, when the sensitive data stored in the first database changes, the data previously stored in the second database may contain sensitive data. If the sensitive data in the second database is not de-identified in a timely manner, there will still be adverse effects caused by the presence of sensitive data.

[0067] Based on this, when the sensitive data in the first database storing sensitive data changes, i.e. after the first database is updated, the present invention performs sensitive data matching on the second database storing user-uploaded data. This allows for the re-matching of sensitive data in the second database that does not contain sensitive data, thereby discovering the sensitive data contained in the second database. This enables the sensitive data to be processed quickly, reducing the adverse effects caused by the sensitive data.

[0068] When using the updated first database to perform sensitive data matching on the data stored in the second database, different matching methods can be used according to different data types.

[0069] When the data stored in the second database is text-based, such as articles, comments, and bullet comments, keyword matching or object detection methods can be used for sensitive data matching. The keyword matching method can be either a full match or a similarity match. A full match compares whether two data points are identical, while a similarity match checks if their similarity reaches a threshold. Specific keyword matching methods include DFA (Deterministic Finite Automaton), KMP (Knuth-Morris-Pratt), trie algorithms, and AC automata algorithms. The object detection method can be a deep learning-based object detection algorithm. Alternatively, sensitive data matching can also be performed using regular expressions.

[0070] When the data stored in the second database is of image-based data type, such as videos and images, machine vision algorithms can be used to perform sensitive data matching on the data stored in the second database. These machine vision algorithms can include perceptual hashing, local perceptual hashing, convolutional neural network algorithms, etc.

[0071] Step 104: If the sensitive data is successfully matched, perform data anonymization processing on the sensitive data in the second database.

[0072] Optionally, sensitive data in the second database may be anonymized, including:

[0073] Sensitive data in the second database was taken offline from the network and then deleted from the second database.

[0074] Of course, when performing data anonymization on sensitive data in the second database, the user who uploaded the sensitive data can also be notified that the sensitive data will be taken offline or has already been taken offline and deleted.

[0075] As an optional implementation, the sensitive data processing method provided by the present invention, such as... Figure 2 As shown, it also includes:

[0076] Step 201: Retrieve the changed data stored in the second database.

[0077] When a user uploads data, the data in the second database will change. At this time, it is necessary to perform sensitive data detection on the user-uploaded data. If sensitive data is found, it needs to be processed to prevent the sensitive data from causing adverse effects.

[0078] Step 202: Determine the data type of the changed data.

[0079] The data type of the changed data can be text or image.

[0080] Step 203: Extract keywords from the changed data based on its data type to obtain the keywords for the changed data.

[0081] Optionally, based on the data type of the changed data, keywords can be extracted from the changed data to obtain keywords for the changed data, including:

[0082] If the changed data is text data, then natural language processing is used to preprocess the changed data to obtain preprocessed text, and then text classification algorithm is used to process the preprocessed text to obtain the keywords of the changed data;

[0083] If the data is changed to an image type, feature extraction is performed on the changed data to obtain data features, and the data features are processed using a text classification algorithm to obtain the keywords of the changed data.

[0084] Text-type data can include user-uploaded articles, comments, bullet comments, etc. Natural language processing methods are used for preprocessing, such as text content segmentation, word segmentation, part-of-speech tagging, and entity recognition. Then, text classification algorithms, such as Naive Bayes and Support Vector Machine, are used to classify the text, thereby obtaining the keywords of the changed data.

[0085] Image data can be user-uploaded videos or pictures. Feature extraction and keyframe extraction can be performed. Visual technology or facial recognition can be used to identify the names or key information of people in the image. Then, the data features are processed using text classification algorithms, such as support vector machines, to obtain the keywords of the changed data.

[0086] Step 204: Match the keywords of the changed data with the sensitive data in the first database according to the data type of the changed data.

[0087] Optionally, based on the data type of the changed data, keywords in the changed data are matched with sensitive data in the first database, including:

[0088] If the changed data is text data, then keyword matching or object detection methods are used to perform keyword matching on the changed data;

[0089] If the changed data is image data, then a machine vision algorithm is used to perform keyword matching on the changed data.

[0090] When the changed data is text-based, such as articles, comments, or bullet comments, keyword matching or object detection methods can be used for sensitive data matching. The keyword matching method can be either a full match or a similarity match. A full match compares whether two data points are identical, while a similarity match checks if their similarity reaches a threshold. Specific keyword matching methods include DFA, KMP, trie algorithms, and AC automata algorithms. The object detection method can be a deep learning-based object detection algorithm. Alternatively, sensitive data matching can be performed using defined regular expressions.

[0091] When the changed data is of image-based data type, such as videos or images, machine vision algorithms can be used for sensitive data matching. These machine vision algorithms can include perceptual hashing, local perceptual hashing, and convolutional neural network algorithms.

[0092] When performing keyword matching, high-concurrency multi-threading technology can be used to make full use of multi-core computing resources, thereby improving the system's concurrent processing capabilities and response speed.

[0093] Step 205: If the match is successful, perform data anonymization processing on the changed data.

[0094] Optionally, data anonymization processing can be performed on the changed data, including:

[0095] The changed data was taken offline from the network and deleted from the second database.

[0096] Of course, when performing data anonymization on the changed data, the user who uploaded the sensitive data can also be notified that the sensitive data will be taken offline or has already been taken offline and deleted.

[0097] This invention also provides a sensitive data processing system, such as... Figure 3 As shown, the system includes:

[0098] The first extraction module 301 is configured to acquire network data and extract keywords from the network data.

[0099] The keyword adding module 302 is configured to add the keywords of the network data to the first database if the keywords of the network data are sensitive data and are not stored in the first database, thereby obtaining an updated first database; wherein, the first database stores sensitive data.

[0100] The first matching module 303 is configured to perform sensitive data matching on the data stored in the second database using the updated first database; wherein, the second database stores user-uploaded data.

[0101] The first desensitization processing module 304 is configured to perform data desensitization processing on sensitive data in the second database when the sensitive data is successfully matched.

[0102] The first desensitization processing module 304 is specifically configured to take the sensitive data in the second database offline and delete the sensitive data from the second database.

[0103] The sensitive data processing system provided by this invention further includes:

[0104] The change data acquisition module is configured to acquire change data stored in the second database;

[0105] The data type determination module is configured to determine the data type of the changed data.

[0106] The second extraction module is configured to extract keywords from the changed data based on the data type of the changed data, thereby obtaining the keywords of the changed data;

[0107] The second matching module is configured to match the keywords of the changed data with sensitive data in the first database based on the data type of the changed data.

[0108] The second data masking module is configured to perform data masking on the changed data if a match is found.

[0109] The second extraction module is specifically configured as follows:

[0110] If the changed data is text data, then natural language processing is used to preprocess the changed data to obtain preprocessed text, and then text classification algorithm is used to process the preprocessed text to obtain the keywords of the changed data;

[0111] If the data is changed to an image type, feature extraction is performed on the changed data to obtain data features, and the data features are processed using a text classification algorithm to obtain the keywords of the changed data.

[0112] The second matching module is specifically configured as follows:

[0113] If the changed data is text data, then keyword matching or object detection methods are used to perform keyword matching on the changed data;

[0114] If the changed data is image data, then a machine vision algorithm is used to perform keyword matching on the changed data.

[0115] The sensitive data processing system provided by this invention further includes:

[0116] The tag module is configured to: obtain the popularity information of network data; determine whether the keywords of the network data are sensitive data based on the popularity information of the network data; if the keywords of the network data are sensitive data, obtain the keyword tags of the network data based on the network data; and store the keyword tags of the network data in the first database if the keywords of the network data are not stored in the first database.

[0117] This invention provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the aforementioned sensitive data processing method.

[0118] This invention provides an electronic device, such as... Figure 4 As shown, the electronic device 40 includes at least one processor 401, at least one memory 402 connected to the processor 401, and a bus 403; wherein the processor 401 and the memory 402 communicate with each other through the bus 403; the processor 401 is used to call program instructions in the memory 402 to execute the aforementioned sensitive data processing method. The electronic device in this document may be a server, PC, PAD, mobile phone, etc.

[0119] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform a program that initializes the steps included in the aforementioned sensitive data processing method.

[0120] This application is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0121] In a typical configuration, the device includes one or more processors (CPUs), memory, and a bus. The device may also include input / output interfaces, network interfaces, etc.

[0122] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM, and memory includes at least one memory chip. Memory is an example of computer-readable media.

[0123] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0124] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0126] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0127] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing sensitive data, characterized in that, include: Real-time acquisition of network data, and extraction of keywords from the network data; Obtain the popularity information of the network data, and determine whether the keywords of the network data are sensitive data based on the popularity information of the network data; If the keywords of the network data are sensitive data, and if the keywords of the network data are not stored in the first database, then the keywords of the network data are added to the first database to obtain an updated first database; wherein, the first database stores sensitive data. The updated first database is used to perform sensitive data matching on the data stored in the second database, so as to re-perform sensitive data matching on the data content that does not contain sensitive data already stored in the second database; wherein, the second database stores user-uploaded data; If a sensitive data match is successful, the sensitive data in the second database is desensitized.

2. The sensitive data processing method according to claim 1, characterized in that, Also includes: Retrieve the changed data stored in the second database; Determine the data type of the changed data; Based on the data type of the changed data, keywords are extracted from the changed data to obtain the keywords of the changed data; Based on the data type of the changed data, the keywords of the changed data are matched with sensitive data in the first database; If a match is found, the changed data will be anonymized.

3. The sensitive data processing method according to claim 2, characterized in that, The step of extracting keywords from the changed data based on its data type to obtain keywords for the changed data includes: If the changed data is text data, then the changed data is preprocessed using natural language processing methods to obtain preprocessed text, and the preprocessed text is processed using a text classification algorithm to obtain the keywords of the changed data; If the changed data is image data, then feature extraction is performed on the changed data to obtain data features, and the data features are processed using a text classification algorithm to obtain the keywords of the changed data.

4. The sensitive data processing method according to claim 2, characterized in that, The step of matching the keywords of the changed data with sensitive data in the first database according to the data type of the changed data includes: If the changed data is text data, then keyword matching or object detection method is used to perform keyword matching on the changed data; If the changed data is image data, then a machine vision algorithm is used to perform keyword matching on the changed data.

5. The sensitive data processing method according to claim 1, characterized in that, After acquiring network data in real time and extracting keywords from the network data, the method further includes: If the keywords of the network data are sensitive data, then obtain the keyword tags of the network data based on the network data; If the keywords of the network data are not stored in the first database, the keyword tags of the network data are stored in the first database.

6. The sensitive data processing method according to claim 1, characterized in that, The data anonymization process for sensitive data in the second database includes: The sensitive data in the second database is taken offline from the network, and the sensitive data in the second database is deleted from the second database.

7. A sensitive data processing system, characterized in that, include: The first extraction module is configured to acquire network data in real time and extract keywords from the network data. The keyword addition module is configured to obtain the popularity information of the network data and determine whether the keywords of the network data are sensitive data based on the popularity information of the network data. If the keywords of the network data are sensitive data, and if the keywords of the network data are not stored in the first database, then the keywords of the network data are added to the first database to obtain an updated first database; wherein, the first database stores sensitive data. The first matching module is configured to perform sensitive data matching on the data stored in the second database using the updated first database, so as to re-perform sensitive data matching on the data content that does not contain sensitive data already stored in the second database; wherein, the second database stores user-uploaded data; The first desensitization module is configured to perform data desensitization on sensitive data in the second database when the sensitive data is successfully matched.

8. The sensitive data processing system according to claim 7, characterized in that, Also includes: The change data acquisition module is configured to acquire change data stored in the second database; The data type determination module is configured to determine the data type of the changed data; The second extraction module is configured to extract keywords from the changed data based on the data type of the changed data, thereby obtaining the keywords of the changed data; The second matching module is configured to match the keywords of the changed data with sensitive data in the first database based on the data type of the changed data. The second desensitization module is configured to perform data desensitization processing on the changed data if a match is successful.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the sensitive data processing method according to any one of claims 1-6.

10. An electronic device, characterized in that, include: At least one processor, and at least one memory and bus connected to the processor; The processor and the memory communicate with each other via the bus; The processor is used to invoke program instructions in the memory to execute the sensitive data processing method according to any one of claims 1-6.

Images