Data processing method for reducing possibility of data re-identification

Abstract

To relate to a data processing method for reducing the possibility of data re-identification.SOLUTION: A method includes the steps of: calculating a category-specific degree of frequency, for each column of a to-be-processed dataset; determining, as singular information, a category smaller than a critical value of singular information with category-specific degree of frequency determined in advance; calculating the number of pieces of singular information for each row of the to-be-processed dataset, the number of pieces of singular information being the number of data values determined to be singular information in the row; and determining a singular information processing method for each row on the basis of the number of pieces of singular information calculated for each row. The invention has an effect of eliminating or minimizing intervention of specialists, and processing singular information of data for reducing the possibility of data re-identification.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a data processing method for reducing data re-identifiability. [Background technology]

[0002] As interest in the use of pseudonymized personal information grows, the issue of the possibility of re-identification of pseudonymized information is becoming more prominent. In particular, it is necessary to process unique information that has a high possibility of identifying a specific individual, but there is little research on how to process this, and outlier detection techniques used in statistics and artificial intelligence learning are not suitable.

[0003] In pseudonymous information, peculiar information is information that is clearly distinguished from other information or is abnormally out of distribution, such as a rare surname, occupation, or high salary recipient in a specific region, or a large amount of overdue tax, and has a high possibility of identifying a specific individual. Although the risk of re-identification of pseudonymous information can be reduced simply by processing such peculiar information, the judgment of peculiar information is often subjectively detected and processed using the experience or knowledge of an expert. Since it is detected qualitatively by the judgment of an expert, problems such as human error, in which part of the peculiar information cannot be seen due to human error, may occur. In addition, in a specific field such as the medical field, where the help of an expert is essential because there is a lot of information that is difficult for the general public to understand, such as disease names and drug codes, if the help of an expert is not available, there is a high possibility that the data will be exposed to the risk of re-identification.

[0004] Common data types are roughly divided into category data and numerical data. Category data is data classified into categories, such as gender, blood type, and educational background, and is mainly composed of text, while numerical data is data composed of numbers, such as age, weight, and salary. Category data identifies anomalous information by analyzing frequencies through frequency distribution tables and histograms, while numerical data identifies minimum or maximum values ​​that are abnormally outside the distribution using outlier detection techniques in the statistical field.

[0005] However, the method of detecting outliers in numerical data is suitable for detecting outliers in general statistical or AI learning data, but is not suitable for determining idiosyncratic information in pseudonymous information where data may be concentrated at minimum or maximum values. In addition, if the above-mentioned outlier detection technique is applied directly to pseudonymous information expressed in a two-dimensional table format, data analysis may be difficult due to indiscriminate data processing during idiosyncratic information processing, such as row deletion, local deletion, and value replacement. Unlike general AI learning or statistical data, in pseudonymous information, it is necessary to first detect the presence or absence of idiosyncratic information in the entire data, and then judge it row by row to flexibly process the idiosyncratic information. Summary of the Invention [Problem to be solved by the invention]

[0006] The technical problem to be solved by the present invention is to provide a data processing method that can process idiosyncratic information of data to reduce the possibility of data re-identification while eliminating or minimizing the need for expert intervention. [Means for solving the problem]

[0007] In order to solve the above technical problems, a data processing method for reducing data re-identification possibility, which is executed by a computer according to the present invention, includes the steps of: determining a frequency of each category for each column of a dataset to be processed; determining a category whose frequency for each category is smaller than a predetermined threshold value of peculiar information as peculiar information; determining a number of peculiar information cases for each row of the dataset to be processed, the number of peculiar information cases being the number of data values ​​determined to be peculiar information in a corresponding row; and determining a peculiar information processing technique for each row according to the number of peculiar information cases determined for each row.

[0008] The differential information processing techniques may include one or more of value replacement, local deletion, and line deletion.

[0009] The value replacement may be a replacement of a data value determined to be anomalous information with another predetermined value.

[0010] The local deletion may involve replacing data values ​​determined to be anomalous with null values.

[0011] The row deletion may involve deleting an entire row from the data set to be processed.

[0012] The peculiar information processing technique may be determined as value replacement when the number of peculiar information items is greater than or equal to α and less than β, as local deletion when the number of peculiar information items is greater than or equal to β and less than γ, and as row deletion when the number of peculiar information items is greater than or equal to γ.

[0013] Here, α, β, and γ are positive integers, and α<β<γ.

[0014] The method may further include recommending a specific information processing technique for each of the rows.

[0015] The method may further include processing the target data set using a specific information processing technique determined for each row.

[0016] A threshold value of the unique information may be determined for each column of the data set to be processed.

[0017] In order to solve the above technical problem, an embodiment of the present invention may include a computer-readable recording medium having a program recorded thereon for executing the data processing method on a computer. In order to solve the above technical problems, a computing device according to an embodiment of the present invention includes a processor and a memory that stores instructions or programs executable by the processor, and can perform the data processing method when the instructions or programs are executed by the processor. Effect of the Invention

[0018] The present invention has the advantage of being able to process idiosyncratic information of data to reduce the possibility of data re-identification while eliminating or minimizing the need for expert intervention. [Brief description of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating a schematic configuration of a computing device according to an embodiment of the present invention. [Diagram 2] 1 is a flowchart provided to explain a data processing method for reducing data re-identifiability according to one embodiment of the present invention. [Diagram 3] An example of a pseudonymized dataset is shown. [Figure 4] FIG. 3 shows an example of frequency counts by category for the occupation column from among the columns of the data set. [Diagram 5] 13 illustrates an example of determining peculiar information based on frequency of each category according to an embodiment of the present invention. [Figure 6] 11 is a diagram illustrating an example of calculating the number of unique information items for each row according to an embodiment of the present invention. [Figure 7] 1 is a diagram illustrating a unique information processing technique according to an embodiment of the present invention; [Figure 8] 1 is a diagram illustrating an example in which a unique information processing technique for each row is determined according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Now, the preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily embody the present invention.

[0021] The terms used in the present specification are for the purpose of describing the embodiments and are not intended to limit the present invention. In the present specification, the singular form includes the plural form unless otherwise specified in the phrase. The terms "comprises" and / or "comprising" used in the specification do not exclude the presence or addition of one or more other components other than the components mentioned. The same reference numerals throughout the specification refer to the same components, and "and / or" includes each and every combination of one or more of the components mentioned. Although "first", "second", etc. are used to describe various components, it is understood that these components are not limited by these terms. These terms are used merely to distinguish one component from another component. Therefore, it is understood that the first component mentioned below may also be the second component within the technical concept of the present invention.

[0022] In this specification, the term "computing device" includes all kinds of devices that can perform computation and provide a result to a user. For example, computing devices include not only desktop PCs, notebook computers, and server computers, but also smart phones, tablet PCs, cellular phones, PCS (Personal Communication Service) phones, synchronous / asynchronous IMT-2000 (International Mobile Telecommunication-2000) mobile terminals, Palm Personal Computers, and PDAs (Personal Digital Assistants).

[0023] FIG. 1 is a diagram illustrating a configuration of a computing device according to an embodiment of the present invention.

[0024] With reference to FIG. 1, a computing device 100 may include a memory 110 and a processor 120 .

[0025] The memory 110 may store at least one instruction and / or program, and may also store data used for various operations related to data processing methods for reducing the likelihood of data being re-identified in the computing device 100.

[0026] The processor 120 may execute instructions and / or computer programs stored in the memory 110 to perform data processing methods for reducing data re-identifiability on the computing device 100 .

[0027] Although FIG. 1 illustrates one memory 110 and one processor 120 for ease of illustration, the computing device 100 may include one or more memories 110 and one or more processors 120 .

[0028] FIG. 2 is a flow chart provided to explain a data processing method for reducing data re-identifiability according to one embodiment of the present invention.

[0029] 1 and 2, the computing device 100 may first perform an operation of calculating a frequency of occurrence of each category for each column of a data set to be processed (S210).

[0030] Here, the dataset to be processed may be a pseudonymized dataset consisting of personal information whose identifiability has been reduced by deleting or replacing information that can identify individuals, such as names and phone numbers (pseudonymization).

[0031] Figure 3 shows an example of a pseudonymized dataset.

[0032] The computing device 100 can compile the frequency of occurrence by category for each column of the age group, sex, residential area, occupation, contact information, membership rank, etc. of the data set to be processed as shown in FIG.

[0033] FIG. 4 shows an example of frequency counts by category for the occupation column from among the columns of the data set shown in FIG.

[0034] Specifically, Figure 4 shows an example in which the frequency of "singer" is "11", the frequency of "firefighter" is "45", ..., the frequency of "engineer" is "4268", and the frequency of "sales representative" is "16201" for the "occupation" column of the dataset shown in Figure 3.

[0035] Thereafter, the computing device 100 may determine that a category having a frequency smaller than a predetermined threshold value of peculiar information is peculiar information (S220).

[0036] FIG. 5 illustrates an example of determining peculiar information based on frequency of occurrence for each category according to an embodiment of the present invention.

[0037] Referring to FIG. 5, for example, in the “Occupation” column, the frequency of “singer” is “11”, and the frequency of “firefighter” is “45”, both of which are less than the critical value of peculiar information of “50”. Therefore, in the “Occupation” column, “singer” and “firefighter” can be determined as peculiar information.

[0038] The critical value of the idiosyncratic information may be preset by a user before step S220 or may use a predetermined default value, or may be set for each column of the data set to be processed.

[0039] Next, the computing device 100 may obtain the number of unique information items for each row (record) of the data set to be processed (S230). Specifically, the number of unique information items obtained for each row in step S230 is the number of data values ​​determined to be unique information in the corresponding row.

[0040] FIG. 6 is a diagram illustrating an example of calculating the number of unique information items for each row according to an embodiment of the present invention.

[0041] In Fig. 6, "Outlier" indicates a data value that is determined to be peculiar information. For example, in the examples shown in Fig. 3 and Fig. 4, the data values ​​corresponding to "singer" and "firefighter" in the "occupation" column are determined to be peculiar information, and therefore correspond to "outliers" as shown in Fig. 6.

[0042] In Figure 6, the "Number of peculiar information" in the rightmost column indicates the number of peculiar information items tallied for each row. For example, in the row corresponding to Index 4, there is one "Outlier" in column 3, so the number of peculiar information items is "1."

[0043] Next, the computing device 100 may determine a specific information processing technique for each record according to the number of unique information items found for each record (S240).

[0044] The peculiar information processing techniques may include value replacement, local deletion, and row deletion. Here, value replacement refers to replacing a data value determined to be peculiar information with another predetermined value. Local deletion refers to replacing a data value determined to be peculiar information with a null value. Row deletion refers to deleting an entire row from a data set to be processed.

[0045] The computing device 100 may compare the number of distinct pieces of information obtained for each row with the variables α, β, and γ to determine a distinct piece of information processing technique.

[0046] FIG. 7 is a diagram illustrating a method for processing unique information according to an embodiment of the present invention.

[0047] As shown in FIG. 7, if the number of unique information items found for the corresponding row is less than α, no processing is required. If the number of unique information items found for the corresponding row is equal to or greater than α and less than β, the unique information processing technique can be determined as value replacement. On the other hand, if the number of unique information items found for the corresponding row is equal to or greater than β and less than γ, local deletion can be determined, and if the number of unique information items is equal to or greater than γ, row deletion can be determined. Here, α, β, and γ are positive integers, and α<β<γ.

[0048] FIG. 8 is a diagram illustrating an example in which a unique information processing technique for each row is determined according to an embodiment of the present invention.

[0049] Figure 8 shows the results when α=1, β=2, γ=3, and the processing technique is determined based on the number of peculiar information items found for each row in Figure 6. For example, the row corresponding to Index 4 has the number of peculiar information items of "1", so it can be seen that "value replacement" was selected as the processing technique. And, the row corresponding to Index 4 has the number of peculiar information items of "4", which is greater than γ=3, so it can be seen that "row deletion" was selected as the processing technique.

[0050] Further referring to FIG. 2, the computing device 100 may recommend to the user a specific information processing technique that has been determined for each line in advance, for example, by displaying the technique on a computer screen (S250).

[0051] In step S250, the computing device 100 may allow the user to confirm whether to process the target data set using the idiosyncratic information processing technique recommended for each row. If the idiosyncratic information processing technique is value replacement, the computing device 100 may be executed so that the user selects a value to be replaced or a criterion for determining a value to be replaced. For example, the computing device 100 may select one of a method of replacing idiosyncratic information with a category with the highest frequency in the corresponding column and a method of replacing idiosyncratic information with a category with the lowest frequency that is not idiosyncratic information. Of course, it is also possible to present other methods in addition to the value replacement methods exemplified here and allow the user to select. Also, the computing device 100 may determine a data value to replace the idiosyncratic information according to a predetermined criterion and replace the idiosyncratic information according to the data value.

[0052] Finally, the computing device 100 may process the data set to be processed using a specific information processing technique determined for each row (S260).

[0053] Depending on the embodiment, step S250 may be omitted, in which case the computing device 100 may automatically process the target data set according to a specific information processing technique determined for each row in step S240.

[0054] Of course, depending on the embodiment, it is possible to perform the step of recommending a specific information processing technique for each row (S250) and omit the step of processing the target data set using the specific information processing technique (S260).

[0055] On the other hand, in the case of numeric data, each data value can be considered as a category and the method described above can be applied.

[0056] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computing devices, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, the processing device may be described as being one in which one is used, but those skilled in the art will appreciate that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

[0057] The software may include a computer program, code, instructions, or a combination of one or more of these, and may configure or instruct the processing device to operate as desired, either independently or collectively. The software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual device, computer storage medium, or device to be analyzed by the processing device or to provide instructions or data to the processing device. The software may also be distributed over network-coupled computer systems, and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0058] The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer readable medium. The computer readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions recorded on the medium may be those specifically designed and constructed for the embodiment, or may be those that are well known and available to those skilled in the art of computer software. Examples of computer readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code, such as that produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, and the like. The above hardware devices may be configured to operate as one or more software modules to perform operations of the embodiment, and vice versa.

[0059] As described above, even if the embodiments are described by limited drawings, a person having ordinary skill in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be counterbalanced or replaced by other components or equivalents, and still achieve appropriate results.

Claims

1. 1. A data processing method for reducing data re-identifiability executed on a computing device including a processor and a memory storing instructions or programs executable by said processor, comprising: The computing device determines a frequency of each category for each column of the data set to be processed; The computing device determines that a category having a frequency count smaller than a predetermined threshold value of peculiar information is peculiar information; The computing device calculates a number of unique information items for each row of the data set to be processed, the number of unique information items being the number of data values ​​determined to be unique information in the corresponding row; the computing device determines a specific information processing technique for each row according to the number of unique information items obtained for each row; said computing device processing each row of said data set to be processed using a specific information processing technique determined for each row; 13. A data processing method comprising:

2. The singular information processing technique includes: Includes one or more of value replacement, local deletion and line deletion, The value replacement is a process of replacing a data value determined to be anomalous information with another predetermined value, The local deletion is to replace a data value determined to be anomalous information with a null value (null); 2. The data processing method according to claim 1, wherein the row deletion comprises deleting an entire row from the data set to be processed.

3. The specific information processing technique determined for each row is as follows: For rows where the number of unique information items is greater than or equal to α and less than β, value replacement is performed. For rows with idiosyncratic information counts greater than or equal to β and less than γ, local deletion is performed. If the number of peculiar information items is γ or more, the row is deleted.

3. The data processing method according to claim 2, wherein α, β, and γ are positive integers, and α<β<γ.

4. 4. The data processing method of claim 3, further comprising the step of: the computing device recommending the specific information processing technique determined for each row by displaying it on a screen of the computing device.

5. 2. The data processing method of claim 1, further comprising determining a threshold value of the idiosyncratic information for each column of the data set to be processed.

6. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the data processing method according to any one of claims 1 to 5.

7. 1. A computing device comprising: A processor; a memory storing instructions or programs executable by said processor; A computing device, the instructions or programs being executed by the processor to perform the data processing method of any one of claims 1 to 5.

Images