Method and system for detecting and classifying confidential data

Abstract

A method for eliminating false positives in a system for detecting and classifying confidential data includes the steps (100) of: receiving a set of structured documents (110); searching for confidential data in a document and categorizing said data using a machine learning model (120); forming a first and a second set of confidential data, containing correctly and falsely identified confidential data types (130); determining, on the basis of the second set of data, values for the possible inclusion of a confidential data type (140); determining, on the basis of a possible inclusion value, threshold possible confidential data type inclusion values (150); saving the threshold possible confidential data type inclusion values in a machine learning model (160); adjusting the parameters of the model according to the threshold values (170). The claimed method makes it possible to increase the accuracy with which confidential data in structured documents are detected and classified.

Description

METHOD AND SYSTEM FOR IDENTIFYING AND CLASSIFYING CONFIDENTIAL DATA AREA OF TECHNOLOGY

[0001] The claimed technical solution generally relates to the field of data protection, and in particular to a method and system for identifying and classifying confidential data in structured documents. LEVEL OF TECHNOLOGY

[0002] Currently, large enterprises store, process and transmit huge amounts of data, including confidential data used in the course of the enterprise's operation. Such confidential data may be restricted access information, which includes, in particular, personal data of clients and employees, information constituting a banking and commercial secret. At the same time, enterprises are obliged to ensure the complete safety and security of confidential data in order to avoid leaks and ensure the information security of the enterprise as a whole, for example, in accordance with the Federal Law "On Personal Data", the Federal Law "On Banks and Banking Activities", the Federal Law "On Commercial Secrets", etc., which impose restrictions on the transfer and methods of storing such data.In addition, since most enterprises operate in the cloud computing environment and the volume of big data increases, enterprises are faced with the problem of unintentional leakage of confidential data, such as due to the transmission of confidential data through open channels, etc., and therefore have difficulty in taking appropriate measures to protect confidential data.

[0003] Thus, there is a need in organizations to identify restricted information and classify it (information categorization) in order to prevent unintentional transfer and meet storage requirements for such data depending on the category of confidential data.

[0004] There are a number of ways to identify sensitive data in organizations, such as using artificial intelligence algorithms. The data is presented in the form of files containing data in a tabular format (structured data). This format is characterized by a huge number of values ​​at the intersections of rows and columns. Based on these algorithms, it is possible to implement categorization of each specific value and, thus, the entire file.

[0005] The disadvantages of such methods include low accuracy of categorizing confidential data, and, as a consequence, low accuracy of identifying confidential data in files, since due to the data format (structured data), the probability of incorrectly categorizing the entire file is extremely high even with incorrect categorization of one value in the file. For example, if we assume that the probabilities of categorizing values ​​are the same and equal to 0.999, then when calculating the probability of an error in a table consisting of 10 columns and 1000 rows, the value approaches 1 and is calculated as P егг = 1 - 0.999 10000 . Thus, when categorizing files containing a huge number of values ​​in a tabular format, the probability of incorrectly categorizing the entire file approaches 1, i.e. it is almost impossible to correctly categorize the file.

[0006] In addition, a solution disclosed in US Patent No. US 8635691 B2 (VOGEL DJ [US] et al.), published on 21.01.2014, is known from the prior art. This solution, in particular, discloses a method for identifying confidential data in documents and performing subsequent data verification to eliminate false positives when identifying confidential data, wherein the elimination of false positives is carried out by comparing the collected data with templates of false positive confidential data.

[0007] The disadvantages of this solution are the low accuracy of eliminating false positives when identifying confidential data due to the strong use of templates, which obviously cannot physically cover all possible types of confidential and falsely confidential data. Also, this solution is not designed to work with structured documents containing millions of records. In addition, this solution does not provide the ability to categorize (classify) confidential data, in which eliminating false positives in this way becomes practically impossible.

[0008] Common shortcomings of existing solutions include the lack of an effective way to identify and categorize sensitive data in structured documents, ensuring high accuracy by eliminating false positives in the process of identifying and categorizing. Also, such a solution should ensure high accuracy of document categorization by removing errors in categorizing individual values ​​within the document. In addition, such a solution allows to significantly reduce the volume of manual checks of files containing data in a tabular representation. DISCLOSURE OF INVENTION

[0009] The claimed technical solution proposes a new approach to identifying and classifying (categorizing) confidential data in structured documents. This technical solution uses an algorithm for eliminating false positives, which allows for increased accuracy in classifying and identifying confidential data.

[0010] This solves the technical problem of creating a new and effective way to identify and classify confidential data. [UN] The technical result achieved by solving this problem is an increase in the accuracy of identifying and classifying confidential data in structured documents.

[0012] An additional technical result that emerges from solving the above problem is a reduction in false positives when identifying and classifying confidential data.

[0013] The specified technical results are achieved by implementing a method for eliminating false positives in a system for identifying and classifying confidential data, performed by at least one computing device and containing the steps of: a) obtaining a set of structured documents containing data in tabular form; b) searching for confidential data in a document and categorizing it at least using a machine learning model. training based on a neural network trained on categorized sets of confidential data, during which: • definition of confidential data in each cell of a table of a structured document; • defining a tag corresponding to the type of confidential information for each cell of a column of a structured document; • defining tags corresponding to the type of confidential information for each column of a structured document based on the tags of the cells contained in the column; • forming a categorized confidential data set based on the determined tags; c) forming, based on the data obtained in step b), a first and a second set of confidential data containing correctly and falsely determined types of confidential data; d) determining, based on the second set of data, values ​​of input acceptability of a type of confidential information different from the type of confidential information assigned to the column; e) determining, based on the input acceptability value determined in step d), threshold values ​​of input acceptability of types of confidential information; f) storing threshold values ​​of input acceptability of types of confidential information in a machine learning model of the system for identifying and classifying confidential data; g) adjusting parameters of the machine learning model in accordance with the threshold values.

[0014] In addition, the claimed technical results are achieved by a method for identifying and classifying confidential data in structured documents, performed by at least one computing device, and containing the steps of: a) obtaining at least one structured document containing data in tabular form; b) processing the data obtained in step a) is carried out at least with the help of a machine learning model based on a neural network configured in accordance with the steps of the method according to claim 1, during which the following is carried out: i. determining confidential data in each cell of the table of the structured document; ij. determining a tag corresponding to the type of confidential information for each cell of the column of the structured document; iii. assigning a tag corresponding to the type of confidential information to columns based on the tags for the cells of this column; iv. comparing, in each column, the content of tags with the type of confidential information with a threshold value for each such tag corresponding to the type of confidential data; i.determining at least one column tag based on the results of comparison with the threshold value obtained in step iv; c) performing classification of the structured document based on the comparison of the assigned tags of each column of the document; d) sending the classified document to the security system for transmission over communication channels in accordance with security requirements.

[0015] In addition, the stated technical results are achieved through a false alarm elimination system containing: • at least one processor; • at least one memory connected to the processor, which contains machine-readable instructions that, when executed by at least one processor, ensure the execution of the method for eliminating false positives.

[0016] In addition, the stated technical results are achieved through a system for identifying and classifying confidential data in structured documents, containing: • at least one processor; • at least one memory connected to the processor that contains machine-readable instructions that, when executed by at least one processor, provide for the execution of a method for identifying and classifying confidential data in structured documents. BRIEF DESCRIPTION OF DRAWINGS

[0017] The features and advantages of the present invention will become apparent from the following detailed description of the invention and the accompanying drawings.

[0018] Fig. 1 illustrates a block diagram of the implementation of a method for eliminating false positives in a system for identifying and classifying confidential data.

[0019] Fig. 2 illustrates a flow chart of the method for identifying and classifying confidential data.

[0020] Fig. 3 illustrates an example of the general appearance of a computing device that ensures the implementation of the claimed solution. IMPLEMENTATION OF THE INVENTION

[0021] Below we will describe the concepts and terms necessary for understanding this technical solution.

[0022] A model in machine learning (ML) is a set of artificial intelligence methods, the characteristic feature of which is not a direct solution to a problem, but learning in the process of applying solutions to many similar problems.

[0023] The declared technical solution offers a new approach that ensures increased accuracy of detection and classification of confidential data, consisting of eliminating false positives (removing errors in categorizing individual values) in the classification process. One of the features of the declared technical solution is the ability to categorize files containing data in tabular form with high accuracy.

[0024] The claimed technical solution can be implemented on a computer, in the form of an automated information system (AIS) or a machine-readable medium containing instructions for performing the above-mentioned method.

[0025] The technical solution can also be implemented as a distributed computer system or computing device.

[0026] In this solution, the term “system” refers to a computer system, a computer (electronic computer), a numerical control (CNC), a PLC (programmable logic controller), computerized control systems, and any other devices capable of performing a given, clearly defined sequence of computing operations (actions, instructions).

[0027] A command processing unit is an electronic unit or integrated circuit (microprocessor) that executes machine instructions (programs) /

[0028] The command processing unit reads and executes machine instructions (programs) from one or more data storage devices, such as random access memory (RAM) and / or read-only memory (ROM). ROM may include, but is not limited to, hard disk drives (HDD), flash memory, solid-state drives (SSD), optical storage media (CD, DVD, BD, MD, etc.), etc.

[0029] A program is a sequence of instructions intended for execution by a computer's control unit or command processing device.

[0030] The term "instructions" as used in this application may refer generally to software instructions or software commands that are written in a given programming language to implement a specific function, such as, for example, encoding and decoding texts, filtering, ranking, translating texts into a dialog system, etc. The instructions may be implemented in a variety of ways, including, for example, object-oriented methods. For example, the instructions may be implemented by means of a programming language, C++, Java, Python, various libraries (e.g., MFC; Microsoft Foundation Classes), etc. The instructions that implement the processes described in this solution may be transmitted both via wired and wireless data transmission channels, such as Wi-Fi, Bluetooth, USB, WLAN, LAN, etc.

[0031] Fig. 1 shows a block diagram of a method 100 for eliminating false positives in a system for identifying and classifying confidential data, which is disclosed step by step in more detail below. Said method 100 consists of performing steps aimed at processing various digital data. The processing is typically performed by a system, which may represent, for example, a server, a computer, a mobile device, a computing device, etc. The elements of the system are disclosed in more detail in Fig. 3. In one particular embodiment, the method 100 may be performed by a system 300, which is disclosed in more detail below.

[0032] The term confidential data (sensitive data) in this solution shall be understood as data, access to which is restricted in accordance with the security policies of organizations and / or legislative acts. Thus, confidential data may include personal data, information constituting a banking and commercial secret, full names of employees and clients, partners and suppliers, addresses, telephone numbers, email addresses, social security numbers, bank card information, TIN number, vehicle registration number, BIC number, IP address, geolocation data, marriage document number, education document number, date, URL address, MAC address, work book number, military ID number, OKPO code, etc., without limitation.

[0033] Elimination of false positives (removal of errors in classification of individual values) should be understood as - erroneous and / or incorrect determination of the category of confidential data or assignment / non-assignment of confidential / non-confidential data to any category of confidential. The appearance of false positives in the process of classification of confidential data can lead to both leaks of particularly critical confidential information, for example, during further transmission of data via open channels, and to information security incidents.

[0034] Classification of confidential data means assigning a tag / tags (a short string that mutually corresponds to types of confidential information) to confidential data from a predefined set of tags. Tags unambiguously describe the relationship of confidential data to a certain type, for example, the tag CARD - card number, NAME - name, etc. Tags are written in Latin so that they have a common appearance in all encodings. Types of confidential information fall into one of the categories of legally regulated data, for example, personal data, banking secrets, commercial secrets, etc. In addition, classification of confidential data can be carried out by criticality confidential data, so confidential data can be divided into several categories that characterize the requirements for their distribution and processing (the most critical are card numbers, full names, critical are telephone numbers, less critical are postcodes, etc.).

[0035] At step 110, at least one structured document containing data in tabular form is obtained.

[0036] At the specified step 110, the system for identifying and classifying confidential data receives at least one document, for example, in the form of a data file containing data in a table representation. The data in the table representation stores structured data in the form of columns, each of which stores more or less homogeneous information (the same type of information). For example, one column contains data about the client's last name, and another column contains his telephone number. The data file can be received via open data transmission channels, for example, a LAN (local area network). In addition, the data file can be loaded into the system, for example, the system 300 for identifying and classifying confidential data via input / output interfaces, for example, USB.

[0037] At step 120, the document is searched for and classified for confidential data.

[0038] At the specified step 120, at the first step, the presence of confidential data is determined in each cell of the table of the structured document. At the second step, a tag corresponding to the type of confidential information is determined for each cell of the column of the structured document. At the third step, a tag corresponding to the type of confidential information is determined for each column of the structured document based on the tags of the cells contained in the column. At the fourth step, a categorized confidential data set is formed based on the determined tags.

[0039] Thus, in one particular embodiment, search algorithms based on rules and regular expressions are used. In another particular embodiment, the search for sensitive data is performed using a machine learning model based on a neural network trained on categorized sets of sensitive data.

[0040] Thus, continuing the particular implementation option, a detection and classification model was developed and trained as a machine learning model confidential data. The model is designed to solve the text classification problem (NER problem). Named-entity recognition (NER) is a subtask of information extraction that aims to find and classify mentions of named entities into predefined categories, such as names of people, organizations, locations, monetary values, percentages, etc.

[0041] The ML model was trained on pre-labeled data. At the time of model creation, a dataset of labeled confidential data consisting of over a million tokens was used. The dataset was formed based on the original values ​​of tabular data containing personal data, information constituting a banking and commercial secret, etc. Training the model to recognize confidential data in structured documents consisted of identifying confidential data and classifying such data by category (full name, bank card number, social security number, etc.). The following schemes can be used for data labeling: the VYu / YuV scheme, the BILUO scheme, etc., without limitation. The dataset can be represented, for example, as an ordered list of tokens separated by a separator from the class of the named entity, for example, a space character.It is obvious to a specialist in this field of technology that any dataset markup and presentation scheme known from the state of the art can be applied, and this solution is not limited to the examples given above. 80% of the tokens from the dataset were used to train the ML model, and 20% were used to calculate quality metrics.

[0042] The main metric of the ML model quality, the class-weighted F1 measure, is about 97.9%, which exceeds the results of automatic machine learning (AutoML) by more than 15%. At the same time, the spread between recall and precision does not exceed 0.8%, which indicates a high level of model quality both from the point of view of security - low probability of missing confidential information, and from the point of view of the end user - a small number of false detections of confidential information. The degree of drop in the quality metric when augmenting texts at the character level, showing the stability of the model, does not exceed 5%.

[0043] The specified neural network-based machine learning model has been successfully implemented and tested in the organization in the work processes of departments.

[0044] In one particular embodiment, the neural network may be, for example, a Transformer architecture neural network, a recurrent neural network (RNN), etc., without limitation.

[0045] Search and classification of sensitive data in a structured document using a machine learning model can be performed by tokenization, vectorization and processing of the specified data. Thus, the input data is tokenized, i.e. segmented into parts, for example, into words or symbols. In this solution, a token should be understood as a sequence of symbols in a document that is important for analysis. Thus, tokens can be, for example, individual words, parts of words, etc. Tokenization into sentences can be carried out using, for example, lexical analyzers such as razdel, rusenttokenize, NLTK, etc. Lexical analysis is the process of analytically parsing the input sequence of symbols into recognized groups (lexemes) in order to obtain identified sequences called "tokens" at the output. In addition, tokenization of the input text can be carried out based on regular expressions.It is obvious to a person skilled in the art that any lexical analyzer known from the prior art can be used and this solution is not limited to the examples given above.

[0046] Next, vectorization of each token obtained in the tokenization process is performed, for example, using embeddings or direct encoding (one hot encoding). For example, during tokenization, each token is represented in the dictionary by its index, which displays the position in the specified dictionary. Thus, each token represents an index in the dictionary, and, accordingly, the vectorization process is carried out by replacing each token with its index in the dictionary. Then, the indices are grouped taking into account the sparseness of the dictionary and the semantic proximity of the tokens. It will be obvious to a specialist in this field of technology that other vectorization algorithms can be used for token vectorization, for example, using the TransformersBertEmbedder, Word2vec, fastText algorithms, etc., without limitation. The specified vectorization process is a preparatory stage for data processing by a machine learning model that identifies and classifies confidential data.

[0047] Thus, at step 120, for example, by means of a machine learning model trained on categorized sets of confidential data, a search is performed for confidential data in cells, a tag is assigned to each cell, a tag is assigned to a column of cells based on the tags of the cells, and a set of classified confidential data is generated. An example of classification of a structured document containing 17 columns is given in Table 1. Table 1

[0048] Thus, in the specified embodiment, the columns marked with tags - conf tag represent a category of confidential data, related either to numeric confidential data or to named ones, for example, O or NUM. It is worth noting that the category O is a category of named entities, for example, Full Name, Address, Place of Residence, etc., and the category NUM represents numeric categories of confidential information, for example, Taxpayer Identification Number, card number, etc. In another particular embodiment, conf tag can represent a specific category of confidential information, for example, NAME, INN, CARD, etc. Obviously, instead of the two tags presented in As an example, in the implementation of this technical solution, the classification of confidential data can be carried out directly by multiple categories. The above example is given to describe the general operating principle.

[0049] In this case, the value defined for each conf tag represents the percentage of each type of confidential information included in the column from the total value (for NUM, only the numeric values ​​of the column will be taken into account, etc.).

[0050] The column marked with the True tag represents the category of confidential data assigned to the column in question (column 1 - column 17, etc.)

[0051] In this way, a set of classified confidential data is formed.

[0052] Method 100 then proceeds to step 130.

[0053] At step 130, based on the data obtained at step 120, a first and second set of confidential data is formed, containing correctly and falsely determined types of confidential data.

[0054] Since when categorizing values ​​in one column, we can find out that most of the values ​​belong to one type and, at the same time, there is a certain number of values ​​that belong to another type, then with standard classification methods, the specified column will be assigned an incorrect class (for example, by the type of confidential data that is lower in criticality). This can lead to a leak of critical confidential data. In addition, in practice, there are columns containing different lengths and / or different types (for example, part of the column contains text data, and part - numeric, also, part can be a mixed set of characters, consisting of both numbers and letters). Also, the appearance of incorrectly classified confidential data can lead to a complete match of the expression structure (numeric or mixed).The specified feature is typical when searching for confidential information using regular expressions and methods based on them (for example, calculating checksums). In addition, another problem when searching and classifying confidential data using machine learning algorithms may relate to similarly written expressions. When analyzing similarly written text expressions (for example, morphological similarity, register, etc.), similar words may be. are assigned to incorrect categories. Since the information is presented in a structured form, each entry in the table is an independent expression that has no context on the right or left, so it is impossible to train models to more accurately determine the type of confidential information (for example, the similarity of addresses, namely street names and surnames).

[0055] All of the above features are related to false positives. To eliminate these errors in classifying confidential data, this technical solution offers a new approach based on defining threshold values ​​that characterize the level at which it is considered that the type of information of some values ​​is determined falsely in the analyzed column.

[0056] Thus, to determine the threshold values, the data obtained in step 120 is divided into values ​​that relate to correctly defined tags and those that relate to false positives of the AI ​​model and regular expressions.

[0057] To classify a value as a false or true result, a system for automatically checking classification results can be used, implemented on the basis of a computing device, as shown in Fig. 3.

[0058] For example, the obtained results are sent for verification, during which the obtained results are compared with the reference results for the document being processed. For example, a pre-labeled data array can be used as reference results. Accordingly, after the classification of confidential data, the specified results are compared with the reference results. In addition, the types of information and statistical values ​​for them, which are defined correctly for the column, from the point of view of the operation of the AI ​​model and regular expressions, are in a row with true values. The remaining tags and values ​​for them are in a row with false results.

[0059] Next, at step 140, based on the second set of data containing false positive results, values ​​of the acceptability of an input of a type of confidential information different from the type of confidential information assigned to the column are determined.

[0060] To do this, an analysis of a series of false positives is performed.

[0061] Hypotheses about the normality of the distribution of the series across all analyzed columns are tested using the following tests:

[0062] The Shapiro-Wilco test, which is suitable for relatively small samples (less than or equal to 1000 observations). The null hypothesis of this criterion is that the random variable is normally distributed. The statistics of the criterion are as follows: , the coefficient a n-i+1 is tabular. If the calculated statistical value W is less than the tabulated value for a given significance level (in our case, 5%), then the null hypothesis of normal distribution is rejected.

[0063] The D'Agostino test, based on the kurtosis and skewness transformations, shows how much the distribution deviates from normal. As in the previous test, the null hypothesis assumes that the random variable belongs to the normal distribution. If the obtained p-value is small relative to the selected significance level (5%), the data most likely do not belong to the normal distribution. The criterion value is calculated as follows: K vatue = Z value 2 kew + Z value kurtosis , i.e. it is a combination of transformed values ​​of excess and asymmetry.

[0064] The Anderson-Darling test, which is most often used to detect deviations from normality of the maximum and minimum values ​​of a distribution. The value of the statistic is calculated as follows: critical value the criterion is tabular (a2 S* ), the null hypothesis is not rejected if for the obtained values ​​P{S > S*} = 1 - a2(S*) > a. This criterion is calculated for 5 levels of significance (a = [15, 10, 5, 2.5, 1]).

[0065] Thus, at step 140, the values ​​of the acceptability of the input of a confidential information type different from the confidential information type assigned to the column are determined.

[0066] Method 100 then proceeds to step 150.

[0067] At step 150, threshold values ​​of entry acceptability of types of confidential information are determined based on the entry acceptability value determined at step 140.

[0068] The results of the tests are used to decide whether the series is normally distributed. The following combinations of normality test results are used for samples:

[0069] For small samples (less than or equal to 1000 observations), the Shapiro-Wilco test has the greatest weight, the general combination takes the form:

[0070] For large samples (more than 1000 observations), the criteria that evaluate the similarity of the distribution to the normal one have greater weight, based on the deviations of the maximum and minimum values, the general combination takes the form: r = 0. IfW + 0.15 fK value + 0.15 / 5 П15 + 0.15 / 5 П10 + 0.15 / 5 П5 + 0.15 / 5 П2 5 + 0.15 / 5 ftl

[0071] In both combinations, f followed by the criteria designation is a binary value, where 1 means that the null hypothesis of the "normality" of the random variable distribution is not rejected. The maximum value of the criteria combination is 1, if all tests have shown that the distribution can be considered normal, in which case the threshold statistical value is taken to be equal to the 80th percentile for erroneous class definitions. The minimum value is 0, characterizing the case when all tests for similarity to the normal distribution have shown that the null hypothesis is rejected, in which case the threshold value is taken to be equal to the 99th percentile, based on the assumption that the minimum possible number of calculated values ​​for the categorized columns exceeds the permissible threshold for false positives in categorizing the column cells.

[0072] If the combination takes a value between 0 and 1, the threshold value is determined according to the following scheme - statistical values ​​are calculated: mode, median, mean - if these 3 values ​​are equal, then it is assigned as the threshold - if the mode is not zero and is not equal to the median or mean, then it is recommended to take the mode as the threshold value - in all other cases, the 99th percentile value is set as the threshold

[0073] Once the threshold recommendations are defined, a series of true values ​​is taken for analysis. The minimum true value for the series being analyzed is determined.

[0074] If the minimum value is greater than the recommended threshold, a cycle of decreasing the recommended value is started based on percentile values ​​with a decreasing step of 0.1.

[0075] Next, at step 160, threshold values ​​for the admissibility of entering types of confidential information into the machine learning model of the system for identifying and classifying confidential data are stored.

[0076] The obtained threshold values ​​are then stored in the machine learning model. The specified step 160 can be performed by providing a file with threshold values ​​to the machine learning model.

[0077] Next, at step 170, the parameters of the machine learning model are adjusted in accordance with the threshold values.

[0078] At said step 170, the determined threshold values ​​are then set into a machine learning model that performs classification and detection of sensitive data.

[0079] Due to certain threshold values, false positives are excluded during the classification of confidential data, which, accordingly, significantly increases the security of data in the organization.

[0080] Next, we will consider an example of implementing method 100.

[0081] Let's consider an example with 10,000 observations, they can be represented as 10 columns of 1000 independent observations. This means that the probabilities of events A егг (wrong) and A true (correctly) determine the classes of observations in the column are: PA err ) = 1 - 0.999 1000 = 0.63, in addition to this P(A t rue) = O.999 1000 = 0.37.

[0082] Further, if we introduce the assumption of homogeneity of observations in the column, which is satisfied when the calculated threshold value exceeds the actual value of the proportion of classes relative to the entire column, it becomes possible to increase the probability of correctly identifying classes in the analyzed column. Also, in the example considered below, we introduce the assumption that the minimum true value of falsely identified types of confidential information is 0. Therefore, the percentile value is not reduced.

[0083] Let's assume that the distribution of accumulated false values ​​by classes in the column is normal, then the threshold is taken as the 99th percentile value, which means that in 1% of the analyzed columns the actual value of the proportion of classes exceeded the threshold, which means that the assumption of data homogeneity is rejected and the calculation of the probability of correct / false definition of classes in the column coincides with the example above and is equal to: Pb2(A егг = 0.63 and Pb2(Acre) — 0.37. In the remaining 99% of cases, the probability corresponds to the following values: = 0.001, P B1 ^true) = 0.999

[0084] In this case, the probability of correctly categorizing the entire column of the table is calculated as follows: 0.00162 0.99271, where P(Bi) = 0.99, and P(B2) = 0.1, i.e. B1 and B2 are the events of satisfaction and rejection of the assumption of homogeneity of data in the columns.

[0085] Continuing with the example, the calculation of the threshold value is given directly.

[0086] For example, the following data were obtained as initial data: there were 10 columns, each containing at least 1000 values. Let us consider one of the 10 columns with 1000 observations, which are initially numeric (i.e. it is assumed that the column contains only numeric values, for example, the TIN of individuals). It is worth noting that 1000 observations were chosen for simplicity and convenience of disclosing this example and can be any value, therefore, it cannot limit the present technical solution. When classifying the specified values, for example, as described above, 2 types of confidential information were identified: number (NUM) and TIN of an individual (INNFL).

[0087] In the normal case, the most critical information in the column in question - the INN (related to personal data) will characterize the entire column and affect the category of confidential data for the entire data set. However, when applying threshold values, i.e. implementing the declared technical solution, the system will perform the following actions. Since all values ​​in the column were defined as numeric, the number of cells in which the INN was found will be divided by the number of non-empty cells (that is, the same type as the INN - numeric) in the column under consideration. Let's assume that in the example under consideration this value was equal to 0.123. In this case, in accordance with the steps of method 100, a threshold value of 0.15 was obtained for the category of confidential information INN. Consequently, the 12.3 percent of INNFL tags found during the classification of the values ​​of the specified column belong to the false positive (i.e. incorrectly determined) category and are excluded by the system from those affecting the category of the column and the entire document.

[0088] Next, let's look at the definition of the threshold value using an example: for the INNFL tag, statistics have been accumulated with false positives in the data that were categorized earlier: In addition, statistics have been accumulated with true values ​​for determining the tag for the individual taxpayer identification number in the cells of the column in question:

[0089] To determine the threshold value, as indicated above, the series with false positives is analyzed for normality, for example, using the various tests disclosed in step 140.

[0090] Since the considered series with false positive values ​​contains less than 1000 observations, the Shapiro-Wilco test has the greatest influence on the final decision. According to the available data, x = 0.1 * (0.15 + 0.1 + 0.01 + ••• + 0.13) = 0.12.

[0091] Therefore, normality will be defined as follows: WITH 2 Z to her formula: 10 (Xi - 0.12) 2 = i=l = (0.15 - 0.12) 2 + (0.1 - 0.12) 2 + (0.01 - 0.12) 2 + ••• + (0.13 - 0.12) 2 == 0.07 - 10 2 14.29 * ' n-i+l ^n-1+1 ^i) -i=l = 14.29 * (0.5739 * (0.13 - 0.15) + 0.3291 * (0.12 - 0.1) + 0.2141 * (0.11 - 0.01) + ••• + 0 * (0.15 - 0.13)) 2= 14.29 * 0.00039 = 0.006

[0092] At a 5 percent significance level, the table value of the statistic is 0.842. Since the calculated value is less than the table value (0.006 < 0.8423), the hypothesis of normality of the series is rejected.

[0093] Accordingly, step 140 performs the following normality test, the D'Agostino test. In our example, value = 1-704256, for which p-value = - 0.041, which is less than 0.05, means the data most likely does not have a normal form.

[0094] A check is also made according to the third test, in the example under consideration for all levels of significance a the null hypothesis about the normality of the data is rejected, which means in the combination for low-dimensional data we received the minimum value of 0, in this case the threshold value is taken equal to the 99th percentile and is equal to 0.15.

[0095] Thus, the introduction of threshold values ​​and the assumption of column homogeneity increases the probability of correct column categorization (from 0.37 to 0.9927) by 2.68 times. In addition, the transition to column analysis using threshold values ​​for homogeneous data allowed us to correctly categorize files of the considered structure and satisfying the described conditions with a probability close to 1.

[0096] Thus, for a certain period of time during which statistics are collected, for each of the identified types of information, 2 numerical series are formed: with statistics on false and true results.

[0097] Fig. 2 shows a block diagram of a method 200 for identifying and classifying confidential data in structured documents, which is disclosed step by step in more detail below. The said method 200 consists of performing steps aimed at processing various digital data. The processing is typically performed using a detection and classification system, which can also represent, for example, a server, a computer, a mobile device, a computing device, etc. The elements of the system are disclosed in more detail in Fig. 3

[0098] At step 210, at least one structured document containing data in tabular form is obtained.

[0099] At the specified step 210, the system for classifying and identifying confidential data receives at least one document, for example, in the form of a data file containing data in a table representation. The data in the table representation stores structured data in the form of columns, in each of which more or less homogeneous information (the same type of information) is stored. Thus, the system for identifying and classifying confidential data can be a system 300, which can be, for example, a server, a computing device, etc. Thus, at the step 210, using an interface, for example, a GUI, of the system for classifying and identifying confidential data, a structured document can be loaded, for example, in XLS format. In addition, in another particular embodiment, the document can be obtained through network interaction, for example, via a LAN.Additionally, in another particular embodiment, the structured document may be obtained via an input / output interface.

[0100] At step 220, the data obtained at step 210 is processed at least using a machine learning model based on a neural network configured in accordance with the steps of method 100, during which confidential data is determined in each cell of the table of the structured document; a tag corresponding to the type of confidential information is determined for each cell of the column of the structured document; a tag corresponding to the type of confidential information is assigned to columns based on the tags for the cells of this column; a comparison, in each column, of the content of the tags with the type of confidential information with a threshold value for each such tag corresponding to the type of confidential data; determining at least one column tag based on the results of the comparison with the threshold value.

[0101] Thus, at the specified stage 220, using a machine learning model trained to identify and classify confidential data and configured (modified) in accordance with the obtained threshold values ​​for each class of confidential data, each column in the structured document is categorized, however, as was As stated above, the final class for a column is assigned based on the threshold values ​​for each category of sensitive data contained in that column.

[0102] At step 230, the structured document is classified based on a comparison of the tags assigned to each column of the document.

[0103] At said step 230, based on the data obtained at step 220, a final class is assigned to the entire structured document.

[0104] Thus, in one particular embodiment, each column class is compared by criticality with other column classes, and the document is assigned a class corresponding to the most critical column class. In another particular embodiment, the document classification may contain an indication of specific parts corresponding to different degrees of criticality of the data contained in the columns. Thus, the resulting class may contain a plurality of data sets distributed by criticality.

[0105] At step 240, the classified document is sent to the security system for transmission over communication channels in accordance with security requirements.

[0106] At the specified step 240, the classified document is sent to the security system. The security system may be a security system of an organization that provides rules for the distribution, storage, and transfer of confidential data. For example, after receiving a document containing critical data, the security system may establish and monitor compliance with the requirements for the distribution of such a file. For example, when attempting to transfer a classified file containing critical data to third parties, the security system may prohibit the transfer of such a file and send a notification to the user and / or temporarily block him. In addition, in another particular embodiment, based on the class assigned to the file, security policies may be established that determine the possibility of user access to the specified file.It is obvious to a person skilled in the art that the presence of a class of confidential data allows for the implementation of other procedures known from the prior art that ensure the security of an organization's data.

[0107] Thus, the claimed technical solution discloses a method for identifying and classifying confidential data, ensuring high classification accuracy by eliminating false positives and, as a result, increasing the security of data in the organization.

[0108] Fig. 3 shows an example of a general view of a computing device 300, on the basis of which a system for identifying and classifying confidential data and a system for eliminating false positives are implemented, which ensure the implementation of the declared methods 100 and 200 or is part of a computer system, for example, a server, a personal computer, part of a computing cluster, processing the necessary data for implementing the declared technical solution.

[0109] In general, the device 300 comprises components such as: one or more processors 301, at least one memory 302, data storage means 303, input / output interfaces 304, I / O means 305, network interaction means 306, which are connected via a universal bus. [IT] The processor 301 performs the basic computing operations necessary for processing data when performing the method 100 or the method 200. The processor 301 executes the necessary machine-readable instructions contained in the RAM 502.

[0111] Memory 302 is usually implemented as RAM and contains the necessary software logic to provide the required functionality.

[0112] Data storage means 303 can be implemented in the form of HDD, SSD disks, RAID array, flash memory, optical storage devices (CD, DVD, MD, Blue-Ray disks), etc. Means 303 allow long-term storage of various types of information, for example, threshold values, classified data, etc.

[0113] To organize the operation of the components of the device 300 and to organize the operation of external connected devices, various types of I / O interfaces 304 are used. The choice of the corresponding interfaces depends on the specific design of the computing device, which may be, but are not limited to: PCI, AGP, PS / 2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS / Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0114] The choice of interfaces 304 depends on the specific design of device 300, which can be implemented on the basis of a wide range of devices, for example, personal computer, mainframe, laptop, server cluster, thin client, smartphone, server, etc.

[0115] The following may be used as I / O data means 305: keyboard, joystick, display (touch display), monitor, touch display, touchpad, mouse, light pen, stylus, touch panel, trackball, speakers, microphone, augmented reality means, optical sensors, tablet, light indicators, projector, camera, biometric identification means (retina scanner, fingerprint scanner, voice recognition module), etc.

[0116] Network interaction means 306 are selected from devices that provide network reception and transmission of data, for example, an Ethernet card, a WLAN / Wi-Fi module, a Bluetooth module, a BLE module, an NFC module, IrDa, an RFID module, a GSM modem, etc. With the help of means 305, data exchange is organized between, for example, a device 300, presented in the form of a server, and a user's computing device, on which the received data (classified structured document) can be displayed via a wired or wireless data transmission channel, for example, a WAN, PAN, LAN, Intranet, Internet, WLAN, WMAN or GSM.

[0117] The specific selection of elements of device 300 for the implementation of various software and hardware architectural solutions may vary while maintaining the required functionality provided.

[0118] The submitted application materials disclose preferred examples of the implementation of the technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in the relevant field of technology. Thus, the scope of the present technical solution is limited only by the scope of the attached formula.

Claims

FORMULA 1. A method for eliminating false positives in a system for identifying and classifying confidential data, performed by at least one computing device, and comprising the steps of: • receive a set of structured documents containing data in tabular form; • search for sensitive data in a document and categorize it using at least a machine learning model based on a neural network trained on categorized sets of sensitive data, which includes: • definition of confidential data in each cell of a table of a structured document; • defining a tag corresponding to the type of confidential information for each cell of a column of a structured document; • defining tags corresponding to the type of confidential information for each column of a structured document based on the tags of the cells contained in the column; • formation of a categorized confidential data set based on certain tags; • form, on the basis of the data obtained in step b), the first and second sets of confidential data containing correctly and falsely determined types of confidential data; • determine, based on the second set of data, the values ​​for accepting the entry of a sensitive information type other than the sensitive information type assigned to the column; • determine, based on the entry tolerance value determined in step d), the entry tolerance threshold values ​​of the types of confidential information; • maintain threshold values ​​for the admissibility of entering types of confidential information into the machine learning model of the system for identifying and classifying confidential data; 25 • adjust the parameters of the machine learning model according to the threshold values.

2. A method for identifying and classifying sensitive data in structured documents, which is performed by at least one computing device and comprises the steps of: a) receiving at least one structured document containing data in tabular form; b) processing the data obtained in step a) using at least a machine learning model based on a neural network configured in accordance with the steps of the method according to claim 1, during which the following is performed: i. determining sensitive data in each cell of the table of the structured document; ii. determining a tag corresponding to the type of sensitive information for each cell of a column of the structured document; iii. assigning a tag corresponding to the type of sensitive information to columns based on tags for the cells of this column; iv.comparing, in each column, the content of tags with the type of confidential information with a threshold value for each such tag corresponding to the type of confidential data; ii. determining at least one column tag based on the results of the comparison with the threshold value obtained in step iv; c) classifying the structured document based on the comparison of the tags of each column of the document; d) sending the classified document to the security system for transmission over communication channels in accordance with the security requirements.

3. A false alarm elimination system comprising: at least one processor; at least one memory, coupled to the processor, that contains machine-readable instructions that, when executed, at least one processor ensures the execution of the method according to paragraph 1.

4. A system for identifying and classifying confidential data in structured documents, containing: • at least one processor; • at least one memory connected to the processor, which contains machine-readable instructions that, when executed by at least one processor, ensure the execution of the method according to claim 2.

Images